ONS Digital

The following blog post is written by Jonathan Porton.

One of the biggest issues that we discovered through our user testing was how we refer to things on the website. Specifically, the labelling of the content that make up our releases. When we started to explore the issue further with users it became clear that there was a lack of agreement over how we should refer to this content, and that no amount of usability testing sessions would give us a clear answer. To provide the level of agreement we required we decided to create an online study in the form of a SurveyMonkey survey.

The survey

Rather than have a completely open-ended survey where users could specify an answer, I wanted to created a multiple choice survey with the option to supply their own answers. Through experience, surveys that present the user with a vast number of multiple line text boxes tend to produce poor completion rates. As a user myself it’s quite daunting to know you’re going to have to do all the work and I much prefer clicking radio buttons or selecting dropdown boxes that having to write lots of text in lots of fields.

In order to come up with the right set of multiple choice answers I went back through the videos from the testing sessions and noted whenever the user referred to any of the content types. To supplement these answers, I also ran a pilot internally allowing users to comment on the multiple choices provided and supply their own. Incorporating these responses I then created our final survey that would be distributed to our users.

The survey consisted of 7 questions, each with either 4 or 5 multiple choice answers plus one ‘Other’ field. It was paramount that we not influence the user in anyway so the questions were put forward as a description of the content type ensuring that none of the text in the question be presented in the same way as the answer. E.g.

For instance, had I phrased the question in this way:

As well as our regular statistical releases we also produce data requested by our users. Based on this, which of the following (if any) best describes this data?

By including ‘data’, ’requested’ and ‘users’ in the question it was likely that we would influence the users’ choice to be ‘User Requested Data’ (coincidently users overwhelmingly chose this option anyway!) It was difficult to completely avoid certain terms, particularly around time series data when it is a series and is over time and is very difficult to explain otherwise, but in the main I think I succeeded. Additionally to help with the decision making I provided links to pages on the beta website that were examples of the content we were describing, again with any labelling removed to avoid any potential association by default.

As a team we had major concerns that any changes we would make as a result of the survey would be to the detriment of our experienced users (our Expert Analysts). To give us insight into their needs I simply added questions around usage of the website. We know through our personas that our Expert Analysts use the website either daily or at least once a week. By adding these questions I would be able to segment the responses by our Expert Analysts against all responses and identify any difference of opinion based on prior knowledge of the website. Additionally I asked whether users had experience with the Alpha or Beta websites to again identify any prior knowledge.

The survey was distributed to our database of users, through the Government Email delivery network and promoted on Twitter. It ran for 7 days and received an astonishing 1,113 responses. 840 of those responses were in the first 4 hours of it being sent out.

The analysis

Here are some of the key highlights (you can access the full analysis here):

• 90% of users had previously used the ONS website of which 98% had used the current website, 7% the Alpha and 20% the Beta
• 41% of users visited the current website either daily or weekly
• Labelling for data related content had much clearer agreement than most of our commentary content. Not surprising as terms such as dataset and time series are generally more established in other similar organisations
• It is interesting to note that Expert Analysts completing this survey did not choose ‘statistical bulletin’ despite being very familiar with this term
• In general comments, users urged us to make more use of Plain English – not just in our content labelling, but how we title our statistical releases and how we talk within those releases. Perhaps surprisingly those comments are equally divided amongst experienced and novice users

In summary

We were very keen to resolve the issues with labelling of data on the website as this caused so many issues during testing. Thankfully the direction we should take with these became very clear following the analysis, so we prioritised those changes first and they are reflected on the website now. The labelling around our written content will be looked at in more detail soon. Statistical bulletins are intrinsically part of the makeup of the ONS so we want ensure our key internal stakeholders fully understand the reasons behind making these changes.

The wider issue of making the language on the website more accessible for all is a trickier piece of work. Striking the balance between making it easier to understand without alienating our more experience technical audience. That said, even users who literally spend their lives on the ONS website agree that we can introduce more plain english and come up with more sensible titling so we’ll certainly continue to work on that.