Its been too long since the EQ team’s last post, mostly because we’ve been incredibly busy working towards getting the first survey live. In this post I’m going to provide some details of what we’ve been up to since starting our Beta in January, some of the decisions we’ve made and what that means for the future of EQ.

Getting Online

We made the decision at the start of Beta to focus on our respondent’s needs, to get our first survey online and securely collect real data as soon as possible. We knew this would tease out a whole raft of issues and also put us into the position of having a ‘production ready’ system to iterate on. Being in this position shouldn’t be undervalued, although it may sound obvious, you really don’t know what is required to get a system into production until you actually manage to achieve it, there are so many unknowns and surprises to be found around every corner, not just in the technical aspects and integration but also across the processes and workflow from how we prepare a survey to go online to how we operationally support the platform once it’s live.

The goods news is that the first survey (usually collected via paper questionnaires) went online on the 26th April and we’ve now had over 90% of the total respondents complete it via our EQ online questionnaire. This is actually much higher than we were expecting for our first online survey, admittedly this survey only went to a small number of the ONS’ total respondents but there’s no denying we’re off to a good start. The survey in question is part of the Retail Sales Index, which is used to provide monthly estimates of total retail sales in Great Britain, a key indicator of the progress of the economy and feeds into results relied upon by the Bank of England and HM Treasury. This survey is run monthly and now it’s live we’ll be continuing to collect the data for it online in future months.

EQ First Live Survey

In reaching this major milestone we did accept some debt, both technical and design based. We’re now spending some of the subsequent weeks paying off those debts, iterating on the design, maturing our solution, and stabilising it operationally before sprinting ahead with significant new capabilities or bringing online additional surveys.

Authoring

At the start of this post I mentioned that our focus was on the respondent and getting the first survey online. For a bit of context I’ll provide a recap; the EQ is split into two major parts, the questionnaire authoring capability and the respondent facing online questionnaire system. These are two separate products that are closely related, one feeds the other. So what does focusing on the respondent mean in this context? It actually means the majority of our work in Beta has been solely on the respondent facing system, we haven’t got the authoring experience up and running yet and it wasn’t required to get the first survey live (the team manually crafted the questionnaire definition that would normally be generated from the authoring system).

Working in this way allowed us to get production ready on the respondent system early (which is arguably the harder part of the EQ) but does have the disadvantage that we have an imbalance in our products now; ideally the authoring experience would be capable of creating questionnaires covering all the capabilities of the respondent system, and as of today this isn’t the case. Obviously this isn’t a sustainable position to be in, the developers can’t be manually cranking out new questionnaires and edits all the time, not only would it be a bottleneck internally it would pull time away from the team’s ability to develop the EQ products that are meant to alleviate this kind of problem.

This is the choice we made at the start of Beta, and we planned to be in this situation so it doesn’t come as a surprise. Now we have the respondent system live it is important we bring our authoring experience in line. To facilitate this we have run another successful Story Map session to gain a shared understanding within the team of our author backlog for Beta, building on the learnings from our Alpha and additional research that has been undertaken since. It is worth mentioning that there are also significant benefits to be gained within the ONS around the authoring workflow across editing, reviewing, testing, QA and publishing of questionnaires that is outside the pure authoring/designing capability that immediately comes to mind when thinking of this product; we’ll probably write something about this in a future post.

Raw EQ Beta Author Story Map

EQ Sandwich

In our Alpha we developed a standalone solution, this was great for rapid prototyping and iterating and meant we had virtually no external dependencies or integration issues to worry about. Naturally this couldn’t remain for Beta, in order to collect real data we needed to integrate with various other teams and products within the ONS. I wanted to highlight those here because without them there would be no online survey.

The EQ actually sits between two other major products and teams within a wider platform, one is the Response and Respondent Management which allows ONS staff to setup their surveys, manage respondents and provides a portal for respondents to enrol, create an account and actually launch the EQ. The second is the Survey Data Exchange, which retrieves the data collected by the EQ, processes it and converts it for processing by the existing ONS statistical systems. These products together form an end-to-end service that is technically enabling the ONS to collect survey data online.

This simplified perspective of the products doesn’t do justice to everyone that is actually involved across the organisation, from the team responsible for front line support and enrolment letters to methodologist and statisticians, survey managers and many others, EQ is part of a much bigger world now.

Survey Data Collection Platform Products and Teams

Summary

In summary we’ve achieved a major milestone in getting our first monthly survey online, which involved multiple teams and products all coming together. We’ve learnt a huge amount in doing so but also incurred some debts along the way which we’re now paying off. In parallel we’ve kicked-off the work on the authoring experience to balance out the two products and create a sustainable service moving forwards.

Finally the EQ team also agreed we should be posting more regular updates to the blog, we’ll do our best to improve on this aspect too!