ONS Digital

We’re in week four of our 12-week Alpha, I wanted to provide an update of what’s happened since the end of our Discovery phase, including a peek at the solution we demoed this week during our Sprint Review.

Sprint Zero

The first week of the Alpha was designated “Sprint 0”.

For us, Sprint 0 meant a focus on ‘getting ready’, setting everything up to start delivering value from Sprint 1. Technically this meant preparing our laptops, source control, infrastructure, environments, testing, automation etc. the so called “delivery pipeline”. At the end of the Sprint it was possible to build, test, deploy and run the embryonic components that would form the start of the Alpha system in a just few automated minutes.

On the surface this appeared as a few simple “Hello World” web pages to show everything was alive and running, however the true value of this kind of automation comes later in the project as things change, grow and evolve, allowing the team to continually verify everything is still working as expected. In addition this pipeline will help with scaling and migrating our solution to different infrastructure in future.

In parallel to the technical work, the team continued with user research activities and refining  the story map and backlog.

Before leaving this topic I want to highlight that this is more controversial than you may think, if you do a search for the concept of Sprint 0 you’ll find some interesting philosophical discussions around the word “Sprint” and how it should be reserved for specific purposes or outcomes,  it almost feels like those who misuse the  name are labelled as heretics! For me, Sprint 0 served a purpose, everyone was clear what it meant and it achieved the outcome we wanted, calling it something else wouldn’t have changed that.

Finally, in my opinion, the effort spent upfront (whether you choose to call it Sprint 0 or something else) to establish the automation and delivery pipeline pays for itself many times over during the course of a project and should not be skipped.

Sprint One

With Sprint 0 complete, we were ready to start delivering. For Sprint 1, we wanted to deliver a thin slice through the system, allowing an author to login, create a questionnaire, add some questions, get it reviewed and published and allow a respondent to submit their answers. We weren’t sure if this would be achievable in 2-weeks, especially as this was the first time the team had developed together, but decided it was a good goal to aim for so jumped straight in.

I’ll let you judge whether we were successful, here’s a video of what we achieved:

…or view it here

We wanted to start testing with real users immediately, so Ben (our user researcher) spent the day after Sprint 1 in the usability lab with ONS staff getting them to try what we had created. This is early to be testing, but we wanted to establish a rhythm and workflow within the project that allowed testing with real users to feedback into the Alpha.

What’s next

Sprint 1 delivered a very thin slice of the end-to-end workflow that we can now build on. In Sprint 2 we’re looking to allow more editing and the addition of some new types of questions (e.g. multiple choice), whilst also allowing respondents to pause and resume completing a questionnaire later or from a different device.

Later Sprints will be expanding on the types of question available, start to tackle branching and routing through a questionnaire as well as various validation rules. We also plan to undertake some load testing in the Alpha to prove the scalability of our architecture.

Technicalities

For those who are so inclined here are some technical details of Sprint 0 and 1, for everyone else, stop reading now!

• The Alpha currently comprises of:
  • eq-author: Django (Python) web app for authoring and editing questionnaires
  • eq-survey-runner: Flask (Python) web app for collecting questionnaire responses
  • eq-terraform: Terraform project that creates the infrastructure for the Alpha.
• All code is on the ONSdigital GitHub in the open, we’re working on feature branches with pull requests for all changes, which are reviewed and merged to master.
• We’re using Travis CI for continuous integration and testing.
• The apps are being deployed on Heroku, with eq-author using a Heroku Postgres database.
• We’re using Foundation to support a responsive front-end.
• The team are using Python virtualenv to isolate their environments.
• We’re using twelve-factor app methodology

Note that this is far from a final setup, in Sprint 2 things are changing, including the introduction of Docker containers, but more on that later…