Make data part of the web

We have it written on stickers, we have it in our open data principles and now we have taken a small step to making more ONS data available to our users in a way that moves it towards being part of the fabric of the web. You can find the technical detail of what this is over on Rob’s post. This is a post about why.

The current version of ONS’s website is a couple of years old. We have made a real commitment to investing in the site and continuing to involve it over time, but at its heart, it is a collection of static HTML pages with excel documents attached to them. This has benefits (it is very fast and simple), but also challenges (the Excel files often have different formatting, don’t follow the same technical standards and can be inconsistently described) so we have spent some considerable time addressing this. This comes, as always, from our user research informing us on the direction to head and that direction is “help us find your data and your words”.

So far this has led to an Alpha phase of work and a Beta phase of work available to you all on our Beta site here. This starts to display considerably more metadata about datasets, starts to factor geography into the mix and exposes the structure of our data in clearer ways. In doing this, it means we can allow humans and computers to ask very specific questions of our data. This means we can do a couple of interesting things

1. Directly link to statistics using permanent URLs
2. Link to individual values in datasets.

Standards

If you have ever spoken to me for more than about 30 seconds you will have probably heard me mention web standards. I passionately believe in them and so does ONS. They are part of the heartbeat of the web and allow the work we are undertaking to hopefully not sit as an island floating on top of the web, but instead as part of the rich interconnected infrastructure that makes the web work so well. This is, I believe a Good Thing, but comes with challenges. The data we are providing via an API needs to be “cleaned” to ensure it is easy to make available and this is a time consuming and complex job. Even after cleaning the data, the API remains focused on individual datasets, not conjoining them together (though this could be something we look at in the much longer term)

However, we think that making the data in this format will help meet a number of additional user needs. It will be easier to create timeseries charts using our API. It will be possible to automatically update web applications with the latest data and it will allow our data to be more deeply woven into complex technical systems with no need with additional human intervention.

Whilst this is a very small start to a wider project, we cannot continue to develop it without your feedback. All of the pages have links to pass information back to us and my direct messages are open on Twitter and I will do my very best to respond to everything we get. This will help focus how we roll this out across the rest of our data. We will also be looking to run a small collection of public show and tells and hacks around the country over the summer to help showcase the capability of the data and get your feedback face to face. More news on this will be available soon.