ONS-API, Just the Numbers
Earlier this week we told you about our new lighter JSON-STAT format that we’re supporting in the API. Although much lighter and simpler than SDMX, there are still a few different sections in the response data, for many applications you might just be interested in ‘The raw numbers’
In this post, we’re going to walk you through a simple bit of code for requesting some data and then turning that into ‘just the raw numbers’. In this example, we want to know what the split of males and females was for England and Wales in the 2011 census. All we really want here is 2 numbers (maybe 3 if we want the total as well!)
This example is written using python but all we’re using is an HTTP library to make the request and a JSON library to parse the data, so it should be possible to do the same sort of thing in most other languages.
First of all we need to get setup:
https://gist.github.com/sammachin/a95caf9c3d129888c47c
https://gist.github.com/sammachin/a95caf9c3d129888c47c
Next we’re going to make the request to the API
https://gist.github.com/sammachin/ec92808750a3577a0d68
https://gist.github.com/sammachin/ec92808750a3577a0d68
Ok let’s take a look at what the API returned us:
https://gist.github.com/sammachin/0d87b5463b0c1e221e5a
https://gist.github.com/sammachin/0d87b5463b0c1e221e5a
Wow! That’s still quite a lot of stuff for just 3 numbers, let’s see if we can process that down:
https://gist.github.com/sammachin/68769b41166245a7e593
https://gist.github.com/sammachin/68769b41166245a7e593
OK, now we’ve got something that looks a lot simpler we can use in our app 🙂
https://gist.github.com/sammachin/f4230619a5141cc8bbe3
https://gist.github.com/sammachin/f4230619a5141cc8bbe3
If you want to grab that piece of code as a nice clean single script, then it can be downloaded from https://gist.github.com/sammachin/671f90c15ec6331598e5
Was this helpful? what sort of posts would you like to see about our API in future? let us know by leaving a comment.
7 comments on “ONS-API, Just the Numbers”
Comments are closed.
It’s not an obvious to use API is it?
If I run this query:
dataset/QS501EW.json?apikey=KEY&context=Census&geog=2011WARDH&dm/2011WARDH=E05008481,E05003606&jsontype=json-stat’
which requests results for two comma separated wards, I get the return values by iterating against index values obtained from ‘2011WARDH’ which is obj[dataset][‘dimension’][obj[dataset][‘dimension’][‘id’][0]][‘category’][‘index’] (that is, [‘id’][0] rather than id[‘1′])?
Is there any readable documentation to help try to make sense of decoding the results?!
In an ideal world, something like a pandas’ remote data source wrapper of the kind used for the World Bank APi would be really handy;-) http://pandas.pydata.org/pandas-docs/stable/remote_data.html
FWIW, I started doodling an IPython notebook to try to get my head round the API – http://nbviewer.ipython.org/gist/psychemedia/be0c279f2e51f6266b45
Still falling short of having easy to use functions to get any actual data into a dataframe though!
Hi, Tony:
I’m the author of pyjstat and have seen your notebook. Good job!
The library contained a bug related to value extraction: a case that was not tested. I have to apologize for this; pyjstat still does not fullfil all json-stat format requirements. It’s a work in progress and I hope it can get better with time and more statistical offices offering their data in this format.
I’ve run your code using this ONS dataset: QS104EW and have no problems with it being parsed by pyjstat 0.1.6. Could you check again?
Hi, Tony:
I’m the author of pyjstat and have seen your notebook. Good job!
The library contained a bug related to value extraction: a case that was not tested. I have to apologize for this; pyjstat still does not fullfil all json-stat format requirements. It’s a work in progress and I hope it can get better with time and more statistical offices offering their data in this format.
I’ve run your code using this ONS dataset: QS104EW and have no problems with it being parsed by pyjstat 0.1.6. Could you check again?