ONS Digital

The other weekend I attended LeedsHack, partly as this is a really nice friendly hackday and also to give the API its first outing since the updates. We were still very much in beta but I did manage to chat to a few developers about what the API can do and what they wanted to see in future, the biggest area of interest seems to be in additional datasets for things other than the Census. This is something we’re acutely aware of and we’ve got a full pipeline of datasets to come over the coming months, more about that in a future post.

I did get some time to build a hack using data from the API, so thought I’d walk you through what I built. Up until now most of the discussion has been about getting statistics from the API, this is probably because ONS is a statistics organisation, so internally that’s what people think about day to day, I wanted to build something that took some data from the API and presented it in a fun (maybe useful) little tool.

Now a tool is only useful if there is some sort of interactionI. If it just displays the same static data regardless, then it’s just information. The census data is broken down by area, so this seemed like a good place to start. I’ve always had a strong interest in mobile technology and just about all smartphones have location sensors in them, so we should be able to do something with that. Finally I needed to select which dataset I’d use from the census, as this was LeedsHack and being built for fun, I decided to use the number of people in an area that had declared their religion as Jedi on the 2011 census. Thus the UK Jedi Detector concept was born! The idea is a simple, one page mobile web app, that as you move around indicates if you are in an area of high Jedi population with some sort of visual indicator. No sign of any statistics or numbers just a glowing lightsaber that grows or shrinks depending in how strong ‘the force’ is 🙂

I’m not going to go into the detailed code in this post as I’ll share a link to the source at the end, however I’ll walk through the steps to build it.

The app is built with HTML/CSS/Javascript on the frontend, with a Python backend server to do the API calls and calculations. You could do the whole thing in javascript but my python is stronger and I only had a short time to build this.

The first thing that the web page needs to do is get the location from the device. This was done using the geolocation API in most browsers as part of the HTML5 updates, this will ask the user for permission to know their location then return the lat & long to the javascript.

Now that the app knows its location, I can send this up to my backend server and wait for the level of Jedi to come back.

On the backend server the first thing we need to do is convert the Lat/Long into the correct ONS geography code, I’ll be covering geography in more depth in a future post, but for now all we need to understand is that each census area is given a 9 digit code, ONS publishes this data along with various geographic mappings including the co-ordinates for the edge of the area. Fortunately for me, a few people have already done the hard work with that data and created some reverse geocoding apis which means I can pass the lat long and they will give me back the ONS 9 digit geography code. The two I found are http://uk-postcodes.com and http://postcodes.io, I went with uk-postcodes as this one gave me the 9 digit numbers, whereas postcodes.io seems to be using an older system. So within my python app I made the first call to uk-postcodes with the lat long and now I have the 9 digit code for that area. This allows me to get the count of Jedi in that area at the last census. For this, I’m using pretty much the same code as I showed in my previous blog post of getting just the numbers, so I won’t go into too much detail and the dataset I used was QS210EW.

At this point I came across a small problem, and something that I’m not completely happy with the fix for just yet, The API returns me a count of the number of Jedi, however I want to display a visual indicator so I need to know the scale, For example, if an area has 150 Jedi is that a high or low reading? The API returns us a total count for all the various responses that were declared in that area, so we can at least work out the percentage of Jedi in that area, I decided to benchmark this against the percentage for the UK as a whole, and have that as my centre value, this doesn’t feel like the best way of doing things but for the purposes of a fun hack it will suffice.

I converted the values into a decimal point number with the England and Wales ‘average’ being 0.003149, on my CSS lightsaber. In the web app, I need to display this as a number of pixels, so having decided the 150 pixels would be a good middle value I applied a multiplier of 47620 to give me a pixel value.

Finally I returned this data to the client in a json stream ready for the javascript to interpret and make the lightsaber grow appropriately. In effect, I have combined 2 API’s to create a new API where you can now send in a lat/long and get values back for the number of Jedi in the area 🙂

You can try out the app, it is best on a smartphone with GPS but it should work fine with desktop browsers too, but your location might be a bit ‘off’ and you can view all the source code.

Although this is mostly just an entertainment tool, I hope you can see how the code could be repurposed to display other short bits of data. For example you might be interested in knowing the average age of people in an area when house hunting, or how people travel to work. I’d love to see some other variations on this so please take my code, build on it and show us what you do.