Beta API: An introduction

As you may have already seen we recently launched our separate API and developer site in Beta. I wanted to build on Andy’s introduction post and talk a bit about the structure of the data available in the API and give some simple examples.

Anyone familiar with the breadth of content that ONS publish will notice quite quickly that not all of it is currently available from the API. Expanding this is the plan over time, and my main focus as part of our Customise My Data (formerly Data Discovery) developments will be just that.

The API currently focuses around the data we do have already available in a structured format, specifically our economic time series. These are already available from the current site-based API by adding ‘/data’ to the end of the page URL – for RPI the individual page for this series is; https://www.ons.gov.uk/economy/inflationandpriceindices/timeseries/czbh/mm23 and the data for this is available in JSON from https://www.ons.gov.uk/economy/inflationandpriceindices/timeseries/czbh/mm23/data.

This method of making the data available has some benefits but also has some issues, in particular that, should the structure of the website change, this URL would too and also you need to know where on the site the series you want is before you can use it.

To access the data itself from the API you will need to use these individual time series. In addition to this you will need to decide which dataset version of the series you want. For a lot of series they only appear in one dataset but some, particularly those around GDP can be published in a number of datasets.

Finding a series

You can find a list of all time series in a specific dataset using the following endpoint, so if you are familiar with these or know the broad subject area you are interested in this is probably the place to start.

Here is the CPI example;

https://api.ons.gov.uk/dataset/mm23/timeseries

You can also use our separate search to identify which time series you want, for example; https://api.ons.gov.uk/search?q=rpi

This will provide details of what is available, both datasets and time series, and once you know what you need, take a note of the field labelled ‘cdid’. This will give you the series identifier of that specific series.

These IDs are also published alongside these series on the website so this is another alternative route to finding them.

Getting data with a CDID (series identifier)

Once you have your series ID you can use the following endpoint to access information about that series; 

https://api.ons.gov.uk/timeseries/czbh

This endpoint will also give details of the dataset, or datasets, that series is published as part of. You can see the following example is published in 5 separate datasets (look at ‘datasetId’);

https://api.ons.gov.uk/timeseries/abmi

You can then access the detail for this variation of the data by including this in your request, for example;

https://api.ons.gov.uk/timeseries/abmi/dataset/ukea

The data for this series is then accessed the same way as on the website, with the addition of ‘/data’, as shown here;

https://api.ons.gov.uk/timeseries/abmi/dataset/ukea/data


This API and developer site have been put out in Beta to gather feedback to inform future developments, particularly our Customise my Data developments which will be looking to build on this with much more ONS data and metadata.

If you have any thoughts or comments on this let me know on twitter @robchamberspfc, on this blog post or to the team @ONSdigital

2 comments on “Beta API: An introduction”

  1. Hello,

    Thanks for this blog. I am trying to explore the data set within the API on the official API documentation page but am a bit confused.

    https://developer.ons.gov.uk/search/search-datasets-id-editions-edition-versions-version-dimensions-name/

    Which section actually gives me access to say regional GDP data timeseries?
    And how do I work with the parameters?

    For example, this is how I am making a request on python jupyter

    # Define the base URL
    # Base URL: the part of the URL common to all requests, not containing the parameters
    base_url = “https://api.beta.ons.gov.uk/v1”

    # The number of results can be set (to a maximum of 200) with the ‘limit’ parameter
    r = requests.get(base_url_1, params = {“id”: “regional-gdp-by-year”, “keywords”: “regional GDP”, “limit”: 200})
    r.ok

    Would appreciate some feedback

    Thanks!

    1. Hello, great question hopefully I can be of some assistance.

      So from the looks of your query, you are trying to access the following dataset via the API (web page equivalent found at this URL) https://www.ons.gov.uk/datasets/regional-gdp-by-year/editions/time-series/versions/2 .

      In short, I can’t help you with Python however I believe this is the API URL you are looking for
      https://api.beta.ons.gov.uk/v1/datasets/regional-gdp-by-year/editions/time-series/versions/2/
      To use parameters you can add a ‘?’ at the end of the URL and separate each parameter with a ‘&’. e.g. https://api.beta.ons.gov.uk/v1/datasets/regional-gdp-by-year/editions/time-series/versions/2/observations?unofficialstandardindustrialclassification=A–T&time=*&growthrate=gra&geography=UKL&prices=cvm . If you would like to know how I worked out what parameters and options were available for this dataset then hopefully the below will help.

      I will start from the top of the API journey and work towards getting data from that dataset. I will take the more comprehensive and scenic route in hopes this might provide the information you need and help you understand how it can be used. However, you will probably notice some shortcuts that can be taken along the way.

      Before I go into this let me first explain a quick concept, a dataset has editions, and an edition has versions. Each version of a dataset can be filtered to retrieve observations (the underlining data).

      So we start with the base URL for the API. Which I can see you already have ‘https://api.beta.ons.gov.uk/v1/’. All API requests are GET requests unless I specify differently below.

      First of all, discover what Datasets are available we can do this by sending the following API request to get a catalogue if you like of all available datasets:
      https://api.beta.ons.gov.uk/v1/datasets
      ( Documentation can be found here https://developer.ons.gov.uk/tour/getting-started/ )

      Let’s say you have now identified from that response which dataset you are interested in and have made a note of the dataset ID. In this case, the dataset ID is ‘regional-gdp-by-year’. We can now query that dataset to see what editions are available. To do this, we can query this API endpoint:
      https://api.beta.ons.gov.uk/v1/datasets/regional-gdp-by-year/editions
      ( Documentation can be found here https://developer.ons.gov.uk/dataset/datasets-id-editions/)

      Now let’s say you have identified the edition of the dataset you are interested in from that response. In this case, the only one available is ‘time-series’. The next step would be to work out what versions of that edition are available. We can do this by sending a request to the following API endpoint
      https://api.beta.ons.gov.uk/v1/datasets/regional-gdp-by-year/editions/time-series/versions
      ( Documentation can be found here https://developer.ons.gov.uk/dataset/datasets-id-editions-edition-versions/ )

      Ok let’s say you have found the version you want, I will assume you want the latest which (at time of writing) is version 2. You can either download the data by following a URL presented in response to an API query to this endpoint:
      https://api.beta.ons.gov.uk/v1/datasets/regional-gdp-by-year/editions/time-series/versions/2
      (Documentation can be found here https://developer.ons.gov.uk/dataset/datasets-id-editions-edition-versions-version/ )

      Alternatively, we can use this information to create a filter to retrieve observations. To do this, we need to know what dimensions are available on a dataset. We can do this by querying
      https://api.beta.ons.gov.uk/v1/datasets/regional-gdp-by-year/editions/time-series/versions/2/dimensions
      ( Documentation can be found here https://developer.ons.gov.uk/dataset/datasets-id-editions-edition-versions-version-dimensions/ )

      In that response, we can see the following dimensions ‘time’, ‘growthrate’, ‘geography’, ‘unofficialstandardindustrialclassification’ and ‘prices’. We now can see what dimension options are available for us to filter on each dimension category. For example, we can do this for the dimension ‘time’ by querying:
      https://api.beta.ons.gov.uk/v1/datasets/regional-gdp-by-year/editions/time-series/versions/2/dimensions/time/options
      ( Documentation can be found here https://developer.ons.gov.uk/dataset/datasets-id-editions-edition-versions-version-dimensions-dimension-options/ )

      Finally, we can put all of this together to get observational level data for this dataset by querying for example:
      https://api.beta.ons.gov.uk/v1/datasets/regional-gdp-by-year/editions/time-series/versions/2/observations?unofficialstandardindustrialclassification=A–T&time=*&growthrate=gra&geography=UKL&prices=cvm
      ( Documentation can be found here https://developer.ons.gov.uk/dataset/datasets-id-editions-edition-versions-version-observations/ )

      The Customise My Data (CMD) API also offers a more comprehensive filter journey rather than just using the observations endpoint https://developer.ons.gov.uk/filter/. Happy to try help if you have any more queries or need further assistance.

Leave a Reply to Dan Cancel reply

Your email address will not be published. Required fields are marked *