Archive

Archive for the ‘linked data’ Category

Ordnance Survey SPARQL Endpoint

August 7, 2013 Leave a comment

I just wanted to quick mention one feature of the Ordnance Survey linked data SPARQL endpoints that I think it pretty neat.  Go to the SPARQL endpoint and try one of the queries from my last four blogs posts. In this post I’ll got with the following simple query (recall this query gets the name, lat, long, gss code and unit_id for all districts in Great Britain):

select ?name ?lat ?long ?gss ?unit_id

where

{

?x <http://www.w3.org/2000/01/rdf-schema#label> ?name .

?x <http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat .

?x <http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long .

?x <http://data.ordnancesurvey.co.uk/ontology/admingeo/gssCode> ?gss .

?x <http://data.ordnancesurvey.co.uk/ontology/admingeo/hasUnitID> ?unit_id .

?x a <http://data.ordnancesurvey.co.uk/ontology/admingeo/District> .

}

You will notice that on hitting the query button that a box will appear that says “Request” and a rather long URL will appear:

SPARQL

You can now use this URL to issue a GET request in PHP, Javascript etc. and use these output within a web application just as you would with any API call. To see this working in a simple way copy the long URL you get from your SPARQL query and at the command line (if running something UNIXy) type:

curl LONG_URL

where LONG_URL is your long URL. You should now see the JSON response from that GET request.

Happy SPARQLing…

Ordnance Survey Linked Data – Combining postcode and spatial queries

August 5, 2013 Leave a comment

In my previous blog posts (here, here and here) I concentrated on a few simple SPARQL queries. However, these SPARQL queries were only performed on one dataset at a time. With linked data things get more interesting when you combine datasets (even if those datasets are both from the same publisher). The original opendata for Code-Point(R) Open was published as a CSV file, and Boundary-Line(TM) was published as a shape file. Both formats are useful, but combining the two datasets can be tricky if you are not sure how to use shape files. The two datasets are, however, implicitly linked via a key. In the linked data versions we have converted both datasets into a common data language (RDF) and made those implicit links explicit (see https://johngoodwin225.wordpress.com/2013/08/05/ordnance-survey-linked-data-a-simple-postcode-query/ ). We can now easily query both datasets together.

To query both datasets go to the SPARQL endpoint for the combined data:

http://data.ordnancesurvey.co.uk/datasets/os-linked-data/explorer/sparql

Recall that the query:

select ?postcode ?lat ?long
where
{
  ?x <http://www.w3.org/2000/01/rdf-schema#label> ?postcode .
  ?x <http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat .
  ?x <http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long .
  ?x <http://data.ordnancesurvey.co.uk/ontology/postcode/ward>
     <http://data.ordnancesurvey.co.uk/id/7000000000017707> .
}

finds all of the postcodes (and their lat/long) in a ward called Bevois. Using the combined datasets we can run queries like: find me all postcodes in the ward Bevois and all the wards touching Bevois. Let’s break this down. To find all postcodes (and their lat/long) in regions that touch Bevois first find all the wards that touch Bevois, and then find all the postcodes within those wards. So the first line in the query below  “?y <http://data.ordnancesurvey.co.uk/ontology/spatialrelations/touches&gt; <http://data.ordnancesurvey.co.uk/id/7000000000017707>&#8221; finds all wards touching Bevois, and the rest of the query matches postcodes in the wards “?y”:

select ?postcode ?lat ?long
where
{
  ?y <http://data.ordnancesurvey.co.uk/ontology/spatialrelations/touches>
  <http://data.ordnancesurvey.co.uk/id/7000000000017707> .
  ?x <http://www.w3.org/2000/01/rdf-schema#label> ?postcode .
  ?x <http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat .
  ?x <http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long .
  ?x <http://data.ordnancesurvey.co.uk/ontology/postcode/ward> ?y .
}

To find all the postcodes in Bevois and the wards touching Bevois we simply union the two queries above together:

select ?postcode ?lat ?long
where
{{
  ?x <http://www.w3.org/2000/01/rdf-schema#label> ?postcode .
  ?x <http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat .
  ?x <http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long .
  ?x <http://data.ordnancesurvey.co.uk/ontology/postcode/ward>
     <http://data.ordnancesurvey.co.uk/id/7000000000017707> .
}

UNION

{
  ?y <http://data.ordnancesurvey.co.uk/ontology/spatialrelations/touches>
  <http://data.ordnancesurvey.co.uk/id/7000000000017707> .
  ?x <http://www.w3.org/2000/01/rdf-schema#label> ?postcode .
  ?x <http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat .
  ?x <http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long .
  ?x <http://data.ordnancesurvey.co.uk/ontology/postcode/ward> ?y .
}}

As an exercise for the reader – find all postcodes in Southampton and districts touching Southampton. Happy SPARQLing…

Ordnance Survey Linked Data – A Simple Postcode Query

August 5, 2013 1 comment

In the previous two blog posts (here and here) I gave some simple examples of queries you could run on the Ordnance Survey Boundary-Line(TM) linked data. Here I want to give some simple examples of what you can do with the Code-Point(R) Open linked data. The Code-Point(R) Open linked data has a URI for every Postcode Unit, Postcode Sector, Postcode District and Postcode Area in England, Scotland and Wales.  Each Postcode Unit is nested within a Postcode Sector, each Postcode Sector is nested within Postcode District and each Postcode District is nested within a Postcode Area. The reciprocal contains relationships are also included.

A Postcode Unit URI takes the form:

http://data.ordnancesurvey.co.uk/id/postcodeunit/SO160AS

A Postcode SectorURI takes the form:

http://data.ordnancesurvey.co.uk/id/postcodesector/SO160

A Postcode District URI takes the form:

http://data.ordnancesurvey.co.uk/id/postcodedistrict/SO16

A Postcode Area URI takes the form:

http://data.ordnancesurvey.co.uk/id/postcodearea/SO

Let us now try some SPARQL. Go to the CodePoint-Open SPARQL endpoint, and for simplicity select the response format to CSV. Supposed we wanted to select all of the postcodes and their lat/long coordinate for postcodes within the SO postcode area. This can be done simply as follows:

select ?postcode ?lat ?long
where
{
  ?x <http://www.w3.org/2000/01/rdf-schema#label> ?postcode .
  ?x <http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat .
  ?x <http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long .
  ?x <http://data.ordnancesurvey.co.uk/ontology/spatialrelations/within>
     <http://data.ordnancesurvey.co.uk/id/postcodearea/SO> .
}

Notice that I do not need to specify that ‘x’ is a postcode unit in this case as only postcode units have a lat/long value.

Let us try a slightly more complicated query – say I wanted all the postcodes, and their lat/long, within the postcode sectors SO16 and SO17. I would do a query similar to the above, but with a UNION to collect together results for each postcode sector:

select ?postcode ?lat ?long
where
{
{
  ?x <http://www.w3.org/2000/01/rdf-schema#label> ?postcode .
  ?x <http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat .
  ?x <http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long .
  ?x <http://data.ordnancesurvey.co.uk/ontology/spatialrelations/within>
     <http://data.ordnancesurvey.co.uk/id/postcodedistrict/SO16> .
}
UNION
{
  ?x <http://www.w3.org/2000/01/rdf-schema#label> ?postcode .
  ?x <http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat .
  ?x <http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long .
  ?x <http://data.ordnancesurvey.co.uk/ontology/spatialrelations/within>
     <http://data.ordnancesurvey.co.uk/id/postcodedistrict/SO17> .
}
}

So far so easy…

The Code-Point Open linked data also contains links to the administrative areas that ‘contain’ the postcode. A word of caution is needed here. Postcodes do not respective administrative boundaries so containment in this case actually means the administrative region that the lat/long for that postcode lies within. There are three predicates for relating postcode units to administrative areas. These are:

  • ward – this relates postcode units to wards and unitary electoral divisions
  • district – this relates postcode units to districts, metropolitan districts, London boroughs and unitary authorities
  • county – this relates postcode units to counties (where applicable)

So say I want a list of all the postcode units (and their lat/longs) that lie within the ward of Bevois. This is a straightforward query. First you can find the URI for Bevois here. You’ll find the URI for Bevois is:

http://data.ordnancesurvey.co.uk/id/7000000000017707

Now enter the following SPARQL query to find all the postcodes in Bevois:

select ?postcode ?lat ?long
where
{
  ?x <http://www.w3.org/2000/01/rdf-schema#label> ?postcode .
  ?x <http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat .
  ?x <http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long .
  ?x <http://data.ordnancesurvey.co.uk/ontology/postcode/ward>
     <http://data.ordnancesurvey.co.uk/id/7000000000017707> .
}

I’ll leave it as an exercise for the reader to find all of the postcode units (and their lat/long) in Southampton.

What if I want to find out the postcode districts that are covered by the ward Bevois? First I find all the postcode units in Bevois, and then I do a query to look up all the postcode districts those postcode units are within as follows:

select distinct ?postcodedistrict
where
{
  ?x <http://data.ordnancesurvey.co.uk/ontology/postcode/ward>
     <http://data.ordnancesurvey.co.uk/id/7000000000017707> .
  ?x <http://data.ordnancesurvey.co.uk/ontology/spatialrelations/within> ?y .
  ?y a <http://data.ordnancesurvey.co.uk/ontology/postcode/PostcodeDistrict> .
  ?y <http://www.w3.org/2000/01/rdf-schema#label> ?postcodedistrict .
}

You will now see all the postcode districts the ward Bevois covers. As a final exercise to the reader perform the same query, but find postcode sectors that cover Southampton. Happy SPARQLing.

Ordnance survey Linked Data – Simple SPARQL example

August 1, 2013 Leave a comment

Yesterday I received a request asking how to extract some simple data from the Ordnance Survey linked data using a SPARQL query. This post is not intended as a SPARQL tutorial – you can find plenty of those here.

A user wanted to know how to retrieve the name, unit-id, GSS Code, lat and long of all the unitary authorities, districts and metropolitan districts in England, Scotland and Wales as a CSV file.

To extract this information for all of the districts go to the Ordnance Survey’s Boundary-Line(TM) linked data SPARQL endpoint explorer and in the response format drop down menu select CSV. Now in the query window enter the following query:

select ?name ?lat ?long ?gss ?unit_id

where

{

?x <http://www.w3.org/2000/01/rdf-schema#label> ?name .

?x <http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat .

?x <http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long .

?x <http://data.ordnancesurvey.co.uk/ontology/admingeo/gssCode> ?gss .

?x <http://data.ordnancesurvey.co.uk/ontology/admingeo/hasUnitID> ?unit_id .

?x a <http://data.ordnancesurvey.co.uk/ontology/admingeo/District> .

}

This query selects the various attributes from the data, and the final line of the query makes sure that all of the entities selected from the data are of type District.

Scroll down the page and you should see the query response. To get the values for the district, unitary authorities and metropolitan districts we need to use a SPARQL union to gather together all of the results as follows:

select ?name ?lat ?long ?gss ?unit_id

where

{

{

?x <http://www.w3.org/2000/01/rdf-schema#label> ?name .

?x <http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat .

?x <http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long .

?x <http://data.ordnancesurvey.co.uk/ontology/admingeo/gssCode> ?gss .

?x <http://data.ordnancesurvey.co.uk/ontology/admingeo/hasUnitID> ?unit_id .

?x a <http://data.ordnancesurvey.co.uk/ontology/admingeo/District> .

}

UNION

{

?x <http://www.w3.org/2000/01/rdf-schema#label> ?name .

?x <http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat .

?x <http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long .

?x <http://data.ordnancesurvey.co.uk/ontology/admingeo/gssCode> ?gss .

?x <http://data.ordnancesurvey.co.uk/ontology/admingeo/hasUnitID> ?unit_id .

?x a <http://data.ordnancesurvey.co.uk/ontology/admingeo/MetropolitanDistrict> .

}

UNION

{

?x <http://www.w3.org/2000/01/rdf-schema#label> ?name .

?x <http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?lat .

?x <http://www.w3.org/2003/01/geo/wgs84_pos#long> ?long .

?x <http://data.ordnancesurvey.co.uk/ontology/admingeo/gssCode> ?gss .

?x <http://data.ordnancesurvey.co.uk/ontology/admingeo/hasUnitID> ?unit_id .

?x a <http://data.ordnancesurvey.co.uk/ontology/admingeo/UnitaryAuthority> .

}

}

order by ?name

The ‘order by’ at the end of the query orders the results in alphabetical order.

To save the query results as a CSV file again make sure that response format in set to CSV and this time, before hitting the query button, make sure the ‘show raw response’ option is selected. Now hit the query button and you should be given the option to save your query result as a CSV file.

Ordnance Survey Linked Data and the Reconciliation API

April 25, 2013 5 comments

The new Ordnance Survey Linked Data has a reconciliation API that allows users to turn text into URIs by matching against the Ordnance Survey linked data using a tool called open refine.

I’m not an expert on open refine but had a quick try of the tool today using some open data about libraries (available here). Instructions on installing Open Refine can be found here.

To use the Open Refine load your data into the tool and create your new project. On loading the library data into Open Refine you should see something like this:

Image

We can use Open Refine to turn the labels in both the ‘county’ column and postcode column into URIs. For the county column click the down arrow next the column name and select reconcile -> start reconciling. Now click ‘Add Standard Service’ and add the following URL http://data.ordnancesurvey.co.uk/datasets/boundary-line/apis/reconciliation. 

As the ‘county’ column will contain a mixture of types select the ‘reconcile against no particular type’ option and click ‘start reconciling’. You should now see that most of the text labels have turned to hyperlinks (note OS linked data does not included Northern Ireland data…this accounts for the missing values).

You can do the same for the postcode column, but this time use the API at: http://data.ordnancesurvey.co.uk/datasets/code-point-open/apis/reconciliation

Your data should now look something like:

Image

You have now successfully replaced the text in these columns with links to the OS linked data.

Another useful thing to try is a simple bit of geocoding based on postcodes. Again go to the postcode column and select “Edit Column -> Add Column by fetching URLs’. Where asked type in a column name (e.g. PC JSON) and in the Expression box type:

http://data.ordnancesurvey.co.uk/datasets/code-point-open/apis/search?output=json&query=’ + escape(value,’url’)

You should now see a column appear full of JSON results:

Screen Shot 2013-04-25 at 15.23.11

On the PC JSON column select “Edit Column -> Add Column Based on this column”. Again add a column name of your choice. I wanted to extract the value of the easting and northing and add it as a column so I called my new column ‘easting,northing’. In the expression box enter the following to get the value of the easting and northing:

with(value.parseJson(), pair, pair.results[0].easting + ‘,’ + pair.results[0].northing)

and you should now see something like:

Screen Shot 2013-04-25 at 15.27.27

Congratulations…you have now geo-coded your libary spreadsheet via a postcode and the OS linked data.

For more info on how to use Open Refine for reconciliation watch this youtube video.

Making things with Ordnance Survey Linked Data…

November 3, 2011 7 comments

Following the example of “Making things with BBC data” I thought I’d ask the same question for Ordnance Survey linked data. Please leave a comment if you’ve used Ordnance Survey linked data for anything from a quick hack, full blown project or if you even just link to it in your data. Thanks!

 

 

/location /location /location – exploring Ordnance Survey Linked Data – Part 2

October 25, 2010 5 comments

Ordnance Survey have now released an update to their linked data, which can be seen here. The new data now includes postcode information as well as a few changes to the administrative geography data. In this post I’ll go through what’s in the data, and give a few sample SPARQL queries.

I spoke a bit about the administrative geography data in a previous blog post – but the data has changed a bit since then. Just to re-cap the administrative geography linked data contains information about administrative and voting geographic regions. These include unitary authorities, counties, wards, constituencies, Welsh Assembly regions and a whole lot more [1]. Here are some examples:

If you want to find a full list of the sorts of thing you can find in the data simply go to the query interface (or SPARQL endpoint as it is know) and try the following query:

select distinct ?type

where { ?a a ?type . }

Now you have the list all of type of things in the data you can as for lists of instances of those types.

For example, the following query will return all of the unitary authorities:

select ?a

where {

?a a <http://data.ordnancesurvey.co.uk/ontology/admingeo/UnitaryAuthority&gt; .

}

All of the names of all the regions have now been modelled using the SKOS vocabulary. If you want to find the official names of all the unitary authorities you can simple issue a query like:

select ?a ?name

where

{

?a a <http://data.ordnancesurvey.co.uk/ontology/admingeo/UnitaryAuthority> .

?a <http://www.w3.org/2004/02/skos/core#prefLabel&gt; ?name  .}

Also included in the data are two attributes called Unit ID and Area Code. These values are useful if you want to produce a mashup using this data and display it by boundary.

So for example, for Southampton (http://data.ordnancesurvey.co.uk/id/7000000000037256) the area code is UTA (for unitary authority) and the unit ID is 37256. These values can be used as follows:

/*here we set-up the our variable called ‘boundaryLayer’ with the strategies that we require. In this case, it is its ID and type i.e. Unitary Authority */

boundaryLayer = new OpenSpace.Layer.Boundary(“Boundaries”,

{ strategies: [new OpenSpace.Strategy.BBOX()], admin_unit_ids: [“37256”], area_code: [“UTA”] });

//then we add the bounadry to the map osMap.addLayer(boundaryLayer);

//this effectively refreshes the map, so that the boundary is visible

osMap.setCenter(osMap.getCenter());

to display the Southampton boundary using the OS OpenSpace API. See http://openspace.ordnancesurvey.co.uk/openspace/support.html for more details.

Arguably the most useful information in this data are the qualitative spatial relationships between different regions. Regions are related to the regions they contain, they are within and they touch. In the case of the touching relationship only regions of the same type have an explicit touching relationship. The exception to this are unitary authorities, counties, district and metropolitan district that also have touching relationships between each other. The following simple query will return a list of all counties, districts and unitary authorities that border The City of Southampton. It will also return their names:

PREFIX spatialrelations: <http://data.ordnancesurvey.co.uk/ontology/spatialrelations/&gt;

select ?a ?name

where

{

?a spatialrelations:touches <http://data.ordnancesurvey.co.uk/id/7000000000037256&gt; .

?a <http://www.w3.org/2004/02/skos/core#prefLabel&gt; ?name  .

}

If you are only interested in the bordering counties you can add an extra line to your query:

PREFIX spatialrelations: <http://data.ordnancesurvey.co.uk/ontology/spatialrelations/&gt;

select ?a ?name

where

{

?a spatialrelations:touches <http://data.ordnancesurvey.co.uk/id/7000000000037256&gt; .

?a <http://www.w3.org/2004/02/skos/core#prefLabel&gt; ?name  .

?a a <http://data.ordnancesurvey.co.uk/ontology/admingeo/County> .

}

Similarly, the following query returns all the county electoral divisions (and their names) within Hampshire:

PREFIX spatialrelations: <http://data.ordnancesurvey.co.uk/ontology/spatialrelations/&gt;

select ?a ?name

where

{

?a spatialrelations:within <http://data.ordnancesurvey.co.uk/id/7000000000017765&gt; .

?a <http://www.w3.org/2004/02/skos/core#prefLabel&gt; ?name  .

?a a <http://data.ordnancesurvey.co.uk/ontology/admingeo/CountyElectoralDivision> .

}

For convenience some shortcuts have been added to the data in this release. For certain nesting geographies, such as the county – district – parish or district – ward nestings, various new properties have been added. For example, the property ‘counyElectoralDivision‘ relates all counties to their constituent county electoral divisions. The above query can now be done in a simpler way:

PREFIX admingeo: <http://data.ordnancesurvey.co.uk/ontology/admingeo/&gt;

select ?a ?name

where

{

<http://data.ordnancesurvey.co.uk/id/7000000000017765&gt; admingeo:countyElectoralDivision ?a .

?a <http://www.w3.org/2004/02/skos/core#prefLabel&gt; ?name  .

}

Similar predicates such as ‘county‘, ‘district‘, ‘ward‘, ‘constituency‘ etc. provide similar shortcuts. For example, the following returns all the Westminster constituencies in South East England.

PREFIX admingeo: <http://data.ordnancesurvey.co.uk/ontology/admingeo/&gt;

select ?a ?name

where {

<http://data.ordnancesurvey.co.uk/id/7000000000041421&gt; admingeo:westminsterConstituency ?a .

?a <http://www.w3.org/2004/02/skos/core#prefLabel&gt; ?name  . }

The most significant introduction to this data is the inclusion of postcode information. The data now contains information about postcode units, postcode sectors, postcode districts and postcode areas. For each postcode unit an easting/northing coordinate value is given [2] along with the district, ward and county (where applicable) that contains said postcode unit. An example of this can be seen for the Ordnance Survey postcode SO16 4GU. Each postcode is also related to its containinb postcode area, sector and district.

The properties ‘ward‘, ‘district‘ and ‘county‘ relate a postcode to the relevant regions. The simple query:

PREFIX postcode: <http://data.ordnancesurvey.co.uk/ontology/postcode/&gt;

select ?district

where {

<http://data.ordnancesurvey.co.uk/id/postcodeunit/SO164GU&gt; postcode:district ?district .

}

returns the unitary authority that contains the postcode SO16 4GU.

This query:

PREFIX spatialrelations: <http://data.ordnancesurvey.co.uk/ontology/spatialrelations/&gt;

select ?postcode

where

{

?postcode spatialrelations:within <http://data.ordnancesurvey.co.uk/id/postcodearea/SO&gt; .

}

returns all the postcodes in the SO postcode area.

We can combine the above two queries to find the areas, along with their names, covered by the postcode area SO:

PREFIX spatialrelations: <http://data.ordnancesurvey.co.uk/ontology/spatialrelations/&gt;

PREFIX postcode: <http://data.ordnancesurvey.co.uk/ontology/postcode/&gt;

select distinct ?district ?name

where

{

?postcode spatialrelations:within <http://data.ordnancesurvey.co.uk/id/postcodearea/SO&gt; .

?postcode postcode:district ?district .

?district <http://www.w3.org/2004/02/skos/core#prefLabel&gt; ?name  .

}

Hopefully these few examples will give you enough information to fully explore this new release of the Ordnance Survey linked data. For those of you who don’t like SPARQL watch this space – hopefully we will soon(ish) have an API built on top of this data to allow for even easy access.

[1] you’ll notice the ‘isDefinedBy’ link currently returns a 404 – not for long I hope 🙂

[2] lat/long to follow