Group Annotation of NECA and Urban Observatory Data Sets

Posted on 2nd Mar 2017 by James

Thanks to Gregory Marler for leading February’s event.

Attendees were invited to dive into the NECA Travel API and Urban Observatory API. Collated notes are available on Hackpad.

Newcastle’s Urban Observatory

Posted on 22nd Mar 2016 by James

A guest post by Phil James – Academic Lead on the Urban Observatory.

Urban Observatory logoThe Urban Observatory is funded as part of the University’s Science Central Programme to develop an urban sensing platform and capacity in Newcastle.  Building on over 10 years’ experience of urban sensing at Newcastle University the Urban Observatory has to date deployed over 150 sensors and holds over 50 million real-time observations with a throughput of over 200,000 observations per day.  Our data is all published in real-time for use by researchers, citizens and business.  Over the next five years an ambitious programme of urban sensing with an investment of over £2m in sensing hardware will enable transformative research and citizen participation and provide a robust evidence base for intervention.  The Urban Observatory is further supported with UKCRIC investment funds.

Smart Cities and Hype

There is an awful lot of hype around the smart cities agenda. Reading the press it would seem that all we need is the will and the finance and our cities will be super smart and somehow this makes life better the citizens. Here in Newcastle we are trying to go beyond the hype and build something usable based around real, open-data. Our dream is for a slightly less dumb city – a city where we are beginning to understand what sensors need to measure, with what level of accuracy and how many we need to understand the complex interactions in the city (or whether that is even possible). Urban sensing is really hard. There are massive procedural and physical deployment issues, the IoT technology, regardless of what the manufacturers claim, is in its infancy and the “science of sensing” barely warrants the epitaph. Casting around other “smart cities” one soon realises this is a marketing heavy, data light environment. When all cities say they are smart and present a couple of IoT pilot projects to prove it then the words become meaningless.


Our Goal and Future Vision

Our goal over the next five years is to keep Newcastle at the forefront of urban sensing. In the long term we want to inculcate an ethos of sensing by default – where policy is examined in the harsh light of data. We can only do this by trying to provide a baseline – when we know what the city is like now we have a chance of being able to say something about planned or actual interventions and their likely effect. Of course monitoring is only one small piece of the jigsaw in turning data into actionable information so the Urban Observatory is central to a wide range of related activity such as cross-sectoral modelling activity, data visualisation, big data management etc. that is happening across the University and beyond.

Making Infrastructure to Build Upon

We firmly believe that these systems must evolve and spread organically, partly due to cost as it is not cheap to deploy and maintain a sensing infrastructure and partly due to design – there are lots of organisations sensing stuff for their own operational needs and where possible we want to integrate and reuse this data. We don’t want to be tied to a single sensor type, manufacturer or technology. We have invested in building a sophisticated sensor-agnostic data platform and are building expertise in communicating with a variety of sensor types and platforms and published data streams. Above all we believe the best way to do this is by being open. Open in our data – it is published warts and all, open in the current level of our technical abilities and data quality, open in the problems and difficulties in deploying and maintaining sensors and open in our reflection on the wider issues such as long term governance and upkeep of monitoring data and open in what we don’t yet know. But we are ahead of the game – we have large scale deployments and many more planned over the next 5 years, we have a strong working and symbiotic relationship with the local councils, we have over 50 million published observations with a throughput of over 200,000 observations a day and rising, we have a sophisticated hybrid nosql-relational database infrastructure built on open source technology and a RESTful API and download facilities. We also want to engage – get people using our data, assist in deploying new sensors, support IoT technology in the region – so if you have an idea then come and talk to us.

Open. Simple. Scalable.

Phil James (email)
Academic Lead on the Urban Observatory.