As previously outlined in our post on March 30 on this blog, the Traffic Data Hack Day was hosted by Amazon Web Services on April 6. The event was attended by TfL employees, academics working within this subject area, data scientists, transport app developers and data visualisation developers.
Another key event in our engagement with those working with our open data, the day was split into two areas, with one focussed on visualising TfL data and the other on extracting data from the source and providing it for analysis.
I’ve not attended a hackathon or hack day before, so it was really interesting to see how they work and to meet participants from all sorts of backgrounds, including universities, consultancies, government agencies and other parts of TfL.
At the beginning of the day we split the focus into two areas; ‘EMR (Elastic Map Reduce)’ where the group was looking at ways to improve the feed of, access to and processing of the SCOOT data and ‘Redshift’ where the group made use of data that had been uploaded to the system and was ready to analyse.
I work in the Outcomes Insight & Analysis team at TfL and am currently working on the SITS (Surface Intelligent Transport Systems) programme where we are developing new ways of working with data and systems. It was great to see the various terms, such as Redshift, Hadoop, machine learning, big data and data fusion come to life and to see them demonstrated in practical ways rather than on architecture diagrams!
An exciting development was the speed within which machine learning was able to predict counts at a location, based on patterns picked up from historic data, and there was agreement that this is something TfL should focus on developing further.
I found it particularly useful to learn more about SCOOT (the traffic light optimiser system) data and what it shows. It was great to see the different uses of SCOOT and the different outputs from the day showed that it can be used to represent various different outputs (profiling, flows, occupancy etc) which would be very beneficial in future work.
Another highlight of the day was a presentation from Vizicities, who showed a 3D map of London. This example highlighted the need for good data visualisation to help explain the data to a wide audience.
We are continuing to make the sensor data as files and processed data in databases available both to the attendees of this hack day, as well as a month-long transport and mobility challenge which launches tomorrow.
Our sensor data is restricted under the terms of its current license but we are working towards making this an open data product and will update you as soon as there is any news on that.
Look out for details of more hack days and events on this blog, both ahead of the events with links to register as well as post-event write-ups like this one. If you were at this event on April 6, we’d love to hear from you in the comments section below with any thoughts you have on the event and anything that was particularly useful, as well as things we could do better in future.