We are excited to launch our new TfL Oyster app on iOS and Android, which allows customers to top up their Oyster cards, purchase Travelcards and view their journey history. The app was launched last week, and has already received lots of great feedback. We wanted to offer you more insight into how we developed it.
An API – or application programming interface – is a set of subroutine definitions, protocols and tools for building application software¹. We already have a wide range of public APIs, which provide information such as line status, bus status and journey information. To build a mobile application allowing customers access to their Oyster card data through, we needed to write a new API to support this.
Authenticating user accounts
Unlike the TfL Unified APIs which provide open data, we needed to secure our new API to only allow customers access to their own personal data. Prior to beginning work on the TfL Customer API, customers already had the ability to create website accounts for the Oyster and contactless websites. These accounts are authenticated using a single sign on (SSO) application, so we set about extending this to use in an API.
Authentication with the TfL Customer API is handled via a three-way handshake between the mobile app, SSO and the TfL Customer API. When a customer enters their email address and password into the mobile app, their information is sent to the SSO API which replies back to the mobile app with a one time token.
This token is then submitted to the TfL Customer API which communicates directly with the SSO to validate it’s authenticity. If the one time token is valid, SSO will respond to the TfL Customer API with an ‘OK’ message. This completes the three-way handshake. Now the TfL Customer API responds to the mobile app with a short lived authentication token and a refresh token, to allow the mobile app to renew the authentication once the short lived token has expired, without the customer having to re-enter their credentials.
Displaying up to date balance and journey information
Now our customers can securely authenticate with the customer API, we needed to go about returning them information about their Oyster card(s). We receive on average 16 million Oyster card taps across the network every day and, as you can imagine, this is a lot of information to process.
When a customer taps their Oyster card on a reader, information about that tap is delivered to the customer API in batches of 8000 taps, or every 1 minute, whichever happens first. If any of these taps are balance affecting, i.e. the customers balance has been altered, then the updated balance is saved to a Microsoft SQL database. Now when a customer requests their balance via the app, the API can retrieve it from the SQL database, with the timestamp of their tap.
We also store all Oyster taps in a Microsoft Azure Table storage account. Azure Table storage is a simple database store – you have tables, partitions and rows. You can put data in quickly and query it back out again. That’s a pretty simplified description of Azure Table storage, but you can read more about it in this post by Julie Lerman – Windows Azure Table Storage – Not Your Father’s Database.
Now we have a complete store of every tap a customer makes with their Oyster card, we can construct their journeys. When a customer requests their journeys in the mobile app, the customer API queries Azure Table storage to receive all taps for the given Oyster card in 7 day periods, and it constructs the journeys on the fly. The journeys are constructed using the fares rules, and a charge for the journey is also displayed.
Topping up and purchasing Travelcards
We wanted our customers to be able to top up their Oyster card and purchase Travelcards on the go, and collect their order when they next tapped their card on an Oyster reader. To do this we had to communicate with an internal API to allow us to send information to the Oyster readers.
When a customer selects to top up their Oyster card or purchase a Travelcard in the app, the app first communicates with the customer API to place a pending order. The API responds with some information to the app about the pending order which the app can then send on to our payment processing provider. Once the payment provider has authorised the customers bank card for the amount requested, the customer API is called to request fulfilment of the order. This is done by the customer API interfacing with the internal TfL API to send the order to the Oyster readers.
Orders are sent out to all readers across the network within 30 minutes of placing an order via the customer API. Customers simply have to tap their Oyster card when starting or finishing a journey and their Oyster card will be topped up, or a Travelcard added to their account.
Technology behind the TfL Customer API
The TfL customer API needs to be highly available and scalable so that millions of Londoners can access their Oyster account and top up their Oyster cards. To allow this we have utilised the cloud, using Microsoft Azure. The TfL Customer API and supporting processors are hosted using Azure Cloud Services. We also use Azure Service Bus for asynchronous messaging, Redis cache for fast data access, Azure Table storage, Azure SQL Server and Azure KeyVault.
We hope you have learnt a lot from this post and if you have any questions or feedback, please do comment below.
We really hope you enjoy the app as much as we have enjoyed building it.
Are there any plans to make the Customer API available to third parties? I can think of lots of interesting ways to use journey history data. I know there’s concern about personal data etc, but wondering whether your SSO solution goes some way to addressing this. If applications are only given short lived authentication tokens when a user authorises them, then users are in control of their own data.
Running both oyster cards for my family and friends, I am dissapointed that I and my freedom pass are excluded from the data heap. The data is on the same system and uses the same verifications tokens so why is the usage data not available. Access is particualarily important as a means of evidence but it seems only available on a restricted basis. There is no additional cost to providing access to the heap only difference is there is no automatic refunds – this is of interest to find out if the local authority is getting benefit back from this system for disrubted kourneys.
TfL comments on this availabilty is awaited with interest
is the tfl customer api available to the public?
Hi,
Will Tfl be allowing developers to access this Customer API like the unified API?
Thanks,