API Days

In my last post I at least tried to make the case for;

Publish for humans, all of the humans. But don’t forget the machines.

This time I’m going to talk a little bit about what we might get from those machines — because I’m not convinced it is always what people are expecting.

While it can be easy to compare our website to something like GOV.UK or the other statistical institutes around the world I often find it more helpful to compare it to something like the Guardian website. Functionally we are essentially a publisher of multiple story/report formats each made up of multiple components (words, tables, charts, interactive tools, maps, images, spreadsheets — lots and lots of spreadsheets) with collaborative, multidisciplinary teams working to strict deadlines.

So when I came across a report about the use of open APIs by news organisations (primarily the Guardian, New York Times and NPR) by one of the original authors of the Cluetrain Manifesto — David Weinberger — I settled down to read and learn.

After all the ONS Beta site is essentially a set of APIs with a user interface (albeit one where we have sweated over every button, label and interaction) and Florence, our publishing application, is the same. We have a commitment, maybe even a responsibility, to encourage the use of our (open) data and providing open, public APIs have long been held up as a way of achieving this. We have made the underlying JSON available from day one (visible by appending /data to any URI) and documenting what is possible/available is task fighting its way up the backlog.

“It was a success in every dimension except the one we thought it would be.” Daniel Jacobson, former Director of Application Development at NPR

One of the stand out findings from the report is that when they released their APIs (all within months of each other back in 2008) the big motivator was that ‘it would let a thousand flowers bloom’. Developers would see this as something on which to build. Like Rufus Pollock once said;

“The best thing to do with your data will be thought of by someone else.”

The reality however was somewhat sobering. Despite an initial burst of development and innovation those thousand flowers never really materialised. However what it did do was almost provide an outsourced R&D function — they could all see what ideas people had even if they weren’t really fully formed and this influenced the direction of internal development.

That is important as where the focus on APIs absolutely proved its worth was in supporting internal development. All the teams spoken to found themselves able to react much more agilely to development demands (the most obvious for all of them being the release of the iPad) where they had APIs to build on. The embodiment of ‘eating your own dog food’.

There were other wins that are interesting to us — the ease and flexibility of syndicating stories and assets improved, it became easier and quicker to experiment and prototype new features and it was possible to constantly improve their CMSs.

Now obviously these lessons might not transfer to us but it is worth considering. I think there is still an expectation that if we can get the API right there will be an explosion of apps using our data.

Robert L. Read, one of the founders of 18F in the US, certainly seems to think there is still a built in audience for Government APIs and that apriority should be to ‘democratize the data’ first and foremost because technologists will provide expert interfaces* to that data/service faster than Government will create the UI. Hhhmmmm.

The more likely, to me, ‘customer’ is likely to be more enterprise in scale and be looking to hook up to their own systems — people like Bloomberg, the Financial Times and Local Authorities spring to mind. This would/will be important but doesn’t really do much for supporting our open data agenda as such — but good set of APIs, with useful documentation and solid performance should make everybody happy — so if there is a chance for those thousand flowers to bloom we need to be ready.

*he seems to suggest that this interface could simply be an expert intermediary.