Show your workings: a digital statistical publication

Russell has written a post that touches on some of his thinking about what a ‘digital white paper’ might look like and in doing so draws attention to Bret Victor’s tour de force of a ‘longread’ about climate change. The real brilliance of the work by Victor is that not only is it wonderfully interactive but it also fulfils that old staple of maths classes; ‘show your workings’.

Given where I work, my primary project and my recent reading & writing it probably isn’t a surprise that I have found this interesting.

One of the things I keep noodling with in my spare moments is what might a truly digital statistical publication look like. To be honest other, better qualified people, are looking at more immediate, practical responses to that question whereas I am really using it as something on which to hang various ideas and hunches about the future of digital publishing to give things some kind of structure.

So the ability to expose the methodology behind a particular statistic and make that explorable in place might make for an interesting experiment. Our user research has identified that there is an expectation that our statistics are methodologically sound above and beyond what is perhaps expected elsewhere and making that visible (it is always available and on our new site much more obvious) would provide pretty radical levels of transparency.

There is almost certainly something that can be learned from ‘open science’ here and in particular ideas about ‘open notebooks’. The more transparent you are the more trust you build in the results. That said we have very important disclosure rules to consider at all times so it isn’t as simple as providing all the underlying data to allow truly replicable ‘experiments’.

Our QMI documents (for example) provide a great source of information already but they are far from ‘digital first’ with most of the pertinent information locked away in a PDF. The challenge would be surfacing that in an ‘of the web’ rather than ‘on the web’ sort of way.

We already do a better job than Russell’s complaint about white papers;

“tables and the diagrams you get are included for the rhetorical power of their presence rather than any explanatory work they might do”

..and every report (we actually call them Bulletins but that is another blogpost) comes with a whole supporting ‘reference tables’ in Excel but it still feels a bit disjointed and the real power (I think) would be presenting the combined narrative and the data seamlessly and in a way where it can be queried and explored (while still providing the data free from words from those who like their statistics straight with no mixer.)

Given my role the big thing I am always thinking about with these ideas is are they repeatable. I have no interest in trying to provide a system that can support a thousand unique snowflakes (or god help us Snowfalls) so an additional challenge would be creating something that could work across multiple outputs.

Maybe. Anyway.

Pretty much at the same time as my writing this Leigh Dodds wrote a complimentary post that shows just how this kind of development could make things better.