Voices of Authority: Towards a history from below in patchwork

This post is intended to very briefly describe a project I am about halfway through - that seeks to experiment with the new permeability that digital technologies seem to make possible - to create a more usable 'history from below', made up of lives knowable only through small fragments of information.

This particular project is called ‘Voices of Authority’, and is a small part of a larger AHRC funded project – The Digital Panopticon – that is seeking to digitise and link up the records reflecting everyone tried in London between around 1780 and 1875, and either sent to prison, or else transported to Australia.  This small element of the wider project is bringing together a series of different ways of knowing about a particular place, time and experience – the Old Bailey courtroom from around 1750-1850, and the experience of being tried for your life and for your liberty.  The conceit behind this project, is really a suggestion that building something in three dimensions, with space, physical form and performance, along with new forms of analysis of text; can change how we understand the experience of the trial process; and to allow a more fully empathetic engagement with defendants; along with a better understanding of how their experience impacted on the exercise of power and authority.

This project is only half completed – so this is very much a report of 'work in progress'.  But, in essence, what seeks to do is bring together three distinct different forms of ‘data’ and to re-organise that data around individual defendants.  

First, it takes the text of the Old Bailey Proceedings – the trial accounts of some 197,745 trials held between 1674 and 1913, and recognises them as comprising two different and distinct things – a bureaucratic record of the trials themselves (names, verdicts, punishments); and at the same time, one of the largest corpora of recorded spoken language created prior to the twentieth century – some 40 million words of direct, recorded testimony for the period under analysis.

These understandings of the Proceedings are, of course, built on projects of much longer duration; including the OldBailey Online, and more particularly, on Magnus Huber’s additional linguistic mark-up of the Proceedings, which allows ‘speech’ to be pulled from the trial text, and to identify the speaker along the way.  This is available via the Old Bailey Corpus.

The project also builds on text and data mining methodologies – including direct counting of word and phrase distributions, and the application of a form of explicit semantic analysis, that allows us to look at the changing character of language used in witness statements over the course of the eighteenth and nineteenth centuries.

 

In other words, the first element of the project is the text and speech, crimes, punishments and dates provided by the Old Bailey Proceedings.

The second element is the body of the criminal - the physical body of the individual  men and women involved.  The broader project is creating a dataset of some 66,000 men and women – with substantial and detailed information about their lives, both before and after transportation or imprisonment – reflecting the inter-relationship between the people who became defendants and criminals with the systems of a global empire. And this material provides a huge amount of data about bodies – to add to the words individuals spoke to power.  Height, weight, eye colour, tattoos among a range of other aspects of a physical self.  Suddenly, we know if a collection of words was spoken by a ten stone, 5 foot two inch woman with brown hair and black eyes, and a withered left arm; or by a six foot man with an anchor tattoo on his left arm, and a scar above squinting blue eyes. 

To think about it another way, this bit of the ‘project’, allows us to worry about the ragged boundary between the ‘physical’ as recorded in a set of numerical and standardised descriptions, and the ‘textual’ – the slippery and ambiguous content of each witness statement. 

In relation to ‘history from below’, this allows us to put together the lives of people like William Curtis, who as a 16 year old, in the summer of 1843, had a perfectly healthy tooth pulled, before stealing the dentists’ coat.  And Sarah Durrant, who was convicted for receiving two banks notes worth 2000 pounds. It all allows us to know their words (their textualities), and at the same time to see them as part of a different kind of truth – of place and body. 

The aspiration is to essentially code for the variabililties of body type at scale, to add a further dimension to both the records of the bureaucracy of trials (charge, verdict punishment), and the measurable content of the textualities of those same trials.

And finally, we are adding one additional dimension – space –  a ‘scene of trial’.

For this we are first of all building on a project called Locating London’s Past, which among other things, maps crime locations on to the historic landscape of London.  And to this we are adding a reconstruction of the courtroom, where all these trials took place. 

Simply using Sketch-up, we have made most progress on the George Dance’s building, finished in the late 1770s and providing the main venue for the relevant trials for the next hundred years – basing the models on the architectural plans from which it was built.  

In the process of creating this model, huge amounts of imformation about trial procedure has been revealed, including the changing layout of the court, and the relative position of the different speakers.  The design itself reflects a hitherto unacknowledged transition in the character of how witnesses and defendants were divided in this evolving space, evidencing a new story of the evolution of the criminal trial. 

The architecture itself, suggests that there was a clear transition from a situation in which witnesses and victims stood in a similar relation to the judge and jury (both facing the judge, relatively close to one another); to one - like a modern anglo-american courtroom  - where the judge and witnesses are on one side, and the defendant on the opposite side of the courtroom.  In other words, the character of the adversarial relationship at the core of the adversarial trial was re-defined, with the witnesses and victim re-located on the either side of the argument, and the judges role, redefined as arbiter between them.  At some level, in the process community resolution was replaced by court judgement. 

If you want to explain why conviction rates at the Old Bailey rose from under 50% in the mid eighteenth century, to over eighty percent at the end of the nineteenth century, starting from precisely the moment when the courtroom was rebuilt, this ‘fact on the ground’ needs to be part of the story.

What has also been revealed is the importance of levels – with lawyers speaking upwards to the judge, jury, witnesses and defendant, from a cock-pit several feet below their eye level.  Like a theatre audience, the judge, jury and defendant looked down on the stage below.  In other words, what was created, at least for a short time (70 years), was the real feel of a ‘theatre’ in which, as a barrister, you were forced to perform to the gods.  

 

Looking forwarding to the next stage, this particular sub-project is seeking to move from the ‘art’ of making and performance, through a humanist and historical appreciation of ‘experience’, towards the tools of social science and informatics – seeking to combine the close reading of a single desperate plea, with the empathy that can only come with physical knowledge, with that macroscopic image of all the similar words spoken over a hundred years – how that one plea fits in a universe of words and bodies.  And all of this, is in turn, being undertaken in pursuit of a more nuanced and empathetic engagement with the lives of working people - both for its own sake, and as part of a new analysis of the workings of power 'from below'.

With luck, this will allow us to move beyond a simple analysis of the courtroom, and the ‘adversarial trial’ - to an analysis through which we can see the whole system from the defendants’ perspectiv.

In other words, the next step is about creating a history of the British criminal justice system, and of transportation from an experiential perspective on a large scale – contributing to a history of common human experience, evidenced from the distributed leavings of the dead, analysed with all the approaches available to hand, from all the perspectives available.

Judging a book by its URLs

It will sound odd, but I have recently had a great time editing URLs.  Robert Shoemaker and I have have just finished a book for CUP, derived from the London Lives project, and called - London Lives: Poverty, Crime and the Making of a Modern City, 1690-1800. It is a long book (170,000 words) and each quote and reference in it is linked via a URL to the original document or article, book or web-resource used as evidence or to contextualize the argument.  It will be published as both an ebook and in hard copy, and the links need to be robust, and secure.  My estimate is that there are in the region of 4,000 URLs included in the manuscript (which was written collaboratively in PMWiki).  In the end, I found that I could identify an appropriate link for 98% of all footnote references, but then had to eliminate around 10% of these, as the relevant URL was just not useable.  The book took some nine years, and I am glad it is finished.

One of my final jobs was editing those 4000 URLs.   It took about three months work, spread over the last year, and I have just finished spending a week or so confirming what I hope will be their final form.  When I have told people about this work many have looked incredulous and suggested that this is the sort of technical implementation process that should be left to others.  A couple of otherwise nice people have suggested I dump this job on the shoulders of the nearest PhD student.  But for myself, it is precisely the kind of thing that an author should do for themselves.  And in doing it, two things kept coming to mind.  First was how the role of the scholar in creating a rigorous academic apparatus is a central part of the intellectual journey that academic writing involves - and that we should see the implementation of the online version of this in the light of the precise writing of footnotes and references that mark out good scholarship.  And second, that URLs encode a system of design and intent, online architecture and system of access, that signal the quality and permanence (the academic credibility and perceived audience) of historical materials online.  And that just as we have always sorted and judged scholarship by its form, we should think a bit harder about how the form of a URL can let us interrogate online materials.

On the first point, I do not know of much discussion of the joys of this kind of academic slog.  There is a lot of good writing on research and archives (by Carolyn Steedman and Arlette Farge among many others), on writing and thinking, but no-one talks much about the painstaking labour that goes in to turning a rough draft in to a final finished piece of scholarship.  And here I am really talking about generating accurate and fully comprehensive footnotes that reflect both the material cited, and the research journey that resulted in the main text.   This has become much easier with online catalogs and citation management packages, but nevertheless remains laborious and a reflection of our collective and individual commitment to a particular kind of evidenced discussion.  But for me it also represents my favourite compromise.  The writing of history is a wonderfully imaginative and creative process.  And in some respects we wish to judge the product of history writing as art.  Is it enjoyable to read? Is it convincing?  Does it do the job of good writing in liberating the readers' imagination?  In making these judgements we tend to appeal to a notion of 'value' that is cultural and that privileges dominant forms of authority.  This aspect of judgement is essentially romantic; with all the implications for western and elite hegemony embedded in that idea.  At the same time history writing is the result of simple hard work of a more technical kind - in the archives, in collating and collecting, re-ordering and interrogating data.  And it is valuable because it encompasses that hard work.  The beauty of the academic apparatus is that it evidences this and in the process generates a different measure of value.  In other words it is where quality is tied to a 'labour theory of value'.  I love the academic slog because it is where un-moored judgement is tied down to hard labour; and where value can be universalized in a common human experience (work).  In other words I really enjoyed editing 4000 URLs precisely because in them and their associated footnotes lies a claim to and evidence of the hard labour that underpins the book itself.

 At the same time, the process also taught me to read URLs differently.  Clearly coders and web designers do this as a matter of course.  But I am a historian and want to read URLs as a scholar, rather than as a programmer or designer.  And for me, the important thing is that URLs embed the structure of a site, making it plain to see for anyone willing to look hard; and that they are made up of both the character of a library reference, and a command directed at the new technology of discovery - the Internet .  There are just lots of different types of URL.

There are 'Search URLs' that include all the elements that  take the user past a collection to a specific object, but don't let you go directly there without the query.  And there are URLs that encode a cataloging hierarchy.  There are URLs that sift data, or work in your browser to change the data delivered, highlighting phrases or sifting material.  And there are URLs that encode licensing, passwords, and access information.  It is easy enough to find that the whole search journey that took you from a library catalog to an individual item is encoded directly in the URL, and even personalized to you, the machine you are using, or the forms of access you can deploy.  It is easy to find URLs that run on for hundreds of characters, each element divided by a '&' or a '%', or such.

But in creating robust reproducible links to credible historical materials most of these URLs are at least problematic if not useless.  If they include details for institutional access, or session information, they cannot be re-used by someone else.  These URLs are friable and fragile things and not fit for scholarly purposes.  And as a result, for the London Lives book we have been forced to eliminate all the links we originally hoped to include to forty or fifty different sites.  To take a single example, most archives structure their online collections with search in mind, making it difficult to link to a single item.  I spent a lot of time finding the catalog entry for every manuscript we cited in the London Metropolitan Archives, and Westminster Archives Centre, only to regretfully strip out the links when confronted by a complex URL that just did not look credible as a long term citation of the item itself.

Even in its simplest, and in the form recommended by the site for sharing a link, a London Metropolitan Archives URL looks like this:

http://search.lma.gov.uk/scripts/mwimain.dll/144/LMA_OPAC/web_detail/REFD+P69~2FBRI~2FB~2F001~2FMS06554~2F004?SESSIONSEARCH


Since we had consulted these items in their physical form in any case, it did not seem too problematic to leave out these links, but a shame nevertheless.  And likewise, with paywall material there seemed little point in dangling real access, and the promise of credible evidence, before the eyes of readers who would not be able to go beyond the login screen.  It seemed better to cite a specific item in combination with a general (unlinked) URL and date of consultation as reflecting our own research journey, rather than to promise access when we could not deliver it.

With few exceptions the URLs that have been retained (and there are still 4000 of them) address specific items with a specific ID, and usually run to 20 to 40 characters.  DOIs are not bad once you figure out their structure and reformulate them as they should be, rather than the way they are normally cited on journal web pages.

dx.doi.org/10.1353/sec.2010.0268

And Google Books creates a very nice URL once you strip out all the complex formatting instructions that are normally generated as part of a search and inserted after the main ID.  This is what a Google Books' URL looks like if you were to use the 'search' version:

 http://books.google.co.uk/books?id=1sMJGt7_rTAC&printsec=frontcover&dq=%22Prosecution+and+Punishment:+Petty+Crime+and+the+Law%22&hl=en&sa=X&ei=rrzGUq_aDsSy7Aa_9YGQCg&redir_esc=y#v=onepage&q=%22Prosecution%20and%20Punishment%3A%20Petty%20Crime%20and%20the%20Law%22&f=false

And this URL will take to the same book:

 books.google.co.uk/books?id=1sMJGt7_rTAC

 And the Eighteenth-century Short Title Catalog generates some of the most elegant URLs I have found:

estc.bl.uk/T174945

And to a lesser extent, so does the Ethos collection of doctoral theses at the British Library.

ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.354762

And London Lives and the Old Bailey Online do pretty well on this score:

www.londonlives.org/browse.jsp?div=LMSMPS501980014
http://www.oldbaileyonline.org/browse.jsp?ref=t17910413-19


In part, I suspect that these issues would all disappear if I had a better sense of the layer of structure that lies beneath the WWW.  But for the moment I am keen to have a short, human-readable URL that looks like it will last longer than the session I am currently logged on for.   All of which simply takes me back to the joy of academic slogging and the importance of the academic apparatus as something that evidences hard work and opens up scholarship to credible criticism that goes beyond simple romantic appreciation and prejudice.

I know all too well that one of the skills of an academic is the ability to judge a book by its cover and the form of the text it contains.   For the online we need to embed URLs into precisely this process - and the joy of all that editing was that at the end of it, I feel I have learned to do just that.

Hearing the Dead - Ten years of the Old Bailey Online

The Old Bailey Homepage, 2003

The Old Bailey Online has now been around for a decade - and we are celebrating.  But it also seems a good moment to take stock of what went right and what went wrong and just reflect for a moment on fifteen years of project work.

My most vivid memories are of the team of people involved.  They were amazing, and I feel hugely privileged to have worked with them all; most especially with Bob Shoemaker, who has been a constant collaborator and friend for over three decades, through at least five projects and two books.  But also a whole crew of people who came together to make something good happen.  Just in relation to the period before the initial launch in 2003, there was Jamie McLaughlin and Simon Tanner, Geoffrey Laycock, Louise Henson, John Black,  Edwina Newman, Kay O'Flaherty, and Gwen Smithson. What they produced was ground breaking, and that was a simple result of their hard work and good sense. Since then a dozen more people have been involved - most importantly Sharon Howard. 

I suppose the real  question is: if we had to do it over again, what would we have done differently?  Speaking for myself, rather than for the project, I can't think of all that much I would change.  Perhaps we could have chosen better software to begin with, and been generally more technically saavy, and worried less about IP.  But these seem small things from this distance.   I guess I regret the six months of my time, and several thousand pounds of project money spent licensing the images we used for the background pages on the site.  The experience demonstrated to me that the copyright system in online images is broken, but it seems a hard lesson won at a relatively high cost!

I also regret over egging some of the early discussion about the project.  The whole 'new history from below' narrative which we developed in thinking about the site simply raised the ire of a group of historians offended by the hubris; or who felt their own expertise was somehow threatened.  I hold my hands up to the charge of hubris (though I still stand by the need for a 'new' history from below).  Digitisation and the Internet, and Digital Humanities more broadly does lend itself to hyperbole, and I am far from immune to its attractions.  But mainly it was unnecessary, as the site has effected the wider historical agenda and the kinds of history people write with no need for any one to lead the way or yell about it.

And what we got right - however fortuitously - seems to me to outweigh these issues.  Looking back on the decisions we made in the late 1990s, the choice of a double-entry rekeyed text, in combination with XML tagging turned out to be perfect, even if it was through luck rather than expertise (though Michael Pidd's experience and initial steer helped!).  We also got the timing right. Because it was 2003, because we started the project in 1998/9, the launch received a lot of news coverage, and generated what seemed at the time a ridiculous amount of usage - without us having to pursue the kinds of detailed 'impact' plans the funding councils now demand. 

But mainly, I like to think what we got right was our decision and commitment to digitise the most compelling source of social history I know; of history from below.  I am continuously moved by the fact that lots of people now read eighteenth-century trials who would never preoviously have thought to seek them out.  Professional historians have always known what powerful voices the Proceedings contain, but putting them online in a form that is easy to use and free has meant that millions of people who would not otherwise have been minded to read this stuff, have done.  They have used what they found in the Proceedings in novels and on television, in endless undergraduate dissertations and in more books than I want to read; and I take neither credit nor blame for their work.  But, I believe that the decision to make freely available a source that prior to 2003 could only be read by a small and privileged group of academics, was an unproblematic good thing. 

Bob Shoemaker (right) and Tim Hitchcock, 2003.  Clock the monitor!

Most people forget just how elitist and exclusive academic libraries could be (and still can); how hard it was to get access to microfilm, special collections and rare books.  Unless you had been vetted (largely for class and race) by some great institution, you were simply out of luck. I like to think the Old Bailey Online undermines that exclusivity just a bit.  I also like to think that in combination with London Lives the site has helped to make a corner of social history I love, and have committed my life to studying, a little more secure from the infinite condescension of the present.

In the next ten years I very much hope that the Proceedings will form the basis of a growing body of more technically sophisticated analyses, using all the techniques of datamining, corpus linguistics and information science, all made easier by the API.  But most importantly, I hope that people continue to hear the voices of anger and pain it contains; and that for just a second they let their imaginations take them to that brutal theatre of judgement where working people were forced to negotiate with power.

The Old Bailey on Steroids

The text of a talk delivered at the Humanities Research Institute at the University of Sheffield on 21 May 2008, as a part of the formal launch event of the Old Bailey Online, 1674-1913 (www.oldbaileyonline.org)

When in 1999, Bob Shoemaker and I sat in what was then the new British Library and first discussed the possibility of creating an online edition of the Proceedings, it seemed to us a good idea. From conception, to getting the money, to the hard slog of digitisation, to the equally hard slog of tagging and preparation, to the second tranche, in partnership with Clive Emsley and the Open University, to the ever inventive crises of implementation, to the whirlwind of publicity, and the equally chaotic whirlwind of online delivery, I still think it was a good idea.

But, along the way it has become something rather better than the idea we started with. In some ways, even the present on-line edition of the Proceedings the people in this room have made a reality, is an extension of an essentially Enlightenment notion of knowledge and publication. It is a massive serial body of text, indexed and structured, huge in its compass, but essentially recognisable to any historian who cut their research teeth on the old Times indexes, or who built upper body strength hefting the catalogue in the old Round Reading Room. As a result, in its current form, the Proceedings are a beautiful culmination of an old fashioned idea. Perhaps ironically, they are just so last century.

But they form the starting point for something else. The promise of online, searchable, mash-upable, analysable, deracinated text is that it offers the opportunity to do something, not just better, but different. We have yet to see the first volume of twenty-first century history; of internet enabled history. We have yet to see the first major piece of historical writing that could only be conceived as a result of the existence of online resources; but it is the promise that such a new historical form might be created that is now driving the evolution of the Old Bailey.

Through the ESRC funded Plebeian Lives and the Making of Modern London project, which we have spent the last three years helping to make happen, and will spend the next two bringing to some kind of fruition, we are taking the scripts of criminality - the events of a trial, or the retrospective biography of an Ordinary’s Account, the things we have – and adding to them 40 million manuscript words of the scripts of both poverty, and just plain normality.

This new Old Bailey on steroids will encompass eight coherent manuscript archives – the records of St Thomas’s Hospital, the Coroners’ Inquests for Westminster, the parish records of St Clement Dane, St Dionis Backchurch, and St Botolph Aldgate, and the records of the Middlesex Bench - Pauper examinations and depositions, account books, and a massive archive of rotting, quotidian paper. And to this, it will add the electronic leavings deposited by a generation historians in the AHDS; the Cantebury Wills, the voting records of Westminster, the Hearth Tax Returns. And to this too, the death and marriage and birth registers created by shoals of family historians and published as CDs.

No one has ever attempted to digitise manuscript materials of this complexity before on this scale; and more importantly no one has ever managed to order this vast complex legacy of ill-digested record keeping, into a single searchable thing. In undertaking this project, we are building on the skills created in the long process of making the Old Bailey happen. Without the world beating experience built up by the Higher Education Digitisation Service in double entry rekeying, in creating usable transcripts of every kind of eighteenth-century hand imaginable; we couldn’t do Plebeian Lives. Without the expertise of Jamie McLaughlin, without his experience of designing search facilities and making them work, we couldn’t do this project. Without Sharon Howard’s management and technical skills, and simple commitment; we couldn’t do this project. Without Ed McKenzie and Kathy Rogers, without Philippa Hardman and the HRI, we couldn’t do this project. In other words, we have taken a body of expertise built up across a decade between three Universities, re-imagined what could be done and turned it to a new and complex purpose.

The really new thing about Plebeian lives is that it relies for its very sense on the ability of search engines to find a single word in a sea of text. If we could not sift these sands of time for historical meaning at the click of a mouse, there would be no point in digitising these archives – better to leave them in their original form. But, if we can search them, analyse them, link them across archival spaces that have hitherto been unbridgeable; we as historians can move beyond scripts that retell limited stories of admittedly gripping and dramatic events created essentially for an eighteenth-century audience; to telling life scripts, to building life lines that snake through the archives, to evidence each interaction between a beggar or a criminal, or just a person, and the systems on which they relied, or feared, or avoided - to map a collection of lives.

The Old Bailey Proceedings were a good idea because they brought into the digital age the single most important and classic source for history from below, for that humane tradition of writing from the bottom up, and from the single individual’s perspective.

The point is now to move beyond that tradition, to what is only now conceivable. In my own mind, this amounts to adding a collective rigour to the humane narratives of History from Below; to – if you will excuse the historiographical contrast – forcing Raphael Samuel to sit down with Lewis Namier; and then figuring out what they together might have to say to Karl Marx. In the process, what the project seeks to achieve is a new way of understanding how individual decisions, made by non-elite men and women, contributed to the evolution of the institutions of modernity – of hospitals and workhouses, prisons, courtrooms and local government. And it will do so by tracing people’s lives through this massive archive; and in the process will argue that the agency deployed by working people, the decisions they made in difficult circumstances, help to explain why and how things changed, both in the past and by extension in the present.

But more than a new intellectual aspiration, a new explanation of historical change; this project is also about recognising that the audience we need to reach is different than it once was. The point about building upwards from individual lives, is that it allows us to connect in ways that most historians cannot, to the greatest body of readers and historical researchers ever – to the family historians. By making this project about lives, we generate something they want to read; at the same time as we analyse something we want to explain.

And as the tools of social software gain a new maturity; as Sharon Howard adds a wiki to the Old Bailey site; as we create facilities for our users to provide new content; to comment on, correct and add links to the resources we create; and more tentatively, as data mining and the searching of distant distributed resources becomes more achievable; a growing and organic intellectual project is gradually being brought into existence. It will have its monographs, articles, and authors; but it will also have its blogs and mash-ups, and its communities of users. In the process it will tie the unbounded historical interest in the everyday, to the intellectual exercise of academic history; and create the infrastructure that allows us to communicate beyond the academy.

Essentially, and in other words, what began as an attempt to make a humane, individual history from below easier to research and write; has become an attempt to add a new layer to that tradition; to update it in light of the full promise of digital resources, and to make it fit for the politics of today. The project is now about rediscovering the magical combination of narrative, and individuals, and politics that characterised the best progressive histories of the 1960s and 1970s; and adding to those the magic of digitisation and the internet, and turning the whole lot to a new purpose.

One of the great arts of academic life is to trust that an interesting, important conclusion will emerge at the end of a project, will present itself at the end of five or ten or twenty years of hard slog – and that it will be important even if it turns out to bear little relationship to the question with which you began. I still like the idea with which we started, but I have to admit that it has become a shed load more interesting along the way.