Historyonics

The blog entries were all much easier to read than...

2012-02-25T16:03:32.090-08:00

The blog entries were all much easier to read than a book, although i don't pretend to understand all the terms and some of the concepts went 'over my head'! The Aiden +Michel culturonomics/ngrams article stands out for me at the moment; and also the information about Locating London's Past and London Lives. It makes me think of London in a different 'light', not just a dusty, old and polluted city, but a city with a colourful and ancient past, and probably an intriguing future- can we visualise it without the hot and stuffy tube/underground transport please?! Shirley-digitalcentury

Really interesting talk. I appreciate you raising...

2012-02-06T07:58:23.455-08:00

Really interesting talk. I appreciate you raising the issue of the relationship between what we might call algorithmic history (to adapt Steve Ramsey's language) and digital public history. I was left wondering if the kinds of visualizations you and Bill Turkel developed and interrogated could be used for public history ends just as effectively as the Locating London's Past map. Last year we launched a modest project on the Virginia Secession Convention proceedings. It's by no means "big data"—about 3000 pages of text. But even at its modest size we realized the amount and nature of the prose meant that it wasn't going to be used in, say, a high school class if we just threw up the text alone. So, we developed some visualizations—maps and timelines—for searches that, we hope, help readers navigate through and ask questions about the proceedings without having to read substantial amounts of text. (Though I certainly don't want to encourage people not to read.) We hoped the data visualizations here would not inhibit but encourage a broader public audience to engage with these materials and this history. I'm trying to do something similar in "Mining the Dispatch", as I said in the AHA panel you were at in Chicago. That is more substantial in size—24 million words. While I wouldn't describe that as a public history project, I do hope that the interactive topic modeling visualizations on the site function as heuristic tools that can help people investigate and study what is an otherwise unwieldy sized corpus. That big challenge posed by big data in terms of public history, it seems to me, is to develop interfaces and visualization tools that engage the historical imagination of a broader public, that encourage historical investigation, and that don't obscure all individual voices and the "weird" through "distant readings" but enable readers to move into close readings by channeling them to the texts that address their individual interests and questions.

Thanks for this very interesting and personal view...

2012-01-31T18:38:27.251-08:00

Thanks for this very interesting and personal view on digital big data. I really appreciate your "narrative", from the beginning to now. We need more essay like yours to understand the evolution of history and story (in french, there is just "histoire" to said both!)
I didn't know the Old Bailey before your works. For me, it's big data, yes, but also "Deep Data", data about "few" historical subjects and topics. The global human history will continue to be an addition of national, regional, personal datas and narratives. And I don't think that any kind of big data will change it... (from Montréal, Québec, Canada)

It was a pleasure to read this, Tim -- particularl...

2012-01-31T16:43:22.274-08:00

It was a pleasure to read this, Tim -- particularly because you expose your own ongoing process of reflection so candidly.

I have some of the same hesitations. And I work in literary rather than social history, so the problems I confront are somewhat different.

But in the literary domain, at any rate, I am not finding that "big data" necessarily pulls me away from "close analysis of individual works."

Admittedly, it can do so. But there are ways that big data can do a lot to illuminate individual works or authors. Topic modeling, in particular, strikes me as a methodology with this sort of double-edged potential. I find that I often learn a lot about individual works by topic modeling a large dataset, because topics are defined by the differences between documents, and can help me see how a given document is concretely *distinguished* from its the background.

Tim, Thanks for getting back. For me the fact tha...

2011-12-14T03:41:50.240-08:00

Tim,

Thanks for getting back.
For me the fact that the first two are red and green is the main problem. Red > Blue > Yellow would be more helpful. But the workaround is fine, and hasn't put me off exploring the site.

James

Dear Dr James, I am afraid at the minute we are ou...

2011-12-13T09:24:00.000-08:00

Dear Dr James, I am afraid at the minute we are out of money and time for this particular site, but let me log it for the next upgrade. I find the easiest way of scrolling through the colours is just to map the same old data a second or third time till you get to a colour that is useful (pretty much as you suggest only easier), and then remove the mappings that are not useful. All of this tends to be part of the Google Maps container - though we do have some control over it, and have eliminated some colours that we could not interpret easily. If it is useful, as far as I can tell the site results are produced in the following order:

Red
Green
Blue
Green/Blue
Pink
Orange

However, I can quite see that this would be irritating! And will try and get something done about it.

Tim. Your post reminded me of a question I had be...

2011-12-13T08:39:16.773-08:00

Tim.

Your post reminded me of a question I had been meaning to ask since I first got my hands on LLP.

I am colour blind (red-green, green-brown, blue-purple). As a result I need to be able to select the colours I am using when generating data on the LLP maps in order to interpret them easily. I have managed to work out how to trick the site (generate a query you want, query you don't want for a colour that clashes with the previous query, generate a query you do want afterwards, then go back and delete the unwanted query), but I wondered if there were any plans to either:
a) allow users to choose their own colours for queries, or
b) include unique symbol markers for each colour (videogames have done this rather well over the years so red - say for threat - can be distinguished against a green environment).

Other than that I've been thoroughly enjoying trawling through LLP.

Thanks for sharing your info. I really appreciate ...

2011-12-01T03:56:09.387-08:00

Thanks for sharing your info. I really appreciate your efforts and I will be waiting for your further write ups thanks once again.
Vee Eee Technologies

Dear John, There are all sorts of problems with &#...

2011-10-24T06:06:10.351-07:00

Dear John, There are all sorts of problems with 'impact' - both methodological, and idealogical, but no one ever spent a dull life of lonely scholarship because they believed it had no impact. The only real question is how you want to describe the sort of 'impact' you think is important, and then what metrics you want to use to measure it. I believe academic history is important as a basis for an informed national political discussion - so let's find ways of measuring this (or whatever aspect of history writing you think we should value). Unfortunately, what most academics who oppose impact are currently doing, is claiming to be above the necessity of actually examining and justifying what is important about what they contribute.

"In Britain we shy away from the metrics that...

2011-10-24T01:44:50.507-07:00

"In Britain we shy away from the metrics that would demonstrate ‘impact’ primarily because we fear that we may not have any."

Is this really so? Isn't the problem that the REF & RAE measure impact through a narrow selection of metrics, and a set that is heavily weighted towards 'old school' scholarship, discounting much of the digital?

And doesn't 'impact' have an ideological aspect, favouring the established publishing system and usefulness for government and business?

I've heard 'public engagement' counterposed to it as a different way, wider in medium and audience, of thinking about these matters.

Dear Laura - I agree with you about the current cr...

2011-10-24T01:19:17.599-07:00

Dear Laura - I agree with you about the current crop of book simulators. They are rather stupid, and don't let me have the kind of dialogic relationship with text that I want. Why shouldn't my 'reader' know what I am working on, and privilege that subject in the text (perhaps emboldening passages it thinks are relevant). Why shouldn't I be able to create a hierarchy of notation, that reflects how I read and re-use text? Why shouldn't each note I make be embedded within a map of what I have been reading on that day, or week, or for that project? I want all the things a book can deliver, and I want a shed load more. But, one way or another, the evolution of e-texts throws in to ever sharper relief the limitations of the book as a technology.

Laura, try writing marginalia in a British Library...

2011-10-23T23:29:28.555-07:00

Laura, try writing marginalia in a British Library book. These online resources are borrowed from a repository and have the same practical restrictions as library books.

If you need a tool that can help you take notes, I'd suggest Zotero.

Tim, I liked your piece, but I don't feel read...

2011-10-23T18:54:05.174-07:00

Tim, I liked your piece, but I don't feel ready yet to bury the book. One reason is that I'm not yet satisfied with how electronic interfaces allow me to read books actively: i.e., mark them up, write marginalia, insert post-it notes. It may be that I just haven't transitioned yet to the new media, but I also don't feel that the new media yet gives me the opportunities I need to interact in a more dialogic form with what I'm reading. I wanted to ask if you really think that you can electronic forms of reading really is where it needs to be right now.

Also, loved the comments on the Burney collection. What's the solution?

Laura Stevens

Might screen grabs work as citations for Google se...

2011-10-23T15:13:00.202-07:00

Might screen grabs work as citations for Google searches? Doesn't deal with replication issues…

2011-10-23T15:11:48.983-07:00

This comment has been removed by the author.

The issue is not that these different forms of evi...

2011-10-23T08:55:59.812-07:00

The issue is not that these different forms of evidence and research experience militate against writing good history; but that we do not have a system to critically engage with the online, in the way that we do for hard copy books and archives. I know how to cite a medieval manuscript (wherever it is); but I don't know how to cite (and replicate) a Google search.

Whilst I agree that to some extent digitised resou...

2011-10-23T08:26:18.406-07:00

Whilst I agree that to some extent digitised resources are drawing people away from using traditional resources, I feel that there are two arguments against the points you made. Firstly, I feel that anything to sate the appetites of the general public and history is ultimately a good thing. It encourages a more broad narrative.
Also, it is possible for digital resources and traditional literature to be used concomitantly. As a student used to the archives as well as the eighteenth-century collections online, I recognise that each resource brings its own uses and evokes differing emotions. Whilst the digital resources allow fast searches and easy open access, nothing can beat finding a primary source in its original form. Perhaps these two media complement research when used together rather than being seen as stultifying.

Dear Jean-Baptiste, Thanks for your comments on th...

2011-06-20T01:29:22.581-07:00

Dear Jean-Baptiste, Thanks for your comments on the post. On the issue of citations - the point I was trying to make was that there are no directions for citing a graph generated using the ngram viewer, as opposed to citing your articles. This is a wider problem than just with the ngram viewer, and until humanists figure out how to cite a search and its results in a repeatable and credible manner, we will be practising increasingly poor scholarship.

On the wider issue of your own and Erez's engagement with the humanities and history; the situation is somewhat out of your control, as the science press has emphasised those aspects of your work that imply a new and newly 'scientific' approach. This is an emphasis that has a powerful appeal to those given to a crude factology, and reflects the continuing distance between approaches to knowledge. There is clearly a sizeable chasm between your own practise and how it is represented. But while you remain at the stage of demonstrating a methodology, rather than using it to write history, the issue of how your practise works with other and older forms of scholarship, will remain - and is worth interrogating.

Let me also re-iterate how much I admire your work.

What I very much hope will emerge in due course is some great history that uses your techniques and methodologies to evidence society's understanding of the past. And I very much hope that you and Erez will be the ones to write it.

My experience of the Digital Humanities community as a whole is that it tends to enthuse over new tools and ways of visualising data, without being sufficiently concerned to critique the usefulness and purpose of the wider project, or to relate that project to the functions of humanist and social science scholarship. In many respects the criticisms I have made of your project are just as true of the wider Digital Humanities community, and form a topic with which I am continually struggling in my own work.

All the best, Tim

Elijah: bingo.

2011-06-19T17:44:36.952-07:00

Elijah: bingo.

I wish the DH community could be as openly critica...

2011-06-19T10:54:17.961-07:00

I wish the DH community could be as openly critical of work done by self-identified DH practitioners as they are of Culturomics.

(Part 3 of 3) You wrote: "Historians interpr...

2011-06-19T09:36:14.834-07:00

(Part 3 of 3)

You wrote:
"Historians interpret the past to the present. They marshal evidence and use all the tools of genre writing to allow a modern reader to engage with the past. And the questions they ask are not driven by the evidence, but by the needs of a modern society...The fundamental flaw with culturomics is that it assumes that history is about the past..."

We make absolutely no such assumption. We are agnostic about what motivates a person's questions. What interests us is the process by which scholars 'marshal evidence'.

Data is evidence. Just as those who read Akkadian can skillfully marshal Akkadian primary sources, those with quantitative skills can skillfully marshal data. Our goal is to contribute data and methods so that this new form of evidence can thrive.

All the best,
Jean-Baptiste Michel
Erez Lieberman Aiden

(Part 2 of 3) You wrote: "At its heart, the ...

2011-06-19T09:35:47.320-07:00

(Part 2 of 3)

You wrote:
"At its heart, the use of word frequency with a reasonably sized (if problematic) data set simply provides one more form of evidence to be added to all the rest...But simply providing a new body of evidence is not what seems to get Nature going. Instead, it is the claim that the ngram viewer lays the basis for a new 'science', and that the results make other forms of historical analysis redundant."

We don't understand why this straw-man keeps coming up, as we have been unambiguous about this point in the past: n-gram results are a new form of evidence. They do not make any extant method of historical analysis redundant. Here are a few primary sources.

From the paper:

"Culturomic results are a new type of evidence in the humanities."

From the very first line of the Culturomics FAQ:

"1. Is this supposed to replace close reading of texts?
Absolutely not. Anyone who has appreciated the work of a great artist - say, Shakespeare - or an insightful scholar - say, Michael Walzer's Exodus and Revolution - couldn't possibly think that quantitative approaches can replace close reading.

Quite the opposite is true: quantitative methods can be a great source of ideas that can then be explored further by studying primary texts.

2. How does this relate to other methods in the humanities?
Our hope is that the culturomic approach will be able to supplement existing techniques."

From the current Nature piece:

"
A.
And yet [Erez] doesn’t think that the old approaches will ever disappear. “I think you should use the best methods available — and all of them,” he says. “And I think that includes carefully reading texts and trying to get behind what authors think.”

B.
"…[Erez] tells the story of Isaac Casaubon, a sixteenth-century Protestant scholar, who undermined the presumed
Egyptian provenance of a set of religious texts by identifying a reference to a Greek play on words — something that could only have been written hundreds of years later. “That point is as objective an interpretive remark as any remark a scientist might make,” says Lieberman Aiden. “So the methods of humanists are very, very formidable. And I think the degree of insecurity they have over whether these methods are here to stay is not really befitting.”
"

We think these texts make it unambiguously clear that we have absolutely no intention of replacing existing methods. There is no viable alternative reading of our statements in this area, and the attribution of this attitude to us is simply incorrect - a Casaubon delusion.

Our goal is no more than to enable such data to provide - in your exact words - "one more form of evidence to be added to all the rest".

---

Hey Tim - Interesting thoughts, It's great to...

2011-06-19T09:34:11.301-07:00

Hey Tim -

Interesting thoughts, It's great to see people diving into the discussion, and especially how much the discussion has changed in only six short months!

A few comments, in three parts.

---
You wrote:
"This use may be delayed by the peculiar lack of any guidance on how to cite the results of a search, but it will be normalised in due course."

Actually, in the "About the Ngram Viewer" (http://ngrams.googlelabs.com/info) section, we write:

"If you're going to use this data for an academic publication, please cite:

Jean-Baptiste Michel*, Yuan Kui Shen, Aviva Presser Aiden, Adrian Veres, Matthew K. Gray, William Brockman, The Google Books Team, Joseph P. Pickett, Dale Hoiberg, Dan Clancy, Peter Norvig, Jon Orwant, Steven Pinker, Martin A. Nowak, and Erez Lieberman Aiden*. Quantitative Analysis of Culture Using Millions of Digitized Books. Science (Published online ahead of print: 12/16/2010)"

All I can say is "bravo". And thank you....

2011-06-19T07:38:54.218-07:00

All I can say is "bravo". And thank you.

A ‘new history from below’, which aims to use the ...

2010-04-19T09:46:02.751-07:00

A ‘new history from below’, which aims to use the variety of sources now available online to produce engaging and accessible writing (a rare commodity in academia), is an exciting and challenging project.

Any 21st-century initiative involving poor Londoners needs to free itself from the stranglehold of Dorothy George’s London Life and the work of historians who, over recent decades, have continued to re-cycle her vision of eighteenth-century London. George, who never actually defined poverty, described a squalid and vice-ridden city, ‘orgies of drunkenness’ during the thirty years of the gin craze and a century of death and disease, misery and degradation among London’s poor. Her work, imbued with the attitudes of her own day, was based largely on the rhetoric and propaganda of eighteenth-century social reformers, who all had their own agenda. We need to move on and seek a more realistic and balanced view.

History is a continuing story, or rather a multiplicity of stories, which deserve to be told and not only in terms of analysis and statistics, where the reality of lived experiences can be masked. Internet sources allow historians the freedom to explore something of that reality and the wider world of London’s poor, to rescue some of the neglected and misrepresented and to write about them in a lively, imaginative and well-illustrated way. Despite remonstrations from the arrogant, the cynical and the ill-informed, online sources, carefully handled, offer endless exciting possibilities.