Peer Review Smells.

Most academics who have thought about it will agree that peer review is just rubbish. It is ridiculously hard work - work volunteered by academics as a mulct on their lives. And it is subject to the politics of the playground. Most subjects and academic communities are simply too small to allow true anonymity, and as a result the outcome of peer review is frequently informed by nothing more than hubris and hurt feelings.

My personal bugbears regarding peer review are that, first, it is inherently conservative - encouraging a narrow definition of scholarship; and second, that it forms a comprehensive subsidy on publishers that in turn shifts money that should be spent on either research or dissemination into the coffers of private companies. Universities pay for the privilege of buying journals that are written and edited by their own staff, and reviewed at their own expense. In an online environment, where the costs of printing is decentralised, this makes no sense whatsoever.

At the same time, there are aspects of peer review I like. I want to know that someone with some expertise thought an article was worth publishing. It saves me the time (the seconds it takes to skim a few hundred words) otherwise spent on determining whether the material being discussed is worth the effort of reading.

The traditional answer to these complaints and observations is the care-worn observation that peer review is the worst system ever, except for all the others.

But, with the advent of new models of online sharing and community interaction, it seems to me the moment to reinvent this particularly irritating wheel. Why not create an open 'academic journal space' for all articles that anyone cares to post, with a tagging regime for subject definition. In other words, anyone could post under the tags 'eighteenth-century British history', or indeed 'past & present'. If editors, or foundations, chose to shepherd particular collections of tags and to create an 'identity' from this association, this could certainly be accommodated. Moves in this direction were commonplace in many science subjects in the 1990s, but it hasn't resulted in a reform of the world of journal publication.

The point, however, would be to ask all contributors (all people - even academics - who choose to post a contribution) to also undertake a series of peer review assessments. These would be post-publication, but since online material could be continuously edited, it would nevertheless act in the way current pre-publication peer review acts in relation to polishing an article.

The point, however, would be to rate the assessors. I am more interested in what Natalie Zemon Davis thinks about an article on 16th century France, than I am in the opinion of a third year undergraduate (at least in the first instance). By allowing assessors to themselves build up an online profile you could incorporate a continuum of quality that is made up of the opinions of all assessors, weighted according to authority (i.e. the opinion of three senior figures in a field, could balance out that of 12 graduate students, or 18 unrated individuals, or whatever). The rating of individual assessors, could be determined by what they have contributed in the past, or based in a metrics of cross posting of the sort being used by the Australian and UK higher education systems to distribute research income. There are real problems with metrics, but they do form a consistent measure of community regard (at least in most subjects).

The result would be an academic commons that nevertheless preserves the 'authority' that is the single valuable aspect of peer review. It would also open up academic publishing to a broader community. There is no reason for 'outsiders' not to post material. Much of it might be denigrated by 'high authority' reviewers, making it possible to still select out articles of a certain 'standard', but it would also create a way in of a sort that doesn't frequently exist in the current system. Groups of reviewers who choose to generate a 'journal' identity, could still do so - with the 'journal' being made up of those articles that a defined group of 'assessors' chose to assessn positively.

As an evolving community of assessment and contribution, the system would be self-validating, and a lot cheaper to run than the plethora of journals currently clogging our libraries and lives.

To rot in a God-made world.

A synopsis for a contribution to a panel discussion to be delivered at the British Society for Eighteenth-century Studies Conference at Oxford in January '09.

The rancid smell of decay; the constant putrefaction of a world of wood, of oil, of fabric, burned sharp in the minds of eighteenth-century people. To create a pre-industrial society that worked, every bit and piece of the man-made world required attention on an almost daily basis. It was this fundamental material reality that underlay most working class notions of the world. In popular biblical and medical conceptions of the body, in the hard and bigoted landscape of ‘civilisation’ and ‘barbarity’, in the fear of the wild, and in the innate attractions of the urban, and the farmed, is found a headlong retreat from chaos – a retreat enacted with every laborious brush stroke of a housewife at her step, or a carpenter with his chisel. Sharp and vertiginous divisions of class and gender and place, divided eighteenth-century people one from the other; but underpinning this was a shared material experience that tied the brick maker, the philosophe, the beggar, the farmer and the hopeful mother into a single unending struggle to wrest order from an all-consuming nature.

The smell of rotting chains.

Jo Guldi has been blogging recently about folksonomy and 'navigation in chains', and has called for a open ended approach in which users tag content, and in which free standing sites are opened to the manipulation of a public audience. But, opening the 'archive' to its users is only an interim solution. The problem is deeper than this, and lies in the very notion of 'keyword searching', and in search based on structured tagging as well. Both are very blunt instruments, and simply re-enforce an older style of iterative research. As a result, the search engines created by free standing sites, of which Guldi complains, are themselves sad interim solutions to new problems, and will wither as new ways of searching are created (the chains will rot in short order). Tagging, is again, just one more interim technology (a strategy derived from the 1980s, and well past its sell-by date). All of these creations are based on the notion of the 'library', on ordered information and the existence of an 'index', and Guldi is simply arguing about who should have the right to order it. I believe this is all just so much renaissance detritus (a worthy subject of study, but not a working methodology). What is missing are the new tools that allow you to do things differently in the infinite archive. With 100 billion words of digitised text (whatever the actual number becomes), I want to find the patterns that I cannot imagine, and which even an infinite army of folksonomic taggers could not reveal.