Showing posts with label research funding. Show all posts
Showing posts with label research funding. Show all posts

Tuesday, 15 July 2014

Doing it in public: Impact, blogging, social media and the academy


The text below is derived from a short talk I gave in February for the Library at the University of Sussex.  At the time (and in the text) I promised to post it as a blog, but never quite found the time. 


Impact is an awkward thing in British Higher Education.  Most of the time it feels like just one more bludgeon used to batter hapless academics into submission.  It is frequently shorthand for an agenda handed down from on high, privileging near-market research and the agendas of government.  And yet no one spends a lifetime researching, teaching and writing about something if they don't believe it is important - if they don't believe that what they do contributes to a better world. We all want to have 'impact'.  The question is how can we do so in a way that reflects our own values, rather than those of whatever government happens to be in power this week?

This question is all the more important because our traditional assumptions about how our work effects a broader social discourse seem increasingly threadbare.  When the print run of most monographs number just a few hundred copies (most of which disappear in to American research libraries, never to be read or used), and when journal articles proliferate beyond number because they serve the needs of big publishing, rather than academic dialogue - we need to think harder about how we do the job of the humanities.  If we simply continue in an older vein - having small (vociferous) conversations amongst ourselves, in professional seminars and at conferences, through book reviews and in the specialist hard copy press - we will lose our place in the broader social dialog. If there is a 'crisis' in the humanities, it lies in how we have our public debates, rather than in their content. 

It seems to me that the solution to this problem is all around us, and that in order to address it, we need to remember that the role of the academic humanist has always been a public one - however mediated through teaching and publication.   By building blogging, and twitter, flickr, and shared libraries in Zotero, in to our research programmes - into the way we work anyway - we both get more research done, and build a community of engaged readers for the work itself.  We can do what we have always done, but do it better; as a public performance, in dialog amongst ourselves, and with a wider public.

The best (and most successful) academics  are the ones who are so caught up in the importance of their work, so caught up with their simple passion for a subject, that they publicise it with every breadth. Twitter and blogs, and embarrassingly enthusiastic drunken conversations at parties, are not add-ons to academic research, but a simple reflection of the passion that underpins it.  

A lot of early career scholars, in particular, worry that exposing their research too early, in too public a manner, will either open them to ridicule, or allow someone else to 'steal' their ideas.  But in my experience, the most successful early career humanists have already started building a form of public dialog in to their academic practise - building an audience for their work, in the process of doing the work itself.  

Perhaps the best example of this is Ben Schmidt, and his hugely influential blog: Sapping Attention.  His blog posts contributed to his doctorate, and will form part of his first book.  In doing this, he has crafted one of the most successful academic careers of his generation - not to mention the television consultation business, and world-wide intellectual network.

Or Helen Rogers, whose maintains two blogs: Conviction: Stories from a Nineteenth-Century Prison - on her own research; and also the collaborative blog, Writing Lives, created as an outlet for the work of her undergraduates. They bring together research and teaching, and in the process are building a substantial community of interest.

Or Adam Crymble and his blog - Thoughts on Public & Digital History - where he melds practical posts addressing straightforward DH problems, with substantial interventions in policy.  Crymble's recent appointment to a lectureship in digital history rested in large measure on his blog.  

The list could go on.  The Many Headed Monster, the collective blog authored by Brodie Waddell, Mark Hailwood,  Laura Sangha and Jonathan Willis, is rapidly emerging as one of the sites where 17th century British history is being re-written.   While Jennifer Evans is writing her next book via her blog, Early Modern Medicine.

The most impressive thing about these blogs (and the academic careers that generate them), is that there is no waste - what starts as a blog, ends as an academic output, and an output with a ready-made audience, eager to cite it.

For myself the point is that these scholars don't waste text, and neither do I.  If I give a talk, I turn it into a blog. Not everything is blogged, but the vast majority of the public presentations I make as part of my job, will be.  And while many of these texts will never contribute to an academic article, about half of them do.   As a result blogging has become part of my own contribution to what I think of as an academic public sphere.  It becomes a way of thinking in public and revising ones work, to make it better, in public.  And knowing that there is an audience (whatever its size), changes how one does it - forcing you to think a little harder about the reader, and to think a little harder about the standards of record keeping and attribution that underpin your research.  

One of my favourite blogging experiences involves embedding blogs in undergraduate assessment.  By forcing students to write 'publicly', their writing rapidly improves.  From being characterized by the worst kind of bad academic prose - all passive voice pomposity - undergraduate writing in blogs is frequently transformed in to something more engaging, simply written, and to the point.  From writing for the eyes of an academic or two,  students are forced to imagine (or actually confront) a real audience.  Blogging has the same effect on more professional academic writers - many of whom assume that if the content is good, the writing somehow doesn't matter.

But as importantly, blogs are part of establishing a public position, and contributing to a debate.  

Twitter is in some ways the same - or at least, like blogging, twitter is good for making communities, and finding collaborators; and letting other people know what you are doing.  But, it also has another purpose.  

Dan Cohen - the director of the Digital Public Library of America - always says about Twitter that the important thing is that at the end of the week, it makes you aware of all the publications  and developments, calls for papers, and conferences, you need to know about in order to keep up with your corner of the academy. It is not about what you had for breakfast.  It is about being on top of your field.

Between them, twitter and blogging just make good academic sense.  And while you need to avoid all the kittens and trolls, click bate and self-promoting gits, these forms of social media are rapidly evolving in to the places where the academic community is embodied.  They are doing the job of the seminar, and the letters page.  They are where our conversation is happening.

And on participating in this 'academic public sphere', there are only a few rules.  First - be yourself.  If you want credit, you have to own your material.  In other words, never be anonymous.  And second, remember that everything from Academia.edu, to Twitter, to Facebook and Flickr, is a form of publication, and should be taken seriously as such.  If you would not say it in an academic review, or in the questions following a public lecture, don't say it on Twitter.

And finally, keep track of it.  Use GoogleAnalytics, or something similar.  Know who you are talking to.  This involves nothing more challenging than cutting and pasting four lines of code, but provides more data, at a more granular level than you can possibly need.  

All of which is simply to say, that social media are building in to what feels like an increasingly coherent environment reflecting communities of interest - allowing us, online, to be just what we claim to be the rest of the time - a community of scholars.  The objections - which usually come down to the fear that someone will steal your ideas, your work, your credit - are best addressed by doing it in public.  And in the process there is every hope that we can rebuild the humanities as a wider public discussion, able to more effectively reach beyond the academy - to have 'impact'.  


Monday, 29 November 2010

The Conundrums of Assessment

To my chagrin I recently realised that I have been assessing research proposals and grant applications for some twenty years, and have done so for most of the major humanities and social science funders in the UK, Europe and North America.  Over the last ten years I have also sat on numerous grant awarding panels, and helped to design the odd funding programme.  I have form in this particular small area of academic life.  So when I was asked for advice about how to write an assessment by a colleague faced with their first request of this sort, I felt obliged to offer some.  And since there is no independent body of advice about how the system works, or how the panels who arbitrate on the final decision use the reports drawn up by assessors, it seemed worthwhile posting that advice:

In the UK, funding bodies have made great efforts to train assessors, and brief them on what is expected; and to all intents and purposes have created a system that seems transparent and clear, with apparently precise criteria laid out in straightforward prose.   A great deal of effort has also been put into the supporting documentation in an attempt to ensure that assessments can be compared against one another, and that the process of decision making undertaken by the panels is speedy and uncontentious.  A real attempt has been made to eliminate special pleading, conflicts of interest, and the administratively perverse.  But, of course, all bureaucratic systems are also cultural systems, and there remain many unstated realities that effectively determine how a grant application and the assessments written in response to it, are read by the people charged with eventually sifting the funded from the unfunded. 

The single most important and determining factor that every assessor needs to keep in mind is that  most funding programmes have a success rate of around 20-25% (some as low as 10%).  From a panel's perspective, four out of five applications must be rejected.  As a result even small issues and problems in an otherwise exemplary application will be used to make a determination.  After reading perhaps thirty solid projects (and these days few applications are less than solid) and when faced with the need to find just five or six to fund, any panel, however well meaning or intelligent its individual members, will begin to reach for the smallest weakness.  

As an assessor you need to be aware of this problem.  This does not mean that you simply laud your favoured project to the skies.  If you do, your assessment will be judged to be insufficiently critical, and therefore worthless.  Instead, it means that if you seriously think a project is likely to be better than 80% of the others, you need to act as a critical advocate, and to place yourself at the heart of the debate that the application you are assessing will inevitably generate.

There will always be one or two applications that sit on the top of the pile, and if you are assessing one of these you can give yourself the freedom to engage with the underlying ideas, and to simply discuss the project's importance for a wider field.  Even in this instance, you might want to suggest where small problems might exist, but have been effectively addressed.

But these few, intellectually exciting and beautifully realised projects are rare and are consistently funded.  Where all the debate will be focussed is over the next tranche of projects.  This usually comprises some 40% of the total.  In a panel meeting (and regardless of how they are organised) the grading schema inevitably breaks down, and this 40% of applications start to bunch around the boundary grades.  Most panels give up on whole number grading of the sort the funders recommend, and end up using some form of 4.257, or 4-+(?) (if they are dominated by older academics from the Russell Group).  If you are asked to assess an application of this sort, the first decision you need to make is whether you think it should be funded; and having made that decision (assuming it is positive) you need to act as an advocate.

In other words, the first thing that any assessor needs to do is make an informed, over-arching judgement about the quality and importance of the application in front of them.  If you think the application you are assessing is compelling, although not so strong as to sit on the top of the pile, then you need to say so, and say why.  Alternatively,  if it is in the bottom 50%, for either technical or intellectual reasons (i.e. not exciting, or not practical, or just not well written) there is little point in expending your time struggling to find something good to say.  It will not affect the outcome, and it is likely to be fed back to the applicant in a way that just encourages them to resubmit something similar, instead of something better.

For myself, I start of with a basic question in my mind:  Am really excited by the project?  Has it caught my imagination, and left me thinking that I would actually want to know the outcome?  Would I want to read the book?  Or in a Knowledge Transfer  context, would it make a significant social difference? 


A minority of  academic projects get past this hurdle.  As a result, the real problem comes with the next stage, which is that while the intellectual case needs assessing (and if you are excited by the project, this is the easy bit) it is essentially all the things around it that will be used to exclude marginal projects.  If the project plan (with methodology and budgeting etc.) looks less than professional and doable, the application will be excluded on these grounds.  But if the fundamental idea is exciting and you decide you want to support it, regardless of its minor faults, you will need to deal with these issues directly and explicitly.  If you don't, the curmudgeon in the corner (and there is always one) will use these small issues to denigrate the project - generally as a stratagem to promote their own discipline or methodology or favoured project.  In this context, if you see a weakness, you need to address it directly and explain why it is not important to the success of the project.

My feeling is that you need to exercise an abstract academic and professional judgement, and in the round (regardless of the hoops and bureaucratic forms the funders want you to jump through) come to a conclusion about the worth of the project.  Once you do this, you are duty bound to do everything you can to ensure that the result is positive, in full recognition that most projects, including innumerable worthy ones, wont be funded.   This can lead to a rather instrumental and manipulative approach (which is a problem), but it at least has the advantage of allowing us to exercise the kind of judgement that is implied in peer review even when the process feels like bureaucratic games playing.