scholarship, impact measurement, and genre

Impact measures are becoming an increasing issue at UB just as they are at many institutions. For those who don’t know, in terms of scholarship this typically means citation analysis, or even more bluntly, how many times do you get cited. It doesn’t take an advanced degree in critical thinking to imagine some of the problems with this. There are technical issues with tracking citations and there are disciplinary/genre issues with the rhetorical practices of citation that not only can vary within a field at-large (e.g. literary studies) but within specializations (so medievalists might have different citation practices from Americanists). This practice also seems to stem from the principle that there’s no such thing as bad press. I.e., being cited as a total fool would seem to be the same as being cited as a genius.

However I think there is a larger question for the humanities. Is “impact” what we are after? The familiar argument is that we value “reputation” rather than impact. Ok, but don’t all fields have a sense of reputation. That is, the scientist with a national reputation accrues value in that in terms of winning grants, attracting good students, landing a better job, etc. I doubt most scientists would know their colleagues’ citation/impact metrics. Clearly there’s a feedback loop here. However in many fields impact/citation has less duration than reputation because research loses value over time. This isn’t the case in the humanities where reputation would tend to drive citation. That is, if my next book is very successful it will probably result in an increase in citation of my earlier work.

But let’s separate impact from the metric of citation and try to think about it more abstractly. I would define impact has having an affect on 1) the work of students and colleagues in my field, 2) broader conversations within my discipline, across disciplines or possibly across higher education (depending on the nature of the scholarship), 3) a general public understanding of the topic. When we shape our research projects, to what extent do we define them in relation to the objective of having an impact? In Kuhnian normal science terms we might say that the established paradigms of a discipline provide an built-in answer to this question. Furthermore, granting agencies provide some commonality of focus and purpose. In the humanities we are more independent and diffuse. We tend to follow our noses, and I would say we have a built-in allergy to the value of utility that would tend to push us away from asking rhetorical questions about our audience and the purpose of our work. In other words, I don’t think we research/write/publish for the purpose of being read, let alone cited, or at least those motives are not strong enough to shape our activities.

That’s why I ask if impact is what we are after, and if it is, should we consider a different approach to the work we do? As I asked a couple posts ago, if I want impact, should I really be writing a book? What are the contradictions between the institutionalized expectations of tenure and promotion and the new expectation that we have impact? We might even push this question down to the level of graduate training. Is writing a dissertation a good way to have impact? Does it provide training in doing the kind of research that will have impact in the future?

Though I have mixed feelings about the quantitative, metrical measurement of impact, I am probably more sympathetic to the implicit value than most people in the humanities. That is, I’d like to have impact, and I’d like to know if I am having an impact. While citation is one value, it seems to me that there are a lot of other potential measures such as pageviews and links to online articles and downloads of pdfs from online databases or open access publications. In theory, it seems possible to know how many times someone has viewed an article of mine in Enculturation or Kairos. I can tell you how many times a given blog post here has been viewed and which one is the most popular in the last year (you would be surprised I think). It also seems like we should be able to know how many times my Computers and Composition article has been downloaded from Elsevier’s servers. I would guess that this is the kind of data that publishers would want to know for themselves anyway. It would seem to me that such numbers would be at least as informative as citation.

But what would that imply about the monograph? As a scholar in the humanities, what’s more useful to you? How many times would you say that an article (or some shorter-than-a-monograph genre) would have been of more benefit to you than the monograph you read? I can’t figure out any convincing impact-based argument for writing a monograph. You tell me.