In the her Chronicle article (subscription required), "Hot Type: No Reviews of Digital Scholarship = No Respect," Jennifer Howard reports on the struggles in getting digital scholarship properly reviewed. Not only is it difficult to establish appropriate standards for review, there are not many people capable of doing the reviewing. And I fully understand the nature of the problem here: the review is an important form of currency in academia.
But once again this can all be filed under missing the point.
Digital scholarship will never make sense within the context of a print scholarship intellectual marketplace. Here's the fundamental difference. In the existing print world, a text "counts" because two or three reviewers read it and said it's good enough to print. It doesn't matter if no one else ever reads it. Yes, we can get into bibliometrics, but such mechanisms cannot explain the forces at work behind a citation. It's as if getting cited as an example of stupidity is better than not getting cited at all. Similarly there can be a variety of unaccounted-for mechanics behind book reviews. On the flip side, of course it is fairly easy to measure a variety of user activities in relation to online scholarship (links, visits, time spent, click throughs, etc); there's also numerous ways for users to give feedback. So even though one might have trouble getting the kinds of reviews Howard is discussing, one would think that with digital scholarship one could get a far more accurate measurement of how people actually use the work.
So we could have a kind of pitched battle over these terms, where the evidence for the value of digital scholarship would suggest that we have reason to doubt the value of any/all print scholarship since we don't have any metrics for understanding how/if users actually do anything with it.
But I still think that's missing the point.
The point as I see it begins with getting beneath the process of scholarly publication and review, which is obviously a print process. Fundamentally to publish simply means to make something public. But obviously it means something more specific in this context. I think we have to ask why we publish articles and monographs (beyond the imperatives of tenure and promotion). What is this publication meant to accomplish? To suggest the most altruistic motives, the purpose of scholarly publication would be to contribute meaningfully to an ongoing conversation of scholarly and intellectual merit. While the article or monograph represent engagements of labor and thought of a certain scope/depth that I think are still important in the humanities, we need to recognize that the process of publication and review is not necessary to our deeper purpose. Therefore, I think we go down the wrong road when we try to accomodate digital scholarship to the process of publication and review.
We need to rethink the entire way we do intellectual work, which extends right down to what we ask our undergraduates to do in the classroom. I don't think this means abandoning the fundamental questions that drive our intellectual inquiry. Nor does it mean giving up the theoretical approaches that we use. But it does mean taking apart many of the unexamined, naturalized aspects of our disciplinary paradigms. Sure, one could ask, "But what is wrong with the scholarly essay? It still works just as well now as 20 years ago." But one could equally ask, what is wrong with public oratory? Why begin writing essays at all?
Digital scholarship clearly allows for a kind of large-scale, collaborative, iterative, scholarly activity that goes far beyond the essay with its citations, thesis statement, and facile structures that tie into a neat, conclusive bow somewhere around 6-7000 words down the road. In the context of digital scholarship, the essay and the monograph are about as useful as oral presentation. Of course we still do such presentations, and I imagine we will still write essayistic prose. But to seek to match digital scholarship to the metrics of print scholarship is simply another sad example of why one might lose hope for the long-term viability of academics. It's just sad, sloppy thinking.