Providing A Familiar Assessment Process for Digital Projects

"Trust Me" by Anderson Mancini

A few weeks ago in my Early Modern British History tutorial the students read a book review that had been printed in a scholarly journal. I asked the students, “Why are we reading this? What’s the point of this piece of writing?”

My cynical self tried not to let on what I was thinking: wasn’t it written so some PhD student could score a free book and put another line on his C.V.? My students thankfully didn’t see it that way.

Historical writing generally comes in one of a handful of tried-and-tested forms. In the course I teach we usually read and discuss articles published in peer reviewed journals. The students are now used to the goals of that particular mode of writing. They know it contains a historian’s contribution to a particular scholarly discussion, which is fairly narrowly defined and tends to correlate rather strongly to the topic we’re discussing that week in lecture.

They understand edited collections contain a series of articles by different authors on a theme defined by the editor. They know books (or monographs as historians like to call them) cover broader topics than a journal article, but they expect to find chapters in those monographs that rival the scope of an article. And for a first year undergraduate, that about covers the known forms of printed historical research.

Scholarly book reviews are generally a fairly new beast for the students. My question about the value of book reviews was of the type I knew I would not have been able to answer as a first year undergraduate, but I’m blessed with very clever students who had all manner of ideas. My favourite comment was that a book review:

provides another perspective on the strengths and weaknesses of the work, which are important to keep in mind when coming to our own conclusions about the author’s arguments.

I believe that idea holds the key for digital humanities scholars, many of whom feel their colleagues have been unwilling to award due credit for digital projects. However, the problem is not necessarily an unwillingness; the problem is that most people aren’t qualified to assess digital humanities work, so they just don’t. Forcing someone to assess something without the skills to do so is not the answer. I think most of us would prefer that a review of our work came from someone with the background to assess it fairly.

Scholarly reviews are a great platform for that assessment process for two reasons. Firstly, they take a form that nearly all humanities scholars already understand. Secondly, they allow the digital humanities community to self-define the major projects so that non-digital colleagues don’t have to.

Familiar Clothes

Like my students, most scholars can visualize where a review fits amongst the various scholarly apparatus floating around in the intellectual space in which we share and critique each other’s work. They understand that a quality review in a reputable journal represents an arms-length critical evaluation of the work by someone with the skills to make the assessment. Reviews certainly are not a perfect model. There is clearly a risk to any creator or team that they will get an unfairly negative treatment by someone who perhaps is not as qualified to write the review as they may think. But this is the same risk authors of monographs face, and to me, the benefits far outweigh the risks.

The world of publishing is changing, but there’s something to be said for making something look familiar. The Journal of Digital Humanities has taken this approach by experimenting with new ways of finding content that is important to the scholarly development of digital humanities – although I imagine that process of identifying content would look rather shocking to a typical historian or English lit. professor. (If you’d like to learn more about that project read their “How to Contribute” page.) However, by wrapping it up in the clothes of a traditional peer reviewed journal, the editors get to experiment, the authors get credit for their contributions, and the readers are drawn to some of the best ideas that appear online. Everyone wins.

For me, the Journal of Digital Humanities has represented a great success as a project that has been able to forge ahead, while acknowledging that sometimes it’s a good idea to change things gradually. In that project, the team decided to make sure they held onto some familiar concepts. They registered an ISSN number (2165-6673). They release the journal in “Volumes” with issue “Numbers” to ensure any citations look like a typical humanist expects them to. Neither of these is technically necessary, but they certainly do not hurt. And by having them those who disapprove of any changes to the mechanisms for scholarly communication have a few less things about which to complain and are instead forced to discuss the actual changes such as the open submission system rather than a lack of an ISSN number.

The Major Contributions

Digital humanities projects – particularly the digital ones – have taken an entirely different path to the familiar clothes approach. They look completely different. Many people still aren’t clear where a scholarly database or a digital tool fits amongst the other scholarly apparatus. Is it akin to a monograph? Is it a chapter in an edited collection? Is it an edited collection? Just how much work was it after all?

By changing the clothes completely digital humanists have made it difficult for non-digital colleagues to assess the work because they don’t necessarily understand how it was constructed or the intellectual considerations that went into building it. I believe we can change that by beginning to write and publish reviews, because within the traditional humanities framework reviews are reserved for major contributions. In historical journals it is generally only books that warrant a review, and I would suggest the book-equivalent amongst digital projects could benefit from the same.

If the community of digital humanists own this process, then it’s the community that gets to decide what the major contributions to the field are, by putting them up for review. And that process means non-digital colleagues not only have an arms-length evaluation of the merits and shortcomings of digital work from a reputable expert, but this review can then become the basis for digital scholars to go to their departments and say: this is what I achieved. And with that form of evidence, I think we’ll be one step closer to slotting digital projects into that mental framework of scholarly contributions.

Let’s Get Started!

Ok, ok. But before we get going, perhaps we should sit back and think for a few moments to ensure the reviews follow that model of familiar clothes. For these reviews to hold weight with non-digital faculty and to play an important role ensuring digital work receives the credit it deserves, reviews need to withstand a certain level of scrutiny. But they cannot merely look like reviews. They have to be rigorous, reliable, and arms length.

The first question then is where should the reviews appear? Many digital humanists would likely suggest blogs are the answer. After all, they are cheap and they are efficient publishing platforms. But I believe that would be an error in judgment. Firstly, they fail the familiar clothes principle. Some academics are still untrusting of blogs and whether we collectively agree with that skepticism or not the goal of reviews is to bridge that gap in trust, not entrench it; therefore we must make reviews as easy to swallow as possible for those we hope to engage.

Secondly, blogs are not arms length because the author controls all levels of the review’s distribution. A few years ago Amazon’s review system showed us why that can be a problem when the wife of an academic wrote poisoned pen reviews of his competitor’s books. If project reviews are to have an important role in assessment and credit then it is important that those reading them can be confident that the review was not written by a friend or colleague (or enemy) of a project leader who may have had an agenda.

For me, the natural home of project reviews is in a scholarly journal. Not that many dedicated scholarly journals out there focus on the digital humanities and from what I can tell only one (Literary and Linguistics Computing) currently publishes reviews - at the moment limited to reviews of books about digital humanities. This suggests to me that there is an opportunity for any of these journals to take the lead on such an initiative and adopt a project reviews section. I would particularly urge the Journal of Digital Humanities to take on this challenge not only as the new kid on the block, but as a group committed to trying new things in publishing and championing those who do things just a little bit differently.

By having reviews centred in a scholarly journal they gain not only a trusted distribution system – again familiar clothes – but they also fall under the control of an impartial editor. This is important for the same reasons that a blog is not the right venue for such work; by putting the system of soliciting reviews in the hands of an arms-length editor, a further check is placed upon the quality of reviews. This will not only reduce the number of friends-helping-friends reviews, but also ensures that the best work gets reviewed, rather than the work of people with large professional and personal networks.

By providing a familiar form of critical assessment for non-digital colleagues, digital humanists can collectively define what makes their work good, innovative, and scholarly. Because it is they who are best positioned to do so. In a few years I hope to be sitting with a new class, asking them: “Why are we reading this digital project review? What’s the point of this piece of writing?”

Then as now, the answer is not because a PhD student wanted to add a line to his C.V. Rather, it is because the digital humanities community needed a mode of assessing their work in a way that reflected the unique challenges and assets of such projects so that their colleagues had the tools available to critique some great if oftentimes overlooked projects.

I’ll even volunteer to write the first review. You know where to find me.

Review: Lunch at the British National Maritime Museum

I'm going to give the collections and exhibits at the British National Maritime Museum (Greenwich), a free pass at the moment, because much of it is still under construction for the next few months. I will say I was under-awed at what was there, and frustrated on occasion that the panels were positioned for 6 year olds, not 6 foot tall adults. Nevertheless, I'll give them the benefit of the doubt for now and will leave my comments about the exhibits at that.

What was noteworthy, however, was lunch. I've learned not to expect much from British restaurants. Particularly those in museums. Usually I'm happy if my prepackaged sandwich has wholewheat wonderbread and at least two distinguishable flavours. If I can find a side made with something other than potatos, I'm awestruck.

So, upon visiting the Museum Café, with its beautiful views of Greenwich park, I was completely bowled over at the quality of the food. It was fresh, home made, healthy, and delicious. I had gone in expecting deep fried fish and chips - appropriate for a Maritime Museum I thought - and instead came out with a homemade pork and pickle pie (clearly made by someone who knew their way around a kitchen), and a wonderful toasted pecan and gorgonzola salad. My wife had delicious smoked salmon on a bagel and a slice of perfectly ripe watermelon. To our surprise, and that of everyone before us, the fresh fruit was included with every sandwich, prompting more than one person to put down their bag of chips (crisps).

The tragedy is of course, that a healthy lunch is noteworthy at all. At institutions that focus on drawing families, it's disheartening to see all too often that the only options are deep fried. In one case - Hampton Court - the food was so bad that my family left most of it on our trays and went hungry until we could get home. Museums have the opportunity to act as examples for the community, as cultural centres of sharing and learning. I hope more of them follow in the footsteps of the British National Maritime Museum, and extend that to a good, healthy meal.

Academic and Public Historians: Working together?

Sean Kheraj, a postdoctoral student at the University of British Columbia recently defended his doctoral thesis on the environmental history of Vancouver's famous Stanley Park. In most people's books, that' makes him an expert on the field.

Sean was recently asked to write a review of the Vancouver Museum's exhibit, "The Unnatural History of Stanley Park." The experience, which Kheraj likened to "witnessing the physical manifestation of my dissertation as a museum display," was a challenge for him. Every detail that the museum had gotten wrong popped out. But, rather than posting a negative review, Kheraj reflected on the role many academic historians could be playing as consultants in instances that they choose not to.

It's something that came up a lot last year in our UWO Public History discussions, but perhaps not something that regularly enters the consciousness of many academic historians.

You can read the review at http://niche.uwo.ca/foresthistory/resources/exhibits

Book Review: "Remember, Remember (the Fifth of November)"

Parkinson, Judy. "Remember, Remember (the Fifth of November): The History of Britain in Bite-Sized Chunks." (London: Michael O'Mara Books, 2008). 182p. 10pounds.

The book attempts to survey all of British history through a chronological series of encyclopedia entries, each one page in length. Each page outlines a major historical event, figure or stage in British history, beginning with the Roman Invasion of AD43 and ending with the founding of the United Nations. The title, of course, refers to the Gunpowder Plot on 5 November, 1605, still remembered with the celebration of Guy Fawkes Day.

Parkinson has succeeded in creating an abundantly clear survey, which will no doubt clear up confusion many people have as to the order of events in British history. Readers can clearly see how events influenced or were influenced by one another; no entries appear to stand out as isolated from the rest, which is a credit to the author's choice of topics.

Each entry can stand alone and the book can be used as a reference, or it can be read through from front to back. Those who elect the latter will note that at times events or persons are explained in more than one entry, especially in cases where one entry is the subset of another - a battle and a war, for instance. This repetition makes it possible for readers to use the work as a reference book and should not be considered a failing.

Someone looking for a brief survey, or an undergraduate about to embark on a British history course, could certainly benefit from this book. Its brevity and journalistic prose make for light reading. Parkinson has done a fabulous job of creating what is essentially a chronological encyclopedia without leaving the reader feeling like they have been sitting with a copy of Britannica.

As the entries are short and without footnotes, the book is not of much scholarly use - though no encyclopedia is. There are a handful of instances that used direct quotations in which even a quick footnote or reference would have been appropriate so that readers interested in pursuing further reading could at least be directed. However, the lack of references was not a major issue.

Nearly all British historians will feel that their area of expertise did not receive proper treatment; many important events are only referenced in an entry devoted to something else. For instance, the South Seas Bubble gets a single sentence in King George I's article (p. 102) and nothing of its own. Likewise, many other important aspects of British history do not appear at all. Nevertheless, the task of writing two-thousand years of history in 150 entries is a major undertaking and many worthy events must be cut to make the project doable.

Since it is impossible for anyone to be an expert on a topic so vast, I am unable to comment upon the accuracy of each entry. However, I can say that the entries are fairly consistent, with a few notable exceptions. The articles "Munich and the run-up to the Second World War" (p. 174) and "Germany Invades" (p. 177) list that Britain formally declared war on Germany on September 1, 1939 and September 3rd respectively. Even a non-historian can see issue with this claim.

The most glaring problem the reviewer noted was in the "Magna Carta" entry (p. 53). In this article, the author makes a bold claim that the Magna Carta is "the most famous and most significant legal document in the history of democracy" (p. 53). This is a very partisan claim that most Americans and a good many others around the world would certainly debate. Given that the book has limited itself and decided not to provide evidence to support such bold claims, they should not be included. One cannot assume that a reader has the time or inclination to look further into the topic to come to an educated conclusion.

Apart from these minor failings, the book is certainly worth the cost and time required to read it. It would make an excellent course reading for the first week of a British history survey. More countries could benefit from a similar survey of their own history.