I‘m writing just ahead of the main deadline for proposals to DH 2013, the primary conference of the international digital humanities community. It is my great privilege to chair the program committee for this year’s conference, which will be held in Lincoln, Nebraska. The PC are the stalwarts who design and coordinate peer review for the conference, pitch in to cover for delinquent reviewers, make final decisions on its intellectual program, and partner with local organizers in selecting invited speakers (we’ll look forward to a keynote from David Ferriero, chief Archivist of the United States, and it’s a Busa Award year, so we’ll honor and hear from the wonderful Willard McCarty).
“DH” is a new name for an old gathering. When I first encountered it in the 1990s the conference was called ACH/ALLC, after two professional associations (themselves dating to the ’70s) that had been sponsoring a joint meeting on computer-assisted humanities scholarship since 1989. The Digital Humanities name came in the mid-aughties, with the formation of a broader Alliance of Digital Humanities Organizations. ADHO has expanded, over time, to include Canadian and Australasian DH associations, as well as an international consortium of digital labs and centers. Shortly, we’ll welcome a Japanese association to the fold as well. (For a whirlwind history of big moments in DH, see John Unsworth’s “What’s ‘Digital Humanities’ and How Did It Get Here?“)
DH is my home conference — the (only) one I look forward to all year, and have attended most religiously since I was a grad student. This event, and the welcoming, rollicking, inventive, pragmatic, learned, egalitarian, global humanities computing community that coalesces around it, are without a doubt the reason I finished my doctorate, stayed in the field and in the academy (not, interestingly, self-identical in DH), and do all the things I do at places like the Scholars’ Lab, ADHO, ACH, NINES, SCI, RBS, MediaCommons, and MLA.
So, I’m pretty invested in getting things right as conference chair.
I’ve reviewed for DH since I was a child (well, almost), served as a program committee member for the conference twice before, was its vice-chair in 2010, and have had helpful conversations with several of our past smart, thoughtful, hard-working chairs — so I know this weekend, as proposals flood in and I gear up the system for the next phase, is one of several moments in the process when I can expect to be biting my nails. To assuage my nerves, communicate some of the good work the PC has been doing so far, and to make things a little less opaque for everybody, I’m writing this post. The cats in my title need herding. The ship is a slow one to turn around.
This cycle — for a variety of reasons defying logic but making practical sense at the time — will be the first in a long while that sees a sitting vice chair (for 2013, the fantastic Melissa Terras) become next year’s PC chair. I came in to the project of DH 2013 with a number of reforms and experiments in mind, but when Melissa was appointed we decided to take special advantage of the continuity our partnership would afford. The plan is (from our vantage point on the 2013 and 2014 PCs, and in collaboration with ADHO’s Conference Coordinating Committee, which sets the big picture rules) to herd a few cats and steer one unwieldy ship in a direction that seems healthy for the conference in the long run. This is needed, because Digital Humanities operates in a vastly larger and increasingly intellectually diverse DH community, not to mention one that has been newly noticed and is highly scrutinized.
Our adventurous PC was game — and the thoughtful, careful feedback I received from them and from ADHO’s CCC (especially from Julia Flanders and John Nerbonne on the latter committee) has been invaluable in putting together a set of experiments for DH 2013. Because setting up for the submissions phase of the conference was to coincide with the migration of the ConfTool system that runs it (along with all of ADHO’s other infrastructure! a speedy miracle wrought by Chris Meister and the committee he chairs), I limited my initial proposals to things that were possible without significant software development or changes to ADHO governing documents, which would require financing and voting. The reforms we ultimately took forward feel subtle in some cases and a little daring in others, particularly when you consider that this conference must make sense for people across a wild array of disciplines, professions, and national and cultural contexts.
Here, in bullet-point form, are the main changes and advancements we’ll be trying to make this year. Some of this could bomb. Some of it could suffer a tech fail. I think most of it will go through, and I will be working as hard as I can to make things operate smoothly and set us up for good assessment of successes and failures. Above all, I hope these experiments will be welcome within the community, meaningful for the intellectual program of the conference, and healthy for the coming-together of the practitioners DH 2013 attracts.
In roughly chronological order for the process, we:
- continued a trend started by recent PCs in resolving not to issue what had become a “customary” 2-week deadline extension on the CFP. As you’re scrambling to submit, you’re probably cursing us. Why’d we do it? The extension was only “customary” if you are a long-time attendee — an insider. Otherwise not fair dealing.
- revised and improved the Guidelines for Authors and Reviewers (also adding brief explanations of some of the new features mentioned below).
- greatly revised the CFP, combining two calls into one, reducing the overall wordiness of the beast, setting all deadlines in one fell swoop, reducing word counts for poster proposals, and simplifying requirements. This was done to promote clarity and ease the path to submissions for participants, but also to help ADHO’s Multilingual and Multicultural Committee with its annual task of translating the call into multiple languages. The CFP we produced can be used year after year with minor updates, and has already been translated into the Italian, Japanese, Spanish, Hungarian, French, Greek, Hebrew, Norwegian, German, Russian, and Serbian languages.
- also on the international front, we’ve started collecting reviewer language competencies in a more standardized way. I hope we’ll extend the rudimentary system I set up next year, so that we can collect even more usable data for the semi-automated system of assigning reviews and for the analytical purposes of researchers and the MLMC, who are interested in understanding our global community better.
- refreshed, extended, and modernized the list of “topics” available to users of the conference system. This is a rough ontology of DH, used to match reviewers’ self-professed expertise with the categories in which authors have submitted proposals. It needed serious updating, and the PC all pitched in on making sure it better represented the variety of work happening now and possible in DH. (If you’re a ConfTool user, let us know for next year what we’ve missed.)
- invited and added many new reviewers in a number of crucial fields, with a goal of generating 4-6 reviewer responses for each proposal submitted. A minimum of three reviews (plus program committee evaluations) will certainly be met. When I’m nervously waiting for reviewers to return their assessments, I’ll probably be cursing myself for not issuing an even broader call — but that was done last year by PC chair Paul Spence to good effect — and I’m hoping that strong attendance at this year’s conference and the overall growth of ADHO will help us reach out to even more new readers for next year.
- are newly allowing reviewers to specify the maximum number of reviews they are hoping to evaluate, in an attempt to improve our rate of return and avoid placing unwelcome burdens on members of the community. (Wish me luck that the numbers add up! Otherwise I will be apologizing for not matching reviewers’ wishes and/or seeking new readers post-haste.)
- set up some new fields in the submissions forms, including a super-brief summary (whose utility I’ll explain shortly) and a tic-box through which authors can indicate if they would like their proposals to be considered in additional program categories as well. This is useful, for instance, if we have run out of room for long papers but would like to offer some presenters the chance to share their work as short papers.
- are instituting a brief “bidding phase” in the peer review process. Here, reviewers will be able to see those abbreviated summaries of each proposal and indicate strong desires or feelings of qualification to review, flag any conflicts of interest, and mark submissions they’d feel particularly unqualified to judge. (With this item and the ones to come, I get into the “possible tech fail” category of experimentation. We’ll be using features of our ConfTool system never employed before for DH. Wish us luck.)
- will make it possible, during our single-blind peer review process, for reviewers of a given paper to see each other’s (anonymous) comments — for that paper only. We hope the sharing of good examples of thoughtful and constructive critique will increase reviewers’ quality of engagement with the proposals and their cordiality to authors, and contribute to the fellow-feeling with which we all undertake the service of reviewing. To minimize any danger of group-think, we will ask reviewers who augment their comments after seeing others’ to offer a thorough justification. (This one’s a little bit on the daring-experiment side — and if you are among the skeptics, you’ll be relieved to hear that I was talked down from suggesting completely open reviews!)
- will work to improve the explanatory language for each score-factor displayed to reviewers (what do we mean by “originality” etc?), reduce and refine the arbitrary number of score-levels possible to assign within a particular category (to pick on a past set of factors: what’s the difference between “ground-breaking” and “pioneering” levels of inventiveness?), remove “Relevance to the Conference” and “Presentation” as scoring categories (because they often feel like unwelcome gate-keeping and may move reviewers to penalize non-native English speakers), and reduce the overwhelming fudge-factor of a category formerly called “Overall Recommendation” (and once set to 50%).
- will institute a short response phase for authors after reviews are back in (ConfTool calls this “rebuttals” — we won’t!). Here, authors of proposals will have a small amount of time to add a formal but exceedingly brief statement as a follow-up on their reviewers’ comments. Authors may choose to tell us what they intend to improve in their proposals, based on reviewer suggestions, or they may have other additional info they feel it’s important to share. Responses aren’t at all necessary, but any offered will be taken under consideration by the PC in making final determinations about the program.
- will experiment in using ConfTool’s capacity to take into account reviewers’ self-reported familiarity with the topic of each proposal as a weighting factor in deriving numerical scores. We’ve collected this info in the past but, it seems, not used it as part of the formal process. If it’s plainly unhelpful, we will disregard the data and re-calculate scores. In any case, the numbers don’t rule the outcome — they just help the PC do some rough sorting and determination of how to use our time for discussion most effectively. (It’s important to add that the numerical scores also help the PC see more clearly if a given reviewer of a paper is a customary high-scorer or low-scorer, and to identify cases where there is marked disagreement among reviewers, which suggests additional reviews and special attention are needed.)
- and, overall, will be working hard to communicate well with authors, reviewers, and the broader community interested in digital humanities and the DH conference.
I may have missed a thing or two, but that’s the basic shape of it! Unlike all of the improvements and experiments I’ve listed here, this post did not benefit from the careful review and contribution of ADHO’s CCC, MLMC, or the DH 2013 PC, who are:
- Craig Bellamy (ACH)
- John Bradley (ALLC)
- Paul Caton (ACH)
- Carolyn Guertain (CSDH/SCHN)
- Ian Johnson (aaDH)
- Bethany Nowviskie (ACH, chair)
- Sarah Potvin (cN)
- Jon Saklofske (CSDH/SCHN)
- Sydney Shep (aaDH)
- Melissa Terras (ALLC, vice-chair)
- Tomoji Tabata (ALLC)
- Deb Verhoeven (aaDH)
- Ethan Watrall (cN)
But it is a small attempt to follow through on our shared commitment to transparency about the process and continued energy and evolution for Digital Humanities.