Evaluating Digital Humanities Work: Guidelines for Librarians

Zach Coble

In this piece, Zach Coble explores the benefits of creating guidelines for the evaluation of librarians’ digital humanities work for the purposes of hiring, appointment, tenure, and promotion, and offers a basic framework for what those guidelines might look like.

Digital humanities, as well as related fields such digital media studies and digital libraries, have presented many opportunities for libraries. These include the establishment of digital humanities centers, the development of new data standards, new forms of scholarly communication, the creation of new resources (and novel ways of asking questions of those resources), and the development of new tools for scholarship and accessing collections.^[1] However, traditional modes of evaluation do not address many of the key aspects of digital humanities work.

As librarians become more involved in digital humanities and begin to take on the title of “Digital Humanities Librarian,” how can we ensure that their work will be appropriately reviewed? While some librarians work individually on personal digital humanities projects or scholarship, most collaborate with faculty, fellow librarians, and information technologists across campus and across institutions. The collaborative nature of digital humanities work often blurs the lines when it comes to defining individual’s responsibilities and contributions. Similarly, new forms of scholarly output, such as a website rather than a paper or presentation, present additional challenges for those tasked with evaluating digital humanities work.

Written guidelines for evaluation ensure that projects are reviewed fairly and provide a clear path for job hiring and advancement. Libraries clearly understand the importance of assessment and evaluation. The Association of College and Research Libraries (ACRL) has guidelines for the evaluation of tenure track librarians and for those without faculty status. In 2010, Megan Oakleaf made waves with her Value of Academic Libraries report, which utilized existing assessment measures, such as college students’ information literacy skills, to demonstrate the positive impact of libraries. As the field of digital humanities continues to grow, libraries will increasingly be called upon to dedicate time and resources to supporting this work. In order to encourage more libraries to support digital humanities, to provide a framework that will encourage individual librarians to participate in digital humanities, and to acknowledge and reward excellent work, libraries should develop guidelines for evaluation of librarians engaging in digital humanities work.

Although librarians are often cited as important collaborators in digital humanities projects, librarianship as a profession lacks a coordinated approach to digital humanities. There are many reasons for this, such as the broad interdisciplinarity and rapidly evolving nature of digital humanities, which makes it difficult to articulate a large-scale response. Yet it also stems from the fact that library involvement in digital humanities varies across institutions: some libraries at large research-intensive universities host active digital humanities centers while many small schools (as well as public libraries, special libraries, and so forth) are only vaguely aware of digital humanities, if at all.

In a recent survey by the Association of College and Research Libraries Digital Humanities Discussion Group, most of the librarians who responded did not have digital humanities in their job title or description. Equally diverse are the types of work that librarians contribute to digital humanities projects. A 2011 report on digital humanities in libraries by the Association of Research Libraries (ARL) noted that digital humanities projects often call upon librarians for consultation and project management, technical and metadata support, instructional services, and resource identification.

A framework for evaluating digital humanities work performed by librarians would ideally be one piece of a program to address digital humanities from libraries. Such a program, possibly from the Association of College and Research Libraries, might also include criteria for undertaking digital projects and best practices for doing digital humanities work. As the 2011 ARL report notes, “The general lack of policies, protocols, and procedures has resulted in a slow and, at times, frustrating experience for both library staff and scholars. This points toward the need for libraries to coordinate their efforts as demand for such collaborative projects increases.” Without an organized response, librarians lack the incentives, resource support, institutional backing, and network of colleagues necessary to be successful.^[2] On the other hand, a coordinated approach could encourage more librarians to get involved in digital humanities, motivate individual libraries to adopt related policies specific to their local needs, foster greater participation among libraries in the digital humanities community, and create the demand for increased training opportunities — both as continuing education for professionals and in library schools.

Other organizations, such as the Modern Language Association, NINES, and 18thConnect, have recognized the distinct nature of digital humanities work and adopted separate guidelines for the evaluation of digital projects.^[3] Libraries would benefit from having a similar set of guidelines. Of course, every institution is different and no one set of guidelines will work for everyone. Also, the context and scope of a librarian’s contribution should be taken into account — a librarian asked to consult on metadata standards should not be faulted if the project fails to follow web design best practices. While acknowledging such nuances, there are certain baseline ideas that should be addressed. The following list draws upon existing guidelines for the evaluation of digital humanities work mentioned above and incorporates additional elements specific to libraries. It is intended to help generate conversation and is not meant to be comprehensive.

1. ^[1]See Association of Research Libraries, Digital Humanities, SPEC Kit 326; Micah Vandegrift, “What Is Digital Humanities and What’s it Doing in the Library?“; Miriam Posner, “Digital Humanities and the Library: A Bibliography“; Matthew Gold, Debates in the Digital Humanities.↩
2. ^[2]For a discussion of barriers to digital humanities in libraries, see Miriam Posner, “What are some challenges to doing DH in the library?” ; Trevor Munoz, “Digital humanities in the library isn’t a service“; Mike Furlough, “Some Institutional Challenges to Supporting DH in the Library“; The Library Loon, “Additional hurdles to novel library services.”↩
3. ^[3]See also University of Florida George A. Smathers Libraries, Facilitated Peer Review Committee; Roy Rosenzweig Center for History and New Media,  Zotero | Groups > Digital Humanities > Library > Assessment and Evaluation; Modern Language Association, Guidelines for Editors of Scholarly Editions; University of Nebraska-Lincoln Center for Digital Research in the Humanities, Promotion & Tenure Criteria for Assessing Digital Research in the Humanities.↩