Mapping Texts: Visualizing American Historical Newspapers

and

Mapping Texts is an ambitious project with a simple mission: to experiment with new methods for finding and analyzing meaningful patterns embedded within massive collections of digitized historical newspapers.

Why do we think this is important? Because, quite simply, historical newspapers are being digitized at a rate that is rapidly overwhelming our traditional methods of research. The Chronicling America project, for example, recently digitized its 5 millionth newspaper page, and predicts that more than 20 million pages will be available within a few years. Numerous other programs are also digitizing newspapers at a rapid pace worldwide, making hundreds of millions of words from the historical record readily available in electronic archives that are reaching staggering proportions.

Such enormous collections offer tantalizing new possibilities for humanities research. Yet without tools and methods capable of sifting meaningful patterns from such massive datasets, the challenges of working with digitized newspapers are becoming equally overwhelming. Researchers, for example, too often find themselves confined to exploring such archives through basic text searches (which, when they produce several million hits, offer too many results to analyze in any meaningful way by hand). And scholars invariably have no ability to evaluate basic metrics (such as how much data is available from a particular time and place, or the quality of the OCR — optical character recognition — digitization process) about a given online collection. Harnessing the promise of digitized newspapers, in other words, requires building more transparent windows into the tremendous wealth of such archives.

Our purpose with Mapping Texts, then, has been to experiment with developing new methods for enabling scholars to sift, sort, and explore digitized historical newspapers for their research. To that end, we have attempted to combine the two most promising methods for analyzing large-scale datasets: data- and text-mining (for discovering meaningful patterns embedded in large bodies of text) and data visualization/mapping (for grouping, discovering, analyzing, and making sense of those patterns). Working with a collection of about 232,500 pages of digitized historical newspapers, we produced two interactive interfaces:

1. Mapping Newspaper Quality maps a quantitative survey of the newspapers, plotting both the volume and quality (OCR recognition rates) of information available in the digitized collection. Through graphs, timelines, and a regional map, users can explore these metrics for any particular time period, location, or newspaper. Clicking on individual newspaper titles also allows users to jump from “distant” to “close” readings of the texts.

Mapping Newspaper Quality

Mapping Newspaper Quality

2. Mapping Language Patterns maps a qualitative survey of the newspapers, plotting major language patterns embedded in the collection. For any given time period, geography, or newspaper title, users can explore the most common words (word counts), named entities (people, places, organizations), and highly correlated words (topic models), which together provide a window into the major language patterns emanating from the newspapers. Clicking on individual newspaper titles also allows users to jump from “distant” to “close” readings of the text.

Mapping Language Patterns

Mapping Language Patterns

These two interfaces are built on top of the large archive of historical newspapers digitized by the University of North Texas (UNT) as part of the Chronicling America project and UNT’s Portal to Texas History. We selected this archive for a number of reasons: with nearly a quarter million pages, we could experiment with scale; the newspapers were digitized to the standard set by Chronicling America, providing a uniform sample; the Texas orientation of all the newspapers gave us a consistent geography for our visualization experiments. It also represented the entire corpus available to us — or any researcher — accessing UNT’s digital newspaper archive when we began the project during the fall of 2010.

The project was, at base, an experiment to see what we could discover about the breadth and depth of a single electronic newspaper archive, and what that might tell us about other similar archives. The project’s interfaces are meant to be used in tandem, with the hope that researchers will combine insights from the two in order to better sift through these collections and perhaps discover previously hidden connections in the newspapers.

This work depended heavily on collaborations between scholars at UNT and Stanford University — UNT’s Rada Mihalcea and Stanford’s Geoff McGhee, in particular. Please see the project website for a full listing of the team behind the project.

About Andrew J. Torget, and Jon Christensen

Andrew J. Torget is a historian of nineteenth-century North America at the University of North Texas, where he directs the Digital History Lab. The founder and director of numerous digital humanities projects -- including Mapping Texts, Texas Slavery Project, Voting America, and the History Engine -- Andrew served as co-editor of the Valley of the Shadow project, and as the founding director of the Digital Scholarship Lab at the University of Richmond. The co-editor of several books on the American Civil War, Andrew has been a featured speaker on the digital humanities at Harvard, Stanford, Rice, and the National Archives in Washington, D. C. In 2011, he was named the inaugural David J. Weber Research Fellow at the Clements Center for Southwest Studies at Southern Methodist University.

Jon Christensen is a founder and principal investigator in the Spatial History Project and the City Nature digital humanities project at Stanford University, and former executive director of the Bill Lane Center for the American West. In the fall of 2012, he will join the History Department and the Institute of the Environment and Sustainability at the University of California, Los Angeles, where he looks forward to working with the Center for Digital Humanities as well.