In the APSA Public Scholarship Program, graduate students in political science produce summaries of new research in the American Political Science Review. This piece, written by Leah Costik, covers the new article by Christopher Darnton, Naval Postgraduate School, “The Provenance Problem: Research Methods and Ethics in the Age of WikiLeaks“.
In the last decade or so, individuals such as Edward Snowden or Chelsea Manning have leaked classified documents to the public. These documents are now frequently drawn upon in published academic articles within political science journals. In his article in the American Political Science Review, Christopher Darnton asks how researchers, particularly those within political science’s subfield of international relations, should approach the “informational fall out” generated by such leaks and raises both the consequences and challenges that stem from the use of such sources.
Darnton’s argument centers around the concept of provenance, or sources’ “history of possession and transmission.” Sources obtained non-consensually and used in research present ethical and methodological challenges. Being stolen, such documents raise questions regarding selection bias, legality, and authenticity. These documents also violate privacy, security, and risk real-world harm to both individuals and institutions. Other disciplines, such as archeology, increasingly investigate the provenance of sources to circumvent these ethical and methodological concerns. Curiously, however, political science largely refrains from this reflection and practice.
Darnton examined scholars’ use of leaked classified documents within published academic articles since the disclosures of classified materials by Private First Class Chelsea Manning to WikiLeaks to determine the scope of the practice. He surveyed the twenty most prominent academic journals for articles from 2010-2020 that mention either “WikiLeaks” or “cable.” This survey yielded 565 articles. Darnton then coded articles on a scale of 1-3. Articles that score a 1 value are “false positives”; they mention WikiLeaks but do not engage cables or leaked material. Articles that score a 2 refer to leaked materials indirectly through the use of other published works rather than cite primary sources themselves. Finally, articles with a 3 score directly quote, paraphrase, or cite WikiLeaks or a cable. 168 articles received a score of 2 or 3. After further analysis, Darnton arrived at three main conclusions. First, many articles used leaked materials, with 116 articles citing leaked material directly (score 3) and 52 articles citing such material indirectly (score 2). Second, by 2018 all twenty analyzed journals published articles that used leaked materials. Finally, Darnton found that the use of leaked material in academic scholarship is persistent across time— in other words, citation of leaked materials did not explode after Private Manning’s leak and then decrease. Rather, use of cited leaked materials persists somewhat steadily, with 2016 seeing the highest number of peer-reviewed code 3 articles and 2019 the second-most. Taken together, these findings raise concerns regarding transparency as it relates to use of leaked materials. Darnton found academic articles that employ leaked materials rarely disclose or reflect upon this practice. When leaked materials are cited, such citations fall short of transparency, with “hyperlinks to media outlets or organizations other than WikiLeaks often left citations unclear regarding whether authors engaged primary or secondary sources.
“By offering a sort of checklist or exercise to researchers, this article is a useful read for those in the beginning stages of a research project and would be a particularly helpful addition to any graduate methodology course syllabus.” Darnton then reviews the guidance academic associations and journals offer regarding the use of leaked classified documents and finds conflicting and insufficient guidance. To redress this, he offers eight normative and evidentiary criteria to assess sources. Two of these eight criteria involve empirical concerns over data richness and reliability. The remaining criteria consider ethical concerns, including questions surrounding legality, national security, public interest, policy relevance, human-subjects protections, and reflexivity. Darnton also urges scholars to practice intellectual humility and avoid four forms of arrogance: entitlement to sources, straightforward inference from them, confidence in public value and assuming minimal harm, and the assumption that readers share authors’ values and need no persuasion, or will not notice questionable methods.
Darnton’s article makes several important contributions. First, the article raises serious questions about both methodology and ethics as they relate to the use of leaked materials. Second, Darnton offers multiple types of criteria scholars or journalists may consult. For example, Darnton presents eight normative and evidentiary criteria to assess sources; four recommendations for researchers considering the tradeoffs between using (or not) leaked materials; and discusses four forms of arrogance as they relate to research to avoid. By offering a sort of checklist or exercise to researchers, this article is a useful read for those in the beginning stages of a research project and would be a particularly helpful addition to any graduate methodology course syllabus. Finally, Darnton presents readers with a compelling argument and question: if other disciplines such as archeology, art studies, and anthropology are concerned with the provenance of sources and judiciously seek to confirm such origins, why is political science as a discipline not similarly concerned? This uncomfortable but deeply important question is now ours to ponder.
- Leah Costik is a third-year Ph.D. student in the Department of Political Science with a minor in Global Public Health at the University of Minnesota, Twin Cities. Her research interests include rebel governance, public health, and civil war.
- DARNTON, CHRISTOPHER. 2021. “The Provenance Problem: Research Methods and Ethics in the Age of WikiLeaks.” American Political Science Review, 1–16.
- About the APSA Public Scholarship Program.