Updated Invitation for Deliberation about Research Transparency and Interpretability

Many political science journals are beginning to work with prospective authors to implement guidelines for research transparency and interpretability, and many political scientists are urging continued deliberation on how best to promote that transparency.  This post is designed to foster deliberation.  You are invited to raise questions, make observations, describe relevant experiences either within political science or in other relevant disciplines, propose revisions or new guidelines, and otherwise explore the ways in which research transparency and interpretability is being implemented, should be, or should not be.

We would also like to alert the political science community to the deliberations on qualitative research transparency currently being organized by the APSA’s section for Qualitative and Multi-Method Research (QMMR). An outline of the QMMR process can be found on the section’s website, here. The QMMR deliberations will be open to all.

Comments on any aspect of implementation are invited, but to help foster systematic discussion we offer below a few categories into which your observations might fall. Alternatively, you may identify a new category in your comment, or simply post comments without indicating a category.

We also ask all participants to remain courteous and focused on the issues at hand.

For your use, we provide URLs to some relevant materials:

Thanks very much, and we look forward to a robust and constructive discussion.

CATEGORIES

  • Confidentiality and anonymity
  • Cost in time or money
  • Transparency or interpretability in archival work
  • Transparency in big data
  • Transparency or interpretability in field work
  • Relevance for political theory or political philosophy
  • Embargos on making public new research materials
  • Editors’ decision-making process
  • Appeals from editors’ decisions
  • Consistency across journals

1 Comment

  1. As I understand it DA-RT procedures are based on the idea that evidence based knowledge claims must be explicated in ways that allow a reader to verify and validate the claims. But beneath all claims are more abstract claims that also must be validated and verified or else the hypotheses cannot be. These include models, and beneath those are theories, and beneath those are paradigms, and beneath those are metaphysics, and beneath those are… Nothing. There is no way to test or empirically substantiate a metaphysical or ontological claim independent of empirical claims that themselves will have to be based on unsubstantiated metaphysical claims. This is as true of Newtonian mechanics as it is of any social science paradigm or weltanshauung. This does not imply that as social scientists we should not encourage opportunities for systematic criticism, but we must avoid speaking and acting or justifying policies on the fundamentally unworkable and impossible principle that everything that is used to produce knowledge claims can be substantiated. To be sure, we can legitimately be asked to specify assumptions, including metaphysical commitments, but I wonder whether journals will really be able to allow article length to expand sufficiently for anything like that to be possible. I also wonder how much more difficult it will be to recruit reviewers if will be required to evaluate the entire epistemological hierarchy operating to enable knowledge claims in any particular manuscript. And then, of course, there is the question of whether the reviewers will have to abide by DA-RT standards and protocols in offering their claims about the manuscripts they evaluate.

Comments are closed.