Phoning It In: Overcoming Implementation Challenges in Field-Experiment Partnerships
by Brian Robert Calfano, University of Cincinnati
The limitations of data generated through observational (e.g., survey) and experimental research in artificial settings (such as laboratories) often compel scholars pursue field experiments. But problems in execution may occur, especially in cases where the researcher cannot be present for treatment delivery. In this article, I briefly describe a long-distance researcher–organization partnership in conducting pro-LGBT canvassing in advance of an April 2015 public referendum on a housing nondiscrimination ordinance in a Midwestern city; discuss (unintended) noncompliance in treatment application; estimate treatment effects following Nickerson’s (2005) protocol when noncompliance occurs; and recommend what researchers can do to avoid these design issues when partnering long distance with an organization to conduct field interventions. Specific recommendations include assuming that problems will arise with long-distance partnerships, developing easy-to-use reference guides for organization works tasked with treatment delivery, engaging in frequent data updates with the partner organization, encouraging detailed record keeping, developing alternate collaboration points with organization partners, being prepared for the possibility that partners abandon the design protocol midstream, and taking ownership as the researcher for any issues that arise with the design—it’s researcher’s responsibility to successfully implement the design, not the partnering organization. Successful field experiment partnerships require researchers to plan ahead, especially when they cannot be present to oversee a design. The article details the problems and recommendations for researchers. Overseeing field experiments from afar requires researchers to think and plan ahead.