The Overlooked Challenges of Generalizing About Mechanisms (QMMR4)
Colin Elman
1:30 pm – 5:30pm
INSTRUCTOR: Derek Beach, University of Aarhus, Denmark (derek@ps.au.dk)
How can we generalize within-case findings about processes and causal mechanisms to other, non-studied cases? The simple answer is that generalizations are possible to other cases that appear to be causally similar to the studied case(s). Existing guidelines suggest that the claim of similarity from the source to cases targeted for the generalization can be substantiated by comparisons – using either large-n, statistical analyses, medium-n comparative methods like QCA, or a small-n most/least-likely case logic (e.g. Lieberman, 2005; Goertz, 2017; Schneider and Rohlfing, 2013, 2016; Levy, 2008).
The core problem with existing guidelines is that they are blind to the potential of mechanistic heterogeneity existing in sets of cases that look similar at the cross-case level. If mechanisms are unpacked into their constituent working parts, how the process works can be very different in different contexts (Bunge, 1997; Falleti and Lynch, 2009; Gryzmala-Busse 2011; Steel, 2008). Mechanistic heterogeneity means that the same cause can trigger different mechanisms due to differences in contextual conditions.
However, the methods we utilize to substantiate the claim of case similarity only look at the level of causes, thereby forcing us to assume that other cases are also homogeneous at the level of mechanisms, when in fact the source case and the cases targeted for generalization might be very different at the process-level, thereby resulting in flawed generalizations.
In this short course, we will first explore the new mechanistic literature from the natural sciences and its recent applications in the social sciences. In this understanding of process, the goal of within-case analysis is to explore how things work in actual, real-world cases using mechanistic evidence. However, taking process seriously has serious but understudied implications for our ability to generalize.
We will then discuss the state-of-the-art regarding generalization about mechanisms and processes post-comparisons, and develop the problems one can encounter when assuming mechanistic homogeneity based solely on cross-case comparisons.
This is followed by an assessment of different potential techniques to deal with the problem of hidden potential mechanistic heterogeneity in sets of cases. These include theoretical tools like lifting the level of abstraction of our theories of mechanisms (and the limits of this), and the actual step-by-step probing of a population using empirical signatures of processes to determine the bounds of valid generalizations about mechanisms (Beach and Pedersen, forthcoming).
The course will utilize the recent book by Haggard and Kaufman (2016) as a working example of mechanistic heterogeneity across cases, but will also include real-world examples of mechanistic heteregeneity and what can be done about it from the field of policy evaluation.
**All Short Courses will take place on Wednesday, August 29 at the APSA 2018 Annual Meeting in Boston, MA.**