Challenges of Electoral Integrity and the Construction and Use of Expert Indicators
Pippa Norris and Svend-Erik Skanning
The Electoral Integrity Project and Varieties of Democracy
9:00 am – 5:00 pm
The event is co-organised by EIP, the Varieties of Democracy project (V-DEM), and the APSA Elections, Public Opinion, and Voting Behavior (EPOVB) organized section. THEME: The morning Workshop session will feature open EIP panels on ‘Challenges of Electoral Integrity’. The afternoon will feature panels organized by EIP and V-DEM on ‘The construction and use of expert indicators in the social sciences: Challenges of validity, reliability and legitimacy.’
Morning session on ‘Challenges of Electoral Integrity’: Electoral Integrity is understood as a complex and multidimensional concept which reflects internationally-agreed standards for the conduct of these contests. This includes flaws in the conduct of elections, raising issues of transparency, accountability, accuracy and ethical standards. Problems of integrity can influence all stages of the electoral process from franchise restrictions and voter registration procedures, boundary delimitation for electoral districts, party/candidate registration, campaigns, media, financing, voting, and vote counting, to the final declaration and outcome of the results.
Afternoon session on ‘The construction and use of expert indicators in the social sciences’: Expert surveys have become increasingly common in comparative social science, in risk analysis by private sector organizations, in evaluation research, and among NGOs and policy makers (Meyer & Booker 1991). Expert surveys are often used to deepen understanding of, among others, the left-right position of political parties and news media outlets, perceived extent of corruption or bribe-paying, and the quality of democratic governance. Expert surveys increasingly supplement alternative sources of information, such as citizen mass surveys, event analysis of media reports, and official statistics. Expert surveys seem especially useful for measuring complex concepts that require expert knowledge and evaluative judgments; and for measuring phenomena for which alternative sources of information are scarce (Schedler 2012). Yet, expert surveys are not risk free and scholars have pointed out their limitations (Budge, 2000; Mair 2001; Steenbergen and Marks, 2007). Moreover, we still lack a common methodology to construct such surveys, as well as agreed technical standards and codes of good practice. There has been debate about the pros and cons of methods used to evaluate the spatial positions of party policies, and about the use of governance indicators more generally, but there has been remarkably little discussion about the challenges of validity, reliability, and legitimacy facing the construction of expert perceptual surveys. Yet it is critical to consider these issues given the lack of a clear conceptualization and sampling universe of ‘experts’, contrasting selection procedures and reliance upon domestic and international experts, variations in the number of respondents and publication of confidence intervals, and lack of consistent standards in levels of transparency and the provision of technical information. Moreover, more research is needed on how to evaluate the consequences of expert and context heterogeneity on the validity of expert judgments (Martinez i Coma and van Ham 2015), for example by using item response models to test and correct for expert heterogeneity (Pemstein et al. 2015), and using techniques such as ‘anchoring vignettes’ (King & Wand 2007) or ‘bridge coders’ (V-Dem) to test and correct for context heterogeneity.
**All Short Courses will take place on Wednesday, August 31 at the APSA 2016 Annual Meeting in Philadelphia, PA.