Tobias Konitzer and David Rothschild on Posted on

This is the first piece of our series, Evolution of Polling Samples from RDD to RDE

In part 2: we discuss Online Panels which are increasingly replacing RDD.
In part 3: we discuss Assisted Crowdsourcing, a brand-new form of polling.
In part 4: we introduce Random Device Engagement (that is what we do!).

Part 1: Old polling at the brink of irrelevance

Random Digit Dialing (RDD): identify a cluster of phone numbers that have reasonable demographic and geographic representation. Then, start calling these numbers at random, trigger a response, and collect poll answers over the phone. The mode is confined, by definition, to a telephone, but it has recently expanded to both landline and cell phones. The mode has high coverage (in that most people have either or both a landline and cell phone), but coverage becomes harder to assess while landline penetration is dropping as cell phone penetration is rising. This makes it hard for survey researchers to map the population in either group or any individual’s inclusion in either group. Response rates are oftentimes in the single digits.

2015 and 2016 saw high-profile polling failures throughout the world. In the summer of 2015, before Brexit and the 2016 US election, The New York Times asked, somewhat rhetorically: “What is the matter with political polling?” Implying that there was already a crisis of confidence in polling. Then in 2016, Great Britain stunned the world by voting in favor of the Brexit – a referendum in favor of leaving the European Union – despite opinion polls shifting towards remain in the last few days. A few months later, despite polling showing Democratic candidate Hillary Clinton winning in enough states to win the US election, and poll aggregators confidently pointing to a Clinton victory, Republican candidate Donald Trump won a fairly comfortable Electoral College victory (but, still lost the popular vote). While there is some nuance to the label of “failure” – the popular vote was forecast spectacularly well by polling aggregators, and “failure” was really a local phenomenon boiling down to a number of state-level polls in the Rust Belt (and applied to the presidential election only, and not congressional elections), the public perception was that of “failure” in broad and absolute terms. As is now well known, this failure (or at least either perception of failure or partial failure) led to a reckoning with the status-quo modus operandi of polling; the whole industry faced a market-threatening question of “Quo Vadis?”.

Culprits were easily identified: Of course, RDD polls, the gold standard of high-quality polling in recent decades, has undergone transformational paradigm shifts: RDD response rates have decreased from 36% in 1997 to single digits in the 2010s. And, this non-response is coupled to political attitudes: today, traditional polls, RDD with a mix of landlines and cellphones, have a hard time reaching those with lower levels of education and lower levels of political knowledge. In consequence, polls in 2016, especially the crucial state-level polls in the contiguous states of the Rust Belt, neglected to weight on education – in hindsight a crucial mistake. Polls have a hard time reaching White blue-collar voters, dubbed Bowling Alone Voters, especially mobile blue-collar voters (“Truck Driver phenomenon”), as a Post Mortem by Civis Analytics has pointed out. In consequence, structural problems with polling have led, for the first time in decades, to a series of innovation.

Even more serious than its current problems: even if RDD can still work, it is doomed in next few years. Do you have a landline? Do you answer unknown (or suppressed) numbers on your cell phone? Will you have a cell phone in 10 years? Will the platform for reaching you be a phone number or a user ID? These are serious questions that further jeopardize the future of random digit dialing: it just does not work without phones!

As is the case with all innovation, some innovation is good and scientifically sound, some innovation is snake oil, with little or no effect, and some innovation is flat-out dangerous. In the remainder of the series, we shed light on three such innovations. Online (non-)probability panels, Assisted Crowdsouring, and Random Device Engagement (RDE). All innovations come with strengths and weaknesses. But, as we spell out here, one is the clear winner: RDE, which is why RDE is at the core of the PredictWise methodology.