David M Rothschild on Posted on

Together with Sam Corbett-Davies and Tobias Konitzer, I ran regular polling on a display poll on MSN. We used the data to discuss support of public policy and quick reactions to unfolding events. The idea was that in 2016 we would study the data collection and analytics to nurture new processes, answering pressing questions now, and then use them for election forecasts rather than tradition polling when it was tested. While we did not hide it, I made a mistake by not pushing the MSN voter intention polling (46 of 51) harder. It persistently pointed to Trump in Wisconsin and Michigan, and tightness in the rest of the rust belt. While the binary accuracy was similar to top public polling, this experimental poll consistently pointed to Clinton’s trouble in the rust belt that ultimately cost her the election to Trump, with: a more Trump leaning voter population and more support for Trump from key demographics.

The public opinion polling we did throughout the year had an impact on the narrative of what people where thinking about public policy and how they were absorbing the campaign. We asked questions about support for dozens of public policy issues ranges from guns to immigration to taxes. We were able to show people answers that may have defied their expectations with unity (everyone like tax increases on the rich and no one wants to increase the flow of immigrants) or confusion (there is no clear partisan divide free trade).

But, while we disseminated articles about public opinion regularly, we only talked sparingly about the voter intention polling. With so much publicly available polling, we figured that this was a great year to build out the technically, but could we really add to the discourse? Well, it turned out that the amount of binary misses in the public polling is not extraordinary, but the public polling missed three critical states: Michigan, Pennsylvania, and Wisconsin. Overall, the national polling average may be off 2 pp when the votes are all counted; not bad. But, public polling missed something about either about white turnout or support in one region and that cost them the only thing that mattered, the winner of the Electoral College. And the MSN polling was able to target detailed geographical and demographic sub-groups. Just look at the Final map we put up before Election Day. We had most states like most public polling, with the glaring exception of Wisconsin and Michigan stubbornly pointing towards Trump:

PredictWise20161110c

Polling does two things: estimate the voter population and the support for each candidate from the voter population. 

The voter population that my colleagues and I created for the MSN polling more closely resembled the true voting population than the estimates from the public polling. This is best demonstrated by Nate Cohn’s interesting experiment of giving four different pollsters (and himself) the exact same polling data and see what topline numbers they generate. We certainty absorbed some of Latino population into our estimate of White votes (i.e., we had too few Latino and too much white, compared with the Florida exit polls, for whatever exit polls are worth). But, we ultimately had a more Republican/Trump make-up state-by-state: older, whiter, and less educated, than the what the polls estimated. That is why for the same sentiment data, our projections were constantly more Trump.

PredictWise20161109c

Further, we had support levels for Trump and Clinton that closely resembled the exit polling. While it dipped at points, towards the end we consistently had Republicans supporting Trump at a rate of 2 pp more than Democrats supported Clinton. To be honest, I was concerned; what had we done wrong? But, the MSN data was solid there. Further, we had strong support for Trump among older males (not surprising), but also surprising tightness in older females. We did not have the ability to parse this data by race, but we anticipate that if we could, it would have shown a strong than expected return from white females driving that. That is probably reflected in our state estimates from states with higher percentage white populations. All of this is backed up by the polling data and why our estimates from MSN were throwing rust-belt states towards Trump.

I believe that we were reaching hard to reach folks in the middle of America that were missed by traditional polling. Now, more than ever, we need a population that understands each other, how we feel about different topics. And, if you think of this in-terms of market intelligence in general, we need to develop strategies that focus on capturing not just people on average (like the national polling), but really can understand detailed demographics (like White/non-college/Wisconsin). That is not just important for elections, but in an increasingly personalized world, for all marketing. I believe we do can this with the right analytics on display-based polling.