David M Rothschild on Posted on

Huffington Post’s Pollster does not include the USC/LA Times poll in their general election polling trend, but RealClearPolitics does. And, today, August 20, 2016 the USC/LA Times poll has Republican Donald Trump up 44.2 to 43.6. I do not believe the level of the poll (i.e., the head-to-head value), but I believe there is a lot of information in the movement of the poll. The sharp movement towards Trump does mean a movement towards Trump among the respondents in their panel, but their actual value of Trump up by 0.8 percentage points could be way off.

The poll uses a panel of 3,200 people who answer their demographics at the start of July and then answer their voting intention once per week through Election Day. They provide each respondent in a trailing seven day period with a weight that assures the sample resembles the voting population from 2012. Then, they further weigh each respondent by their stated likelihood to vote. Then, they report the fraction voting for each candidate.

I love this type of experimental polling, but there are few serious concerns on their methods:

1) Party ID: They are weighing people by their 2012 vote as proxy for latent party identification. This is a bad proxy, because a person’s four year old vote is actually more susceptible to change than their current stated party identification. You read that right. People have a serious problem remembering if and for whom they voted for in past elections. Generally, people overstate their vote for the winning candidate. What that means is the “Romney voters” in their panel are probably a more hardcore sub-section of Romney voters than actual Romney voters.

2) Modelling: They are raking their weights, rather than modeling the data. Depending on how representative their sample is to begin with and the randomness of the dropouts over time, once the weights get lager they become quite an issue. Modeling the data with some form of hierarchical regression provides additional power. I am particularly concerned with African-American support for Clinton dropping from 90 to 80 percentage points (and Trump’s support rising from near 0 to 13.6 percentage points). Could smaller demographics groups, like African-Americans, have too big of weights due to under-representation in the poll?

3) Probability of Voting: While some good work has been done in the past on asking probability of voting, it is not clear how well it will hold up in an election like this. It is possible that inferring likely voting (from past voting records and other implicit questions) would actually be a more stable and realistic measure of likeliness of voting. Asking the respondents probably exaggerates shifts. Further, why derive this each week anew, when they have a full panel of data on the respondents? Surely they can model the likeliness to vote more efficiently with all of that response data.

Most likely the party ID issues is making the poll a few points too favorable for Trump.

But, does that not discount that the movement may still hold valuable information about the race tightening a little. They have a relatively steady group of people and show a 2.7 percentage point drop in support for Clinton from her peak and 2.6 percentage point increase for Trump from his bottom. Someone is moving towards Trump and someone is moving away from Clinton, but it is not clear from where and by how much.