Tentative conclusions on democracy & governance
-
How did the pollsters botch New Hampshire?
DISCLAIMER: Credit for this discovery goes to an (as yet) unnamed friend.
Those following the US presidential primaries will recall that, just about 24 hours ago, Obama was predicted to beat Clinton by a landslide. McCain’s win was expected to be much narrower than it was. How did the polls miss Clinton’s win? How did they underestimate McCain’s lead?
One explanation is that politics move quickly during primaries. Preferences change literally overnight. Not only do candidates triangulate with breakneck speed. Note the TV commentators positing women were updating in response to Obama and Edwards’ “ganging up” on Hillary. But discrepancies between the actual and expected seem too large to confirm that story.
Another concerns survey methodology, namely, the double-counting of independents.
Pollsters sample within parties, but they do not separate Republicans, Democrats and unaffiliated voters into discreet categories. To get the list of Republicans, they find all non-Democrats. To get the Democrats, they find the non-Republicans. Both samples can end up with the same independents.
So the same independents were asked whom they preferred in both fields. Their response: Romney among Republicans and Obama among Democrats. Problem one.
Yet New Hampshire’s primary (like most) is closed. Voters there must pick their party by October 12. In practice, not all independents with opinions were able to vote. Problem two.
So McCain voters were registering ‘in name only’ support for Obama, and vice versa. Meanwhile independents registering preferences in the polls were not going to the voting polls. Obama’s projected landslide rested on the preferences of non-Republicans, not Democrats, and McCain’s projected close call with Romney rested on those of non-Democrats, not Republicans.
Having just two viable parties makes it easier to do numbers work. As some of this blog’s readers have noted, though, the binary depiction of reality makes possible critical errors in measurement.
6 responses to to “How did the pollsters botch New Hampshire?”
-
Jack, actually the NH primary is open. The party affiliation deadline of Oct 12 only applies to declared voters. Undeclared voters can “declare a party at the polls, vote the ballot of that party, and then change their party affiliation back to undeclared simply by completing the form available from the Supervisors of the Checklist at the polling place,” according to the NH Sec. of State’s site, you link to above.
But it seems to me that this makes the double counting problem even more apparent. Basically, the pollsters seem to have no way of knowing what proportion of them will break R or D, especially in a race with candidates on both sides actively recruiting independent voters. The thing that bugs me, though, is that this wasn’t a surprise. Obama and McCain have both been courting independents for a long time, and obviously they were going to split the pot. Why didn’t the pollsters come up with a way to deal with this?
-
Thanks. You’re right; a “pick-a-party” format would exacerbate the effect.
Apparently some pollsters got it better. S. Taylor alludes.
-
Where are you getting this information about pollsters? New Hampshire residents can register as Dem, Republicans, or Independents, so why would pollsters assume that all non-Republican Independents are Democrats (or vice-versa?). Anyway, there are two hugely different types of surveys–those who call known registered Dems or Republicans and those that operate via a random sample. In the latter, the question asked is usually “what is your party ID?” followed by “who do you plan to vote for?” I don’t see how any of this would lead to double-counting. Maybe I’m missing something?
-
Yeah, I don’t understand the claims from Jack’s pollster friend, either.
First of all, as noted already, the primary is open. Technically, it should be called semi-closed, because open implies that anyone can show up and choose a party ballot on primary day. Whatever we want to call it, it clearly is not closed if–as is the case–those not registered with a party can come to the polls and request either major party’s ballot. I can see how pollsters could easily have missed people who thought they were undeclared but did not recall that they actually had re-registered at the polling place last time, or that they had to take the initiative to re-re-register undeclared months before this primary. But missing this type of voter does not seem to be what Jack’s pollster friend is claiming.
As for the claim that independents were being double sampled, I doubt it. My understanding (and I have no experience actually doing polling) is that voters are called and asked which primary they intend to vote in. So, they would get counted as independents voting in only one of them. Of course, these intentions might change, which could skew results.
In any event, the more likely explanations (in my assessment) are those articulated by Charles at Pollster.com. (I see there is a newer post up over there by Mark Blumenthal, which I have not yet read, and it addresses questions about turnout of independents.)
-
This was not my clearest piece of writing. In retrospect it would have been more helpful to reproduce the source verbatim.
The “double-counting” would have been of independents as a theoretical category,not literally a set of respondents.
That was the function of some pollsters having run two separate polls – one each of likely D and R voters. This is what I meant by the unclear term “list.” As Emily notes, they did ask in each poll whether the respondent planned to vote in each primary. Among undecideds and our independents category, this created the theoretical possibility for double-polling among respondents who answer “yes” to both questions. And with turnout so high, there were more of these.
By implication, then, there was either much last-minute deciding, or many expressing a preference among Democrats yet voting in the Republican primary.
-
I don’t know much about the methodology of opinion polling, but it seems that New Hampshire was somewhat of a special case. There were a high number of “undeclared” voters, able to cast ballots in either primary, who decided which one at the last minute. I wonder if it was this set of voters who, when called by pollsters, said they were definitely voting but not which primary that resulted in double counting (or at the least unreliable results).
This would conceivably force the pollsters to develop a model to predict the voting patterns of these “uncommitted” voters based on something other than their stated intentions (i.e. answers to questions on issues and their views on specific candidates).
Just an idea, but it seems like when there are a large number of voters that are, a) definitely casting ballots, and b) are undecided regarding which primary, it would lead to misleading polliing results.
Leave a reply
-
Recent Comments