PrintJanuary 9, 2008
Pollsters, Pundits have egg on their faces
There are a lot of people who are eating some crow this morning.* Pollsters, pundits, even professional political operatives all got it wrong in the New Hampshire Democratic primary.
Hillary Clinton not only defied the odds, she defied expectations, the polls, and the prognosticators who had written her off on Monday - just 24 hours before the voting began. And the professional psephplogists who make a living analyzing and predicting elections are paying for it today with a big hit on their credibility:
There will be a serious, critical look at the final pre-election polls in the Democratic presidential primary in New Hampshire; that is essential. It is simply unprecedented for so many polls to have been so wrong.Last night's results will give ammunition to those who believe that polls are rigged or biased. There have been incidences of results-driven polls but those have always been from the public polls taken by media outlets. What made last night such a shocker was that apparently even internal polling from the campaigns got it wrong as well.
We need to know why. But we need to know it through careful, empirically based analysis. There will be a lot of claims about what happened - about respondents who reputedly lied, about alleged difficulties polling in biracial contests. That may be so. It also may be a smokescreen - a convenient foil for pollsters who'd rather fault their respondents than own up to other possibilities - such as their own failings in sampling and likely voter modeling.
There have been previous races that misstated support for black candidates in biracial races. But most of those were long ago, and there have been plenty of polls in biracial races that were accurate. (For more on past problems with polls in biracial races, see this blog I wrote for Freakonomics last May.)
And there was no overstatement of Obama in Iowa polls. On the other hand, the pre-election polls in the New Hampshire Republican race were accurate. The problem was isolated to the Democratic side - where, it should be noted, we have not just one groundbreaking candidate in Barack Obama, but also another, in Hillary Clinton.
Those guys make their livings off being right. They can't afford to be biased or inaccurate. So when all the polls are so far off, something else is at work, something the pollsters couldn't see and take into account.
Pollster Frank Luntz said on Fox News last night that it was the turnout models that did pollsters in and that the models they are using now are "broken" and pollsters must go back and create models that better reflect what happened in New Hampshire. Obama (and Hillary) are apparently attracting so many new voters - and many more voters in the cumulative - that it's just too difficult to get an accurate prediction of turnout.
We'll see if they can improve as we go along. But one thing is absolutely certain; neither Obama or Hillary should believe any polls they are seeing until the pollsters can prove they have a handle on the new situation.
* Lest there are those of you who think I exclude myself, here's my mea culpa.
