Do Robo-Polls Cheat?

John Sides looks at new research which finds “the errors of the robo-polls were much lower when a live-interviewer poll had already been conducted in a particular state. In other words, the robo-polls were more accurate when there was a previous live-interviewer poll that may have served as a benchmark.”

From the paper: “Pollsters know their results are being compared to the results of prior polls, and polls created for public consumption have incentives to ensure that their results are roughly consistent with the narrative being told in the press if they want to garner public attention. Pollsters also have further financial incentives to get it right which may make them leery of ignoring the information contained in other polls…”

“Beyond the implications for interpreting IVR polls, the larger point here is that if polls take cues from one another, then the hundreds of polls being reported are not really as informative as the number of polls would imply.”

Do Polls Break Towards the Challenger?

Nate Silver: “There are certainly some good reasons to think that the polls could break toward Mitt Romney. For instance, many polls out now were conducted among registered voters; when pollsters switch over to likely voter polls instead — which assess each voter’s probability of actually casting a ballot on Nov. 6 — it is likely that Mr. Romney will gain a point or two. And Barack Obama obviously has a lot of weight to bear from the lukewarm economic recovery.”

“But one hypothesis you should find less persuasive is the notion that the polls will break toward Mr. Romney just because he is the challenger. It is often asserted that this is the case — that the polls move toward the ‘out-party’ candidate rather than the incumbent. But in my view the empirical evidence — although it is somewhat ambiguous — mostly argues against this idea.”

Will Romney Have Edge When Pollsters Move to Likely Voters?

Most national pollsters are still using samples of registered voters rather than “likely” voters and some suggest that Mitt Romney will have an advantage when this change is made but Mark Blumenthal suggests it’s too early.

“In almost every election dating back to 1980, the margins separating the top candidates in horse race polls shifted significantly after the party conventions. Only in 1996 did those margins remain roughly the same throughout the year. In other years, the shifts in voter preferences that occurred after the party conventions, shifts that have benefited both Democratic and Republican candidates, would have overwhelmed the relatively modest differences that earlier likely voter screens would have produced.”

“In the end, if all pollsters applied likely voter screens right now, Romney’s numbers would be slightly better, but there is a long way to go before any horse race poll should be considered an accurate forecast of the outcome.”

Why Gallup Understates Obama’s Support

Mark Blumenthal: “The misses were typically small — usually within a single percentage point, although sometimes slightly bigger — but they made a consistent impact on the non-white composition of Gallup’s samples. Instead of achieving the target of 12.1 percent black set by the March 2011 CPS, the average across the seven surveys was 11.3 percent black. Instead of hitting the CPS target of 13.7 percent Hispanic, the seven surveys averaged just 12.4 percent Hispanic.”

But he notes the “real story here is less about Gallup than about the new reality
of public opinion polling. Sophisticated random samples, live interviews
and rigorous calling procedures alone can no longer guarantee accurate
results. Today’s rapidly declining response rates require more weighting
than ever before to correct demographic skews, a phenomenon that places
growing stress on previously reliable weighting procedures.”

Pollsters Avoiding New Hampshire?

Campaigns & Elections: “To poll or not to poll in New Hampshire, that is the question for an increasing number of survey research firms this cycle. Given the state’s new push polling law, is it better for business to take work in the battleground state and risk getting hit with a fine, or is it best to boycott the state and pressure lawmakers to reform the statute?”

“It’s a Hamletesque dilemma for the polling industry, which has been rallying in recent weeks to force a change to New Hampshire’s law. In the past year, several firms have been slapped with hefty fines, and with more enforcement action in the works both sides are digging in.”

Approval Ratings are Still Important

Nate Silver: “In the early stages of general election campaigns, a president’s approval ratings have often been at least as accurate a guide to his eventual performance as the head-to-head numbers. Thus, for at least the next couple of months, I would pay as much attention to Mr. Obama’s approval ratings as his head-to-head polls against Mr. Romney.”