Study Casts Light on Robo-Polls

A new study “makes a startling suggestion about so-called ‘robo-polls’ in the 2012 Republican presidential primaries, raising the question of whether these automated surveys may have been adjusted to match live-interviewer polls,” Gary Langer reports.

Key excerpt: “There is no difference in the accuracy of IVR polls and human polls when IVR polls occur after a human poll, but IVR polls do significantly worse if human polls are not conducted first. The apparent equivalence of IVR polls and human polls in the 2012 Republican primary appears to depend on human polls being conducted prior to the IVR polls.”

That said, the authors admit they “did not test the reverse
possibility, that traditional polls were altered to match automated

Five Polling Lessons from 2012

Harry Enten says there are five polling lessons we should take from the 2012 election season:

1. When likely and registered voter polls disagree in high turnout elections, you should usually go with the registered voter surveys.
2. Cellphones are generally needed for an accurate telephone poll.
3. Internet polling is the wave of the future.
4. Internal polls published publicly generally should not be trusted.
5. When state and national polls disagree, you should generally go with the state data.

New Robo-Poll Firm Launches

Harper Polling launches this week with the goal of putting the Republican party “on parity with Democrats in the field of IVR polling – a term that stands for interactive voice response polling, commonly known as ‘robo-polling,'” Politico reports.

“For several cycles now, Democrats have benefited from a high-volume, relatively inexpensive flow of survey data from the company Public Policy Polling, which takes hundreds of polls in any given cycle checking up on individual races and national issue debates. Some of those surveys are released to the public, while others are conducted for private purposes by Democratic campaigns and interest groups.”

Hurricane Almost Knocked Out Exit Polls

Mark Blumenthal: “What made the exit polls especially challenging this year is that Edison Research, the company that conducts the exit polls on behalf of the National Election Pool (NEP) consortium of the five television networks and the Associated Press, is in Somerville, N.J. It was directly in the path of Hurricane Sandy, and nearly knocked out of business by the storm at a critical moment in its preparations.”

“The biennial exit polls are an extraordinary undertaking under normal circumstances … Altogether, Edison reports that more than 3,000 interviewers collected nearly 120,000 interviews of Americans who voted in 2012… This year, Hurricane Sandy helped make that final week far more challenging than usual.”

Online Polls Fared Very Well

Nate Silver finds that some of the most accurate polling firms this election cycle were those that conducted their polls online.

“The final poll conducted by Google Consumer Surveys had Mr. Obama ahead in the national popular vote by 2.3 percentage points – very close to his actual margin, which was 2.6 percentage points based on ballots counted through Saturday morning. Ipsos, which conducted online polls for Reuters, came close to the actual results in most places that it surveyed, as did the Canadian online polling firm Angus Reid. Another online polling firm, YouGov, got reasonably good results.”

“Perhaps it won’t be long before Google, not Gallup, is the most trusted name in polling.”

Do Robo-Polls Cheat?

John Sides looks at new research which finds “the errors of the robo-polls were much lower when a live-interviewer poll had already been conducted in a particular state. In other words, the robo-polls were more accurate when there was a previous live-interviewer poll that may have served as a benchmark.”

From the paper: “Pollsters know their results are being compared to the results of prior polls, and polls created for public consumption have incentives to ensure that their results are roughly consistent with the narrative being told in the press if they want to garner public attention. Pollsters also have further financial incentives to get it right which may make them leery of ignoring the information contained in other polls…”

“Beyond the implications for interpreting IVR polls, the larger point here is that if polls take cues from one another, then the hundreds of polls being reported are not really as informative as the number of polls would imply.”

Do Polls Break Towards the Challenger?

Nate Silver: “There are certainly some good reasons to think that the polls could break toward Mitt Romney. For instance, many polls out now were conducted among registered voters; when pollsters switch over to likely voter polls instead — which assess each voter’s probability of actually casting a ballot on Nov. 6 — it is likely that Mr. Romney will gain a point or two. And Barack Obama obviously has a lot of weight to bear from the lukewarm economic recovery.”

“But one hypothesis you should find less persuasive is the notion that the polls will break toward Mr. Romney just because he is the challenger. It is often asserted that this is the case — that the polls move toward the ‘out-party’ candidate rather than the incumbent. But in my view the empirical evidence — although it is somewhat ambiguous — mostly argues against this idea.”

Will Romney Have Edge When Pollsters Move to Likely Voters?

Most national pollsters are still using samples of registered voters rather than “likely” voters and some suggest that Mitt Romney will have an advantage when this change is made but Mark Blumenthal suggests it’s too early.

“In almost every election dating back to 1980, the margins separating the top candidates in horse race polls shifted significantly after the party conventions. Only in 1996 did those margins remain roughly the same throughout the year. In other years, the shifts in voter preferences that occurred after the party conventions, shifts that have benefited both Democratic and Republican candidates, would have overwhelmed the relatively modest differences that earlier likely voter screens would have produced.”

“In the end, if all pollsters applied likely voter screens right now, Romney’s numbers would be slightly better, but there is a long way to go before any horse race poll should be considered an accurate forecast of the outcome.”

Why Gallup Understates Obama’s Support

Mark Blumenthal: “The misses were typically small — usually within a single percentage point, although sometimes slightly bigger — but they made a consistent impact on the non-white composition of Gallup’s samples. Instead of achieving the target of 12.1 percent black set by the March 2011 CPS, the average across the seven surveys was 11.3 percent black. Instead of hitting the CPS target of 13.7 percent Hispanic, the seven surveys averaged just 12.4 percent Hispanic.”

But he notes the “real story here is less about Gallup than about the new reality
of public opinion polling. Sophisticated random samples, live interviews
and rigorous calling procedures alone can no longer guarantee accurate
results. Today’s rapidly declining response rates require more weighting
than ever before to correct demographic skews, a phenomenon that places
growing stress on previously reliable weighting procedures.”

Pollsters Avoiding New Hampshire?

Campaigns & Elections: “To poll or not to poll in New Hampshire, that is the question for an increasing number of survey research firms this cycle. Given the state’s new push polling law, is it better for business to take work in the battleground state and risk getting hit with a fine, or is it best to boycott the state and pressure lawmakers to reform the statute?”

“It’s a Hamletesque dilemma for the polling industry, which has been rallying in recent weeks to force a change to New Hampshire’s law. In the past year, several firms have been slapped with hefty fines, and with more enforcement action in the works both sides are digging in.”