Why Pollsters Should Poll Until the End

Carl Bialik: “Most Iowa polls showed Donald Trump winning the state’s Republican caucuses. He didn’t. Some Iowa polls showed Hillary Clinton winning Iowa easily. She didn’t. It’s notoriously hard to poll Iowa, but what can pollsters learn from Monday night’s results to improve their work over the next few months — and for the 2020 caucuses?”

“One of the biggest lessons was a simple one: Keep on contacting voters as late as possible.”

Iowa Polls Were Way Off on Trump

Nate Cohn: “Mr. Trump was at 31 percent in the final polls, but finished with just 24 percent. In our data set of early primary polls from New Hampshire and Iowa since 2004, no candidate underperformed the final surveys by as much as Mr. Trump. Mrs. Clinton, for instance, mainly beat Mr. Obama by outperforming her polling, not because Mr. Obama fell short.”

“It’s always hard to figure out why polls are wrong, but this time the stakes are higher. Republican strategists have hoped for months that Mr. Trump’s lead was an illusion. The results in Iowa at least raise the possibility that they’re right — which would call into question Mr. Trump’s advantage elsewhere.”

“This time there is evidence to support one of two possibilities for why polls overestimated Mr. Trump: Voters broke strongly against Mr. Trump in the final days or the electorate was more conservative and more religious than polls anticipated.”

In Defense of Iowa Polling

Sam Wang: “There seems to be a persistent meme that polls are in trouble. There was no evidence for this. Primaries and caucuses are volatile situations – this is a well-known fact. I have been assuming that home-stretch polls can be off by an average of 5 percentage points. Any fuss tonight is based on the fact that in Iowa, with its tiny turnout and odd voting procedure, Trump was polling 3 points ahead of Cruz, and ended up losing by 3 points. It would be a mistake to conclude that Trump’s support is illusory in other states. Quite the opposite. A 6-point error would not affect his ranking anywhere else. For now.”

Politico: “According to calculations from the website FiveThirtyEight back in 2014, the average polling error in presidential primary and caucus polls in the final three weeks before an election has hovered between 7 and 9 percentage points.The results for the Iowa GOP caucuses fall within that error range.”

The Most Important Poll Is Coming Tonight

This piece is only available to Political Wire members.

Sign in to your account or sign up today!

Members get exclusive content, new features and no advertising.

Extended Special Offer: The next 25 new annual members will receive free pack of Politicards, a deck of 54 playing cards (including jokers!) depicting the 2016 candidates and pundits.

Monthly – $5 per month or Annual – $50 per year


Likely Voter Models Are Failing

Pew Research: “In recent years, polling has missed the mark in several high-profile elections, drawing particular attention to the difficulties inherent in using surveys to predict election outcomes. These failures typically result from one or more of three causes: biased samples that include an incorrect proportion of each candidate’s supporters; change in voter preferences between the time of the poll and the election; or incorrect forecasts about who will vote. While not a new concern, the third of these – the difficulty of identifying likely voters – may be the most serious, and that is the focus of this study. Election polls face a unique problem in survey research: They are asked to produce a model of a population that does not yet exist at the time the poll is conducted, the future electorate.”

Pollsters Say Their Reputations Have Suffered

Carl Bialik: “No votes have been cast yet in the 2016 election, but there may already be one set of losers in the campaign: pollsters’ reputations. And that’s according to the pollsters themselves.”

“We asked people working at some of the nation’s most prominent polling outfits whether pollsters’ public image has improved or declined since the 2012 election. Of the 21 who answered, none said their public image had improved, and two-thirds said it had declined.”

Paying Survey Respondents Could Mean Better Answers

New York Times: “When survey respondents were offered a small cash reward — a dollar or two — for producing a correct answer about the unemployment rate and other economic conditions, they were more likely to be accurate and less likely to produce an answer that fit their partisan biases.”

“In other words, when money was added to the equation, questions about the economy became less like asking people which football team they thought was best, and more like asking them to place a wager. Even a little bit of cash gets people to think harder about the situation and answer more objectively.”

An Explosion of Polls

“The number of polls of Republican voters in the first three primary and caucus states — Iowa, New Hampshire, and South Carolina — has skyrocketed nearly 90 percent compared with the 2012 GOP primary,”: according to a Boston Globe review of polls.

“The trend toward saturation polling shows little sign of abating, with online polls now cheaper than ever and polling firms and universities competing to satisfy an insatiable media appetite for the latest upticks and downturns, the trends in the minute-by-minute drama of the contest.”

Polls May Underestimate Trump’s Support

A Morning Consult analysis looks at the fact that Donald Trump generally has done better in online polls than in phone surveys.

Los Angeles Times: “The firm conducted an experiment aimed at understanding why that happens and which polls are more accurate — online surveys that have tended to show Trump with support of nearly four-in-10 GOP voters or the telephone surveys that have typically shown him with the backing of one-third or fewer. Their results suggest that the higher figure probably provides the more accurate measure.”

“Some significant number of Trump supporters, especially those with college educations, are ‘less likely to say that they support him when they’re talking to a live human’ than when they are in the ‘anonymous environment’ of an online survey.”

Polls 60 Days Out Aren’t Very Helpful

Alfred Tuchfarber: “Bottom line, the national polls are very poor predictors of primary/caucus results. While the state polls are better, they are still faced with the very difficult task of basing their predictions on what is at best an artful guess as to which individuals in the samples will actually vote.”

“The current polls almost surely do not provide a good read on the winners of the early February contests in Iowa and New Hampshire. Rather than relying on these polls to handicap the races, an eclectic multi-factor approach is necessary. That analysis needs to take into account history, candidate strengths and weaknesses, money, ground games, likely turnout, intensity of support, and interest in the election, among other factors.”

Two Trend Lines in Political Polling

Nate Cohn: “One is the rise of online polling. The other is the decline of telephone surveys because of rising costs and possibly declining quality, tied to the fact that fewer Americans are willing to participate in a telephone poll. Many public opinion researchers have thought that Internet polling would eventually overtake phone polling, but they hoped it would not happen before online polling was ready.”

“Ready or not, online polling has arrived. Political analysts and casual poll readers now face a deluge of data from new firms employing new, promising, but not always proven methodologies. Nowhere is the question of the accuracy of the new online polls more evident than in the survey results for Donald Trump. He fares better in online polls than in traditional polls, and it’s not clear which method is capturing the public’s true opinions.”

Can Polls Really Be Trusted Anymore?

New Yorker: “Election pollsters sample only a minuscule portion of the electorate, not uncommonly something on the order of a couple of thousand people out of the more than two hundred million Americans who are eligible to vote. The promise of this work is that the sample is exquisitely representative. But the lower the response rate the harder and more expensive it becomes to realize that promise, which requires both calling many more people and trying to correct for “non-response bias” by giving greater weight to the answers of people from demographic groups that are less likely to respond… A typical response rate is now in the single digits.”

“Meanwhile, polls are wielding greater influence over American elections than ever.”

New Exit Polls Being Tested

“When voters go to the polls in Kentucky and Mississippi on Nov. 3, The Associated Press will be launching experiments aimed at finding more accurate and less expensive ways to survey them. With funding from the Knight Foundation, the AP has hired GfK Custom Research to identify and survey voters online, instead of conducting a traditional exit poll where precinct-based interviewers ask voters to fill out questionnaires,” the AP reports.

Gallup Sits Out the Primaries

“After a bruising 2012 cycle, in which its polls were farther off than most of its competitors, Gallup told Politico it isn’t planning any polls for the presidential primary horse race this cycle. And, even following an internal probe into what went wrong last time around, Gallup won’t commit to tracking the general election next year.”

“It’s a stunning move for an organization that built its reputation on predicting the winners of presidential elections. But it comes at a time of unusual tumult in the polling world.”