Polling

Pay Little Attention to the Alabama Polls

Politico: “Roy Moore appears to have inched back in front of Democrat Doug Jones in the latest Alabama Senate election polls, according to the oft-cited RealClearPolitics average — a change in fortune from mid-November, when sexual misconduct allegations against Moore first surfaced. The reality? No one really has a clue about where things stand with Alabama voters in the December 12 special election.”

“For all the national attention and the millions of dollars spent to win the seat, there’s relatively little public polling in the contest. Only three public surveys in the average have been conducted since the Thanksgiving holiday, and odds are you’ve never heard of two of the three pollsters. And that’s precisely the problem. The most important and closely watched election in the nation is taking place in the equivalent of a polling black box.”

G. Elliot Morris: The media stopped covering Roy Moore’s sex scandal, then he bounced back.

Many Pollsters Haven’t Changed Anything

Nate Cohn: “A year after polls broadly overestimated Hillary Clinton’s strength in the decisive Rust Belt battleground states, top pollsters and analysts across the survey industry have reached a broad near-consensus on many of the causes of error in the 2016 presidential election. But so far, public pollsters — typically run by news outlets and colleges — have not changed much about their approach. Few if any of the public pollsters that conducted surveys ahead of Tuesday’s elections for governor in Virginia and New Jersey appear to have adopted significant methodological changes intended to better represent the rural, less-educated white voters who pollsters believe were underrepresented in pre-election surveys.”

“On the other hand, private pollsters — typically employed by campaigns and parties — have already begun to make changes. This is especially true among Democrats stunned by Donald Trump’s upset victory, but Republicans are making changes as well. The adjustments are already playing out in Virginia, where pollsters will have one of their first chances to put postelection shifts to the test.”

U.K. Turnout Models Ignored Surge In Youth Vote

Nate Silver: “The 2017 election therefore seems to be a case of an overcorrection. The pollsters apparently did a good enough job of weighting the raw samples properly, which got them fairly close to the right outcome. Then on top of that, some of them gave extra weight to the Conservatives through their turnout models. As a result, they discounted signs of a youth-driven Labour turnout surge. As was the case in the U.S. with Bernie Sanders, younger voters turned out in a big way for Labour’s left-wing leader, Jeremy Corbyn.”

“It’s one thing for a pollster to get an outcome wrong because voters fail to turn out when they say they will. But if voters tell you they’re going to turn out, you ignore them, and they show up to vote anyway, you really don’t have much of a defense.”

Conventional Wisdom May Be Contaminating Polls

Nate Silver: “When the conventional wisdom tries to outguess the polls, it almost always guesses in the wrong direction. Many experts expected Le Pen to beat her polls. Currency markets implied that she had a much greater chance — perhaps 20 percent — than you’d reasonably infer from the polls. But it was Macron who considerably outperformed his numbers instead.”

“While this was somewhat amusing — the one time the experts decided to take the nationalist candidate’s chances really seriously was the time she lost by 32 points — it should actually worry you, even if you’re a ‘fan’ of polling and data-driven election forecasting. It’s a sign that the polls may be catering to the conventional wisdom, and becoming worse as a result.”

Democrats Burned by Polling Blind Spot

“As they investigate the forces behind the party’s stunning losses in November, Democrats are coming to a troubling conclusion. The party didn’t just lose among rural white voters on Election Day, it may have failed to capture them in its pre-election polling as well,” Politico reports.

“Many pollsters and strategists believe that rural white voters, particularly those without college degrees, eluded the party’s polling altogether — and their absence from poll results may have been both a cause and a symptom of Donald Trump’s upset victory over Hillary Clinton in several states.”

Why Polls Differ on Trump’s Popularity

Nate Silver: “Here’s what we can say for sure: It’s unprecedented for a president to face so much opposition from the electorate so soon. Recent polls show that anywhere between 43 and 56 percent of Americans disapprove of President Trump’s job performance. Even if you take the low end of that range, Trump’s numbers are much worse than any past president a month into his term.”

“But beyond that, there’s a lot of seeming disagreement in the polls about exactly how unpopular Trump is — and even whether his disapproval rating exceeds his approval rating at all.”

“What’s the real story? The differences between the polls aren’t random, or at least they don’t appear to be based on the relatively limited amount of data we have so far. Instead, Trump’s approval ratings are systematically higher in polls of voters — either registered voters or likely voters — than they are in polls of all adults. And they’re systematically higher in polls conducted online or by automated script than they are in polls conducted by live-telephone interviewers.”

Is Trump More Popular Than the Polls Suggest?

Politico: “The debate is a flashback to last fall’s election — in which Trump ran ahead of his poll numbers, particularly in the Upper Midwest states that propelled him to victory. And just like during the campaign season, there’s evidence suggesting that Americans may be less willing to admit they support the president and his actions if they are talking to another person on the phone, compared to polls completed with the anonymity of the internet or an automated phone interface.”

Nate Silver, Then and Now

“The ‘fundamentals’ models, in fact, have had almost no predictive power at all. Over this 16-year period, there has been no relationship between the vote they forecast for the incumbent candidate and how well he actually did.”

— Nate Silver, writing in FiveThirtyEight on March 26, 2012.

“Reporting largely ignored the importance of economic conditions and other ‘fundamentals’ that implied a potentially close race.”

— Silver, writing in FiveThirtEight on January 23, 2017, criticizing media coverage of the 2016 campaign.

Pollsters Missed Less-Educated White Voters

Nate Silver: “In the 10 states with the largest share of white voters without college degrees, Trump beat his polling average by an average of 8 percentage points — a major polling miss. But in the 10 states with the lowest share of white voters without college degrees, Clinton beat her polls by an average of 3 points (or 4 points if you count the District of Columbia as a state). Overall, the correlation between the share of white non-college voters in a state and the amount by which Trump overperformed (or underperformed) his polls is quite high.”

A Catastrophic Polling Error

Politico: “It’s possible Donald Trump’s upset victory this week was powered by a surge of late deciders. Or the mysterious group often referred as “shy Trump” voters somehow escaped their radar. Many in the polling industry are also second-guessing their turnout modeling, trying to discern whether there’s a serious flaw that went unnoticed.”

“No matter the root cause, an industry already reeling from a series of misses in the United States and overseas is engaging is a round of serious introspection. While the data streams required to evaluate whether they modeled the electorate incorrectly — or whether Trump voters disproportionately wouldn’t respond to polls — won’t be available for months, already the nation’s leading professional organization of pollsters is admitting they “clearly got it wrong this time” and pledging to study the causes of the errors.”