Polling

Why Polls Differ on Trump’s Popularity

Nate Silver: “Here’s what we can say for sure: It’s unprecedented for a president to face so much opposition from the electorate so soon. Recent polls show that anywhere between 43 and 56 percent of Americans disapprove of President Trump’s job performance. Even if you take the low end of that range, Trump’s numbers are much worse than any past president a month into his term.”

“But beyond that, there’s a lot of seeming disagreement in the polls about exactly how unpopular Trump is — and even whether his disapproval rating exceeds his approval rating at all.”

“What’s the real story? The differences between the polls aren’t random, or at least they don’t appear to be based on the relatively limited amount of data we have so far. Instead, Trump’s approval ratings are systematically higher in polls of voters — either registered voters or likely voters — than they are in polls of all adults. And they’re systematically higher in polls conducted online or by automated script than they are in polls conducted by live-telephone interviewers.”

Is Trump More Popular Than the Polls Suggest?

Politico: “The debate is a flashback to last fall’s election — in which Trump ran ahead of his poll numbers, particularly in the Upper Midwest states that propelled him to victory. And just like during the campaign season, there’s evidence suggesting that Americans may be less willing to admit they support the president and his actions if they are talking to another person on the phone, compared to polls completed with the anonymity of the internet or an automated phone interface.”

Nate Silver, Then and Now

“The ‘fundamentals’ models, in fact, have had almost no predictive power at all. Over this 16-year period, there has been no relationship between the vote they forecast for the incumbent candidate and how well he actually did.”

— Nate Silver, writing in FiveThirtyEight on March 26, 2012.

“Reporting largely ignored the importance of economic conditions and other ‘fundamentals’ that implied a potentially close race.”

— Silver, writing in FiveThirtEight on January 23, 2017, criticizing media coverage of the 2016 campaign.

Pollsters Missed Less-Educated White Voters

Nate Silver: “In the 10 states with the largest share of white voters without college degrees, Trump beat his polling average by an average of 8 percentage points — a major polling miss. But in the 10 states with the lowest share of white voters without college degrees, Clinton beat her polls by an average of 3 points (or 4 points if you count the District of Columbia as a state). Overall, the correlation between the share of white non-college voters in a state and the amount by which Trump overperformed (or underperformed) his polls is quite high.”

A Catastrophic Polling Error

Politico: “It’s possible Donald Trump’s upset victory this week was powered by a surge of late deciders. Or the mysterious group often referred as “shy Trump” voters somehow escaped their radar. Many in the polling industry are also second-guessing their turnout modeling, trying to discern whether there’s a serious flaw that went unnoticed.”

“No matter the root cause, an industry already reeling from a series of misses in the United States and overseas is engaging is a round of serious introspection. While the data streams required to evaluate whether they modeled the electorate incorrectly — or whether Trump voters disproportionately wouldn’t respond to polls — won’t be available for months, already the nation’s leading professional organization of pollsters is admitting they “clearly got it wrong this time” and pledging to study the causes of the errors.”

The Polls Were Right and Then They Were Wrong

Erick Erickson: “I got it absolutely wrong. I have been writing since February that he would not win. The great paradox of this election for me is that I never saw Mr. Trump as a legitimate contender in the primaries because I believed the polling was wrong, but it was right all along. In the general election, I presumed he would never win because the polling was right, but it was wrong all along.”

The Polls Dramatically Missed Trump

FiveThirtyEight: “The more whites without college degrees were in a state, the more Trump outperformed his FiveThirtyEight polls-only adjusted polling average,1 suggesting the polls underestimated his support with that group. And the bigger the lead we forecast for Trump, the more he outperformed his polls.2 In the average state won by Trump, the polls missed by an average of 7.4 percentage points (in either direction); in Clinton states, they missed by an average of 3.7 points. It’s typical for polls to miss in states that aren’t close, though.”

“The most important concentration of polling errors was regional: Polls understated Trump’s margin by 4 points or more in a group of Midwestern states that he was expected to mostly lose but mostly won: Iowa, Ohio, Pennsylvania, Michigan, Wisconsin and Minnesota.”

Polls Underestimated Trump’s Support

Vox: “The polls were quite close in their forecast for Clinton’s support. Clinton was expected to get 47 percent of the vote; she’s got 47.7 percent so far. On average, the polls understated her state-level support by about a percentage point.”

“Trump, however, was expected to get 44 percent of the vote, and he now has 47.5 percent; the polls undershot his support considerably… As the undecided made up their minds, and as third-party supporters moved toward the major party candidates (as they tend to do in a campaign’s final days), it looks like Trump picked up almost all of them.”

The Polls, Forecasts and Political Scientists Were All Wrong

NBC News: “The polls were wrong — all of them. So were the experts, the political consultants and the seasoned officials in both parties who predicted Donald Trump would lead his party to ruin. So were the markets, which had anticipated a Hillary Clinton victory and crashed overnight as her electoral firewall caught ablaze. So was every living president and past presidential nominee of both parties, save Bob Dole, all of whom opposed Trump. So was this reporter and so many like him — and Trump told me as much.”

“Even the GOP polling numbers and analytics we were privy to didn’t show Trump winning. The biggest mistake we made was viewing what seemed to be a very stable race in the polls (with Clinton holding a durable lead), but what ultimately turned into anything but.”

Jake Tapper, on CNN:  “It’s going to put the polling industry out of business. It’s going to put the voter projection industry out of business.”

The National Polls Suddenly Converge

Nate Silver: “Seven of the 19 polls have Clinton leading by 4 points; another four have her ahead by 3 points… It’s worth raising an eyebrow, though, when the polls (other than the L.A. Times) show a range this tight at the end of an election, especially given that they’d diverged so much earlier in the campaign. That probably reflects some degree of herding — for instance, because pollsters stick surveys that seem to be outliers in a file drawer rather than publishing them. So the tight range of polls shouldn’t be taken to mean that everyone’s figured exactly how to poll this challenging election just in the nick of time.”

“Still, the polls clearly agree that Clinton is the favorite, and perhaps has a slight wind at her back for Election Day.”

Nate Silver Dethroned as King of Political Stats

Wired: “Forget Nate Silver. There’s a new king of the presidential election data mountain. His name is Sam Wang, Ph.D.”

“Haven’t heard of him just yet? Don’t worry. You will. Because Wang has sailed True North all along, while Silver has been cautiously trying to tack his FiveThirtyEight data sailboat (weighted down with ESPN gold bars) through treacherous, Category-Five-level-hurricane headwinds in what has easily been the craziest presidential campaign in the modern political era.”

“When the smoke clears on Tuesday—and it will clear—what will emerge is Wang and his Princeton Election Consortium website and calculations (which have been used, in part, to drive some of the election poll conclusions at The New York Times’ Upshot blog and The Huffington Post’s election site). What will be vindicated is precisely the sort of math approach that Silver once rode to fame and fortune.”

Is Nate Silver Unskewing Polls In Trump’s Direction?

Huffington Post: “The short version is that Nate is changing the results of polls to fit where he thinks the polls truly are, rather than simply entering the poll numbers into his model and crunching them.”

“Silver calls this unskewing a ‘trend line adjustment.’ He compares a poll to previous polls conducted by the same polling firm, makes a series of assumptions, runs a regression analysis, and gets a new poll number. That’s the number he sticks in his model ― not the original number.”

“He may end up being right, but he’s just guessing. A ‘trend line adjustment’ is merely political punditry dressed up as sophisticated mathematical modeling.”