Four Pollsters With Same Data Had Different Results

Nate Cohn: “We decided to share our raw data with four well-respected pollsters and asked them to estimate the result of the poll themselves.”

The result? “A net five-point difference between the five measures, including our own, even though all are based on identical data. Remember: There are no sampling differences in this exercise. Everyone is coming up with a number based on the same interviews.”

Polling Averages Can Sometimes Be Misleading Too

Stuart Rothenberg: “On Election Day 2012, the RealClearPolitics polling average found incumbent President Obama “leading” Republican Mitt Romney by seven-tenths of a point – a statistical dead heat. The actual final result, an Obama victory of 3.9 points, was very different.”

“I cite these numbers not to challenge the RCP average or disparage the pollsters who got the race wrong… Still, the 2012 ‘miss’ should remind us that polling isn’t perfect and even ‘averaging,’ a reasonable strategy, can mislead. Averaging may smooth outliers, but what if one poll always seems to be an outlier, like the Los Angeles Times/USC tracking poll?”

Can We Trust the Polls This Year?

Politico: “While volatility at this stage of the campaign isn’t unexpected, the latest polling data point to an stunningly unstable election environment, unlike any in recent decades.”

Said pollster Neil Newhouse: “There’s every reason to believe that there is huge instability in the current electorate. Four years ago, there were probably 6, 7, 8 percent of voters up for grabs. Now you have 17, 18 percent up for grabs.”

Beware the Unskewers

Harry Enten: “It’s easy to find fault with a poll, even from the best pollsters. Anyone passionately arguing that a poll is wrong because its sample has ‘too many [xx]’ or ‘too many of this group are voting for [xx]’ is probably wrong. As you dig into a survey’s crosstabs — looking at college-educated white men, for example, or Hispanics 65 years or older — you’re sacrificing sample size for specificity. The margins of error of subsamples can get huge. Further, most pollsters weight their results by demographics (such as age and race) and not attitudes (like party identification). They do so because historically this has produced the most accurate result. Picking apart individual polls is usually a bad use of time, and the people doing it tend to have a motive.”

The Pollster’s Dilemma

Mark Blumenthal: “Gary Johnson and the Libertarian Party appeared on state ballots in all but two states in 2012, and Stein and the Green Party appeared on all but 12. They will likely gain access to at least as many state ballots this year, if not more. Without offering supporters of those candidates an explicit option to register their preferences, we have no practical way of tracking their support.”

“Yet if we prompt for third party candidates, many voters who are undecided about Clinton and Trump (or about voting generally) tend to grab for these alternative options as a way of indicating their uncertainty. Whatever the reason, both history and our own data yield evidence that mid-summer polls greatly inflate the true preference for third party candidates.”

Neither Is No Longer an Option

Reuters: “Many voters on both sides have been ambivalent in their support for Democratic nominee Hillary Clinton and Republican nominee Donald Trump, complicating the task of the pollsters trying to track the race.”

“That sentiment may help explain an apparent skew that recently emerged in the Reuters/Ipsos poll results. Given the choice, a relatively large group of voters opted for “Neither/Other” candidate compared with other major polls, leading to an underreporting of several percentage points for one or other of the two major contenders at times in the race.”

“As a result, Reuters/Ipsos is amending the wording of the choice and eliminating the word ‘Neither,’ bringing the option in line with other polls.”

There’s No Gold Standard for Polling Anymore

Huffington Post: “When you take a closer look at the way various polls are conducted, there’s no one method that stands out as inarguably better than all the rest. Some people in the polling industry like to claim that polls done by live interviewers using randomly selected phone numbers are more accurate than surveys that use automated telephone technology or surveys conducted over the Internet. But research shows that’s not necessarily true.”

Do Trump Supporters Lie to Pollsters?

Harry Enten: “Trump did worse than the polling forecast in 19 states; he did better in 15 states. That hardly suggests that Trump outperforms his polling. Still, the difference isn’t so great that we can say Trump usually underperforms his polling. It’s a fairly even split, with Trump missing his average poll by just 1 percentage point in the median state. Two of Trump’s worst performances relative to the polls were in Kansas and Iowa — both states held low-turnout caucuses, which Trump won’t have to deal with in the general election. Overall, Trump’s percentage of the vote versus the polls is about what you’d expect of the average politician.”

For members: How Many People Don’t Admit They Support Trump?

The Most Newsworthy Polls are Often Wrong

Norm Ornstein: “In this highly charged election, it’s no surprise that the news media see every poll like an addict sees a new fix. That is especially true of polls that show large and unexpected changes. Those polls get intense coverage and analysis, adding to their presumed validity.”

“The problem is that the polls that make the news are also the ones most likely to be wrong. And to folks like us, who know the polling game and can sort out real trends from normal perturbations, too many of this year’s polls, and their coverage, have been cringeworthy.

National Polls More Important Than State Polls

Harry Enten: “You’re going to hear a lot about the Electoral College this cycle. At various points, one state or another will be declared pivotal. But stay calm, especially with so long to go until Election Day. It’s too early to take any poll too seriously. We’ll have plenty of time to get into the weeds of different Electoral College scenarios in the months to come.”

“For now, if you’re interested in whether Trump or Clinton is likely to be our next president, I’d pay attention to the average of national polls. Let’s wait until we’re closer to the election and we have a lot more state polling before we zoom in closer than 30,000 feet.”

How Many People Don’t Admit They Support Trump?

This piece is only available to Political Wire members.

Join today for exclusive analysis, new features and no advertising.

College students: Send an email from your college account and get 40% off an annual membership. We’ll respond with a coupon code.

Sign in to your account or join today!

Monthly – $5 per month or Annual – $50 per year


General Election Polls are Now Worth Looking At

Jonathan Bernstein: “Political scientists have found that, historically, polls on potential general-election matchups don’t become more reliable in a steady, gradual arc as the months and weeks go by. Instead, after starting off as essentially meaningless, they ratchet up sharply in two steps.”

“We are just nearing the end of the first stage, when primary and caucus voters first learned about the candidates and formed opinions about them that can be reflected in the November matchups. The second phase occurs at roughly 100 days before the election, corresponding to the parties’ conventions. Once the nominees are known and the general-election campaigns begin, polls do a good job of predicting the outcome.”