Polling

Polls May Underestimate Trump’s Support

A Morning Consult analysis looks at the fact that Donald Trump generally has done better in online polls than in phone surveys.

Los Angeles Times: “The firm conducted an experiment aimed at understanding why that happens and which polls are more accurate — online surveys that have tended to show Trump with support of nearly four-in-10 GOP voters or the telephone surveys that have typically shown him with the backing of one-third or fewer. Their results suggest that the higher figure probably provides the more accurate measure.”

“Some significant number of Trump supporters, especially those with college educations, are ‘less likely to say that they support him when they’re talking to a live human’ than when they are in the ‘anonymous environment’ of an online survey.”

Polls 60 Days Out Aren’t Very Helpful

Alfred Tuchfarber: “Bottom line, the national polls are very poor predictors of primary/caucus results. While the state polls are better, they are still faced with the very difficult task of basing their predictions on what is at best an artful guess as to which individuals in the samples will actually vote.”

“The current polls almost surely do not provide a good read on the winners of the early February contests in Iowa and New Hampshire. Rather than relying on these polls to handicap the races, an eclectic multi-factor approach is necessary. That analysis needs to take into account history, candidate strengths and weaknesses, money, ground games, likely turnout, intensity of support, and interest in the election, among other factors.”

Two Trend Lines in Political Polling

Nate Cohn: “One is the rise of online polling. The other is the decline of telephone surveys because of rising costs and possibly declining quality, tied to the fact that fewer Americans are willing to participate in a telephone poll. Many public opinion researchers have thought that Internet polling would eventually overtake phone polling, but they hoped it would not happen before online polling was ready.”

“Ready or not, online polling has arrived. Political analysts and casual poll readers now face a deluge of data from new firms employing new, promising, but not always proven methodologies. Nowhere is the question of the accuracy of the new online polls more evident than in the survey results for Donald Trump. He fares better in online polls than in traditional polls, and it’s not clear which method is capturing the public’s true opinions.”

Can Polls Really Be Trusted Anymore?

New Yorker: “Election pollsters sample only a minuscule portion of the electorate, not uncommonly something on the order of a couple of thousand people out of the more than two hundred million Americans who are eligible to vote. The promise of this work is that the sample is exquisitely representative. But the lower the response rate the harder and more expensive it becomes to realize that promise, which requires both calling many more people and trying to correct for “non-response bias” by giving greater weight to the answers of people from demographic groups that are less likely to respond… A typical response rate is now in the single digits.”

“Meanwhile, polls are wielding greater influence over American elections than ever.”

New Exit Polls Being Tested

“When voters go to the polls in Kentucky and Mississippi on Nov. 3, The Associated Press will be launching experiments aimed at finding more accurate and less expensive ways to survey them. With funding from the Knight Foundation, the AP has hired GfK Custom Research to identify and survey voters online, instead of conducting a traditional exit poll where precinct-based interviewers ask voters to fill out questionnaires,” the AP reports.

Gallup Sits Out the Primaries

“After a bruising 2012 cycle, in which its polls were farther off than most of its competitors, Gallup told Politico it isn’t planning any polls for the presidential primary horse race this cycle. And, even following an internal probe into what went wrong last time around, Gallup won’t commit to tracking the general election next year.”

“It’s a stunning move for an organization that built its reputation on predicting the winners of presidential elections. But it comes at a time of unusual tumult in the polling world.”

Pollsters Warn Against Using Polls

“Pollsters surveyed by Politico have a unanimous warning for the Republican National Committee and the TV networks who are using public-opinion surveys to exclude presidential candidates from debates: Don’t trust polls to detect often-tiny grades of opinion in a giant field.”

“Indeed, the unprecedented reliance on polls to winnow the Republican field is coming at a time when many pollsters feel they’re blinder than ever to trends in public thinking — and that using polls to keep out candidates who are otherwise well qualified could seriously alter the race.  But when CNBC announced on Wednesday its criteria for the Oct. 28 GOP debate, it marked the third time out of three debates that the sponsoring news organization is using the candidates’ standing in public polling as the main determinant for eligibility.”

Rethinking the Exit Poll

National Journal: “Exit pollsters do their work in a few short days, but the information they glean dominates the political conversation for years. That’s because the surveys conducted outside voting precincts are the single-best tool for understanding American politics. They generate a treasure trove of data—about who votes, who they voted for, and why—that tell the story of how each election was won and lost.”

“But now, as more voters cast ballots early or by mail and polling those voters becomes more costly and potentially less reliable, the exit polls are getting more expensive, too—and might even be losing some of their predictive power.”

What’s Caused the Polling Crisis?

Cliff Zukin: “Election polling is in near crisis, and we pollsters know. Two trends are driving the increasing unreliability of election and other polling in the United States: the growth of cellphones and the decline in people willing to answer surveys. Coupled, they have made high-quality research much more expensive to do, so there is less of it. This has opened the door for less scientifically based, less well-tested techniques. To top it off, a perennial election polling problem, how to identify ‘likely voters,’ has become even thornier.”

How Well Do You Know the News?

Pew Research has a 12 question news quiz which compares your answers to those of the general public. Questions about politics proved particularly challenging.

The Fix: “The takeaway from all of this? Assuming — as lots and lots of people who either live in D.C. or follow politics closely do — that the average person is a deeply-informed consumer of political news is not even close to right. For most people, politics is something that almost never intersects with their daily lives and which they spend zero mind space on day in and day out.”

The Polling Roller Coaster Begins

Nate Cohn: “The wave of candidate announcements that began late last month has set off a new phase of volatile polling, when voters will rally behind news-making candidates and move on as soon as the next arrives.”

“Republican voters are just starting to tune in, and they start with few allegiances to this year’s deep field of candidates. As Republican hopefuls announce their bids and attract media attention, they’ll probably get a bounce in the polls. These bumps could easily be enough to give long-shot candidates the lead. But for now — and for a while — it might be wise to tune out the polls altogether.”

Solid Proof That Some Pollsters Were Cheating

Nate Silver: “As the election season wore on, new polls hewed somewhat more closely to the polling averages. But the change was marginal until the final week or two of the campaign, when they started to track it much more closely. By the eve of the election, new polls came within about 1.7 percentage points of the polling average.”

“Perhaps you could construct some rationale, apart from herding, for why the polls behaved this way. Maybe it became easier to predict who was going to vote and that made methodological differences between polling firms matter less… But there are two dead giveaways that herding happened. One is the unusual shape of the curve. Rather than abiding by a linear progression, it suddenly veers toward zero in the final week or so of the campaign.”

“The other giveaway is… By the end of the campaign, new polls diverged from the polling averages by less than they plausibly could if they were taking random samples and not tinkering with them.”

Ranking the Election Forecasts

The Monkey Cage uses Brier scores to evaluate the accuracy of various election forecast models:

  1. Daily Kos (.024)
  2. Election Lab (.027)
  3. FiveThirty Eight (.032)
  4. PredictWise (.032)
  5. Pollster (.034)
  6. The Upshot (.035)
  7. Princeton Election Consortium (.043)

“Brier scores take into account both whether races were called correctly and the underlying confidence of the forecast.  The best outcome is to be 100 percent certain and correct (a Brier score of 0).  The worst outcome is to be 100 percent certain and incorrect (a Brier score of 1).  Lower scores are better.”