forecasting models

Ranking the Election Forecasts

November 5, 2014 · 1:25 pm ET · 19 Comments

The Monkey Cage uses Brier scores to evaluate the accuracy of various election forecast models:

Daily Kos (.024)
Election Lab (.027)
FiveThirty Eight (.032)
PredictWise (.032)
Pollster (.034)
The Upshot (.035)
Princeton Election Consortium (.043)

“Brier scores take into account both whether races were called correctly and the underlying confidence of the forecast. The best outcome is to be 100 percent certain and correct (a Brier score of 0). The worst outcome is to be 100 percent certain and incorrect (a Brier score of 1). Lower scores are better.”

Live Election Models

November 4, 2014 · 5:48 pm ET · 11 Comments

The Upshot: “To provide a better gauge of results than the raw vote count, The Upshot is adjusting Senate returns, based on where those returns come from and which parts of a state have not yet reported their votes. If we know the vote from a Democratic-leaning city is outstanding, for example, we will adjust the count toward the Democrat.”

FiveThirtyEight: “Throughout the evening on our live blog, we’ll be publishing an updated probability that Republicans or Democrats win the Senate, along with their probabilities of winning the key Senate races. These probabilities will use FiveThirtyEight’s final pre-election forecast as a baseline, which gave Republicans a 76 percent chance of winning the Senate. But they’ll deviate from them as our partners at ABC News project winners in each state.”

Someone Will Be Wrong Tonight

November 4, 2014 · 2:53 pm ET · 52 Comments

Sam Wang: “From 2004 to 2012, only thirteen Senate races have had margins of less than three percentage points in the week before the election. Of these, four were won by the trailing candidate. One more, the Florida 2004 race, was tied in the polls, and was eventually won by the Republican, Mel Martinez, by 2 percentage points. Scoring that one as half correct, the overall rate of wins by a front-runner is 65%, a bit better than chance.”

“In light of that, the probability that all six close Senate races (AK, CO, IA, KS, NH, and NC) will be won by the candidate in the lead is only 7%. A wrong call is almost inevitable.”