The Art and Science of Forecasting an Election

Since Nate Silver began his prediction site FiveThirtyEight over six years ago, there have been multiple political/news websites that have included an election forecasting model, including this very site.  For example, in 2012, we correctly predicted the winner in 49 of the 50 states with an average margin of error of only 1.8%.  

This year is the first that we've had a running Senate forecast, which began in mid September.  As I have explained before, our model takes into account polling, state fundamentals, and other news events in order to calculate a prediction for a candidates chances in winning.  

Other websites use similar formulas, but by simple modifications and tweaks, the different models can give drastically different results.  For example, the Washington Post Election Lab has GOP chances of winning the Senate at 94%, while we currently have the GOP's odds at about 55%.  Why the huge disparity?

This is due to how much weight forecasters give to either polls or state fundamentals. Additionally, a site like WaPo likely assumes that a candidate with a two point deficit in the polls will have an approximate 20% chance of winning, while our model will assume an approximate 40% chance for the same polling deficit.  

Therefore, there are fewer scenarios in which seats can be in play.  While most forecasters have sites like Colorado as a true tossup, the Washington Post rates it as "likely Republican".  

Below, we have a chart showing how the most competitive Senate races are forecasted by Battleground270 as well as 538, the NYT, WaPo, and Huffington Post.  

As you can see, Battleground270 correlates most closely to the results from 538, with an average difference of only 4.5 points.  The largest differences between BG270's and 538's predictions are in Michigan (+7) and North Carolina (-14).  There's not really a great explanation to those larger gaps, but they will obviously change (and possibly converge) as the election nears.

What we do know is that everyone cannot be correct.  On election day, these four sites as well as Battleground270 will have published a final Senate forecast with predicted vote shares as well as likelihood of each party to take control of the Senate.  On November 5th, we'll know who had the better formula.

Senate Update - 8 October

Last night and much of today, the political world was abuzz with news of a new poll out of South Dakota.  The poll, by SurveyUSA, found the Republican, Mike Rounds, stuck in a tighter-than-expected three-way race.  He was leading the Independent, Larry Pressler, by only three points, while the Democrat, Rick Weiland, was four more points behind.  

The reason this is such a huge development is that before this poll, it was thought that South Dakota was a safe pickup opportunity for the GOP.  Without it, they would need to win at least six seats elsewhere, no small feat.  

However, we must caution that this is just one poll.  Despite Nate Silver's Fivethirtyeight ranking Survey USA as one of the 'best' pollsters in the country, we cannot put too much weight on the result until more polls are released showing similar results.  If and when that happens, the model will begin to reflect those changes (we found that if only the SurveyUSA poll was applied to the model, Democrats would have a 19% chance in the state and a 50% chance overall).  Until now, we have the GOP with a 54.6% chance of winning the Senate.

Besides the South Dakota poll, there were a number of other polls released today, mostly from Fox News.  The polls mostly look bad for Democrats.  It had the GOP candidates in Kansas, Colorado, Arkansas, Kentucky, and Alaska up by 5, 6, 7, 4, and 4, respectively. 

However, as Fivethirtyeight noted, Fox News surveys have been strongly favoring the GOP candidates.  In fact, the lean Nate Silver's site has calculated for Fox News' polls is approximately 3.5% towards the Republicans.    

At Battleground270, we have yet to begun adding in corrections to polls based on the pollster's lean, and it would be unfair to begin adding that in now to just these polls.  However, to account for the drastic lean these polls have towards the GOP, we are just rounding down the averages to the nearest 0.5%.

For example, if the average polling lead (based on RealClearPolitics list of polls) for the GOP candidate is 4.3%, and one of the polls is from Fox News, Battleground270's model will automatically round that down to a 4.0% lead for the GOP candidate.  

Finally, SurveyUSA found Georgia Democratic candidate Michelle Nunn up by one point, the first poll to find her leading Perdue in the past month.  Multiple polls have shown the race within the margin of error, and this could be the first one to show the effects from Perdue's outsourcing comments fallout.  

Senate Update - 7 October

The Republicans saw an incremental gain in their chances to control the Senate today.  They now stand a 56.1% chance of winning, and increase of 0.5% since yesterday.  Baby steps, but it did manage to reverse the over week-long decline in their standing.  

Only two notable polls were made available today.  The first, from Georgia, showed David Perdue still maintaining his small but stubborn lead.  The poll had him up 48-45 against Nunn.  Because this is now the fourth poll in a row to show Perdue up by about that same margin, the model is beginning to put more weight into the idea that his lead is, indeed, what the polls are telling us.  That gives him a 70% chance of winning on November 4th, up from 68% yesterday.

Similarly, yet another poll (SurveyUSA) found Kansas Republican Senator Pat Roberts down by five points to his independent challenger, Greg Orman.  Like in Georgia, this result marks a very stable set of polls showing Orman with a lead of 5 or 6 points.  However, unlike in Georgia, Orman's chance of winning remains the same, at 68%.  

Senate Update - 6 October

Unfortunately, there were relatively few Senate polls out today.  That was expected given the plethora of polls released yesterday.  Though the overall forecast remained relatively the same (the GOP has a 55.6% chance of winning control of the Senate), both parties got good news today, for different reasons.  

In Iowa, the GOP got a major boost with two new polls showing Ernst with a lead over Braley.  The first was only "technically" a lead, with Ernst up 0.3% over Braley.  However, the second (which could probably be considered an outlier, but we still include it) had Ernst up by nine points.  The poll finding the tie is likely to be closer to the actual result of the race, and indeed, we currently have Ernst leading by about 2.5%, not 9%.  Still, Democrats should begin to worry that Iowa is slipping away from them, as they now only have a 37% chance of winning.

To serve as good news for the Democrats, SurveyUSA published a poll of Kentucky showing Democrat Alison Lundergan Grimes leading Senate Minority Leader Mitch McConnell by two points.  The lead is obviously within the margin of error, but it seems the race may be closer than we previously thought.  Additionally, despite the fact that we don't include campaign-sponsored polls in the forecast, a poll for the Grimes campaign found her up by the same margin last week.  Still, despite the recent poll, McConnell still has a 77% chance of winning, down from 87% yesterday.

Senate Update - 5 October

There wasn't an update yesterday because no new polls were released.  And what a difference a day can make.  Earlier this morning, no fewer than 40 polls were made public.  The majority of these polls were of Senate races in states that were already safe for one party or the other.  But we also got plenty of new information regarding the "swingier" of the states.  

We'll start with the most competitive states and work our way down to the more safe ones.  Colorado is officially the closest Senate race in the country.  Our model gives each side exactly a 50% chance of winning.  The CBS/NYT/YouGov poll today showed Udall up by three, but that was the first poll so show him up in the past couple weeks, so for now, this race is a true tossup.

In Iowa, Democratic Rep. Bruce Braley got some heartening news.  While the NBC/Marist poll showed his challenger, Joni Ernst, leading by two points, the CBS/NYT/YouGov poll found Braley up by one point.  Both of these are within the margin of error, but the model is still putting a lot of weight on the recent Des Moines Register poll last week which found Ernst up by six points.  At the moment, the race is still leaning towards Ernst, with her chances of winning being 58%.

North Carolina also had two fresh polls today, one showing Hagan up by four and the other having her up by only 1.  Still, Hagan has led in every poll for the last month and a half.  Plus, the results from the CBS/NYT/YouGov poll (Hagan +1) was an improvement over the last poll that agency took of the state (Tillis +1).  Currently, Hagan has a 67% chance of winning re-election.

Next up, we have two states that are equally competitive, but for different parties.  In Kansas, Independent Greg Orman had a great poll and a not-so-great poll released today.  The first, from NBC/Marist, showed Orman up by 10 against Roberts.  However, the other poll had the race tied.  It is odd to have two polls released simultaneously show such drastically different results, but in this case, we simply have to average the two (and use prior information showing the race was leaning in Orman's favor).  Orman has a 68% chance of winning in Kansas.

In Georgia, the Republican, David Perdue, also has a 68% chance of victory.  Two new polls have each shown him with a four point lead against his Democratic challenger, Michelle Nunn.  However, as we noted a couple days ago, these polls are not going to show any effects from Perdue's infamous "outsourcing" comments.  We'll still have to wait a week or so to get those results.  

Still, the closeness of the race in Georgia (relative to some other contests) means the Democrats may have a second path to keeping control of the Senate, one that does not include winning in Alaska, Arkansas, or Louisiana.  This new path would mean Democrats could lose one of either Kansas, Colorado, or Iowa yet still keep the Senate with a pickup in Georgia.

The three races that will be most difficult for Democrats to win are in Arkansas, Alaska, and Louisiana.  In Arkansas, the NYT poll put Republican Rep. Tom Cotton ahead of Senator Mark Pryor by four points.  This result is roughly in line with where the race has been for the past few months.  We will see if former President (and former Arkansas Governor) Bill Clinton's trip to the state on behalf of Pryor moves the needle at all.  Until then, however, Pryor only has a 26% chance of keeping his seat.

The GOP in both Alaska and Louisiana also got heartening news today with polls showing the GOP challenger in each state up by six points.  It would seem as if these are two states Democrats should write off soon in order to concentrate resources in other key battleground states (as should the GOP). In both states the GOP candidate now has a 79% chance of winning.

Finally, the CBS/NYT/YouGov poll found New Hampshire Senator Jeanne Shaheen running ahead of Scott Brown by seven points.  For a short bit, it seemed as though the race was getting very close.  However, in the past week there have been multiple polls showing Shaheen with a solid mid-to-high single digit lead over Brown.  Shaheen now has a 81% chance of winning re-election.  

Overall, the Senate picture did not change much at all.  In fact, the GOP's chances of capturing the Senate decreased by only 0.2%.  They now stand as a 57.6% favorite; not exactly a safe bet, but also a surer thing than the Democrats, at this point.  

Senate Update - 3 October

This cycle, there have been relatively few "game changers" in the individual races for the Senate.  Sure, Kansas has definitely thrown the political world (and Republicans) a curveball with the Democrat dropping out of the race and the Independent candidate surging, but there have been no disqualifying gaffes that could doom a candidate.

Until possibly now.  Politico obtained a transcript from a 2005 deposition in which the Republican candidate David Perdue was essentially bragging about how most of his career had been spent outsourcing.  To understand how damaging this could be, we can look at a recent polls which show American voters greatly dislike outsourcing and the effects they perceive it has on the American economy.

The Democratic candidate Michelle Nunn will no doubt attempt to use this line of attack against her opponent, and it could be quite effective.  Coincidentally, a new poll of Georgia was also released today.  Perdue is still leading Nunn by four points.  However, this poll was taken before the news broke about Perdue's comments.  The full effect of his comments likely won't show up in polls for at least another week.

We also got more information today on the state of the races in Kansas and Michigan.  In the latter, it seems as the though the Democrat has cemented his lead with a new poll showing him having a nine point lead against his Republican challenger.  This came a day after a poll had shown Peters up by thirteen.  Peters now has a 98% chance of winning in November.

Finally, in Kansas, Gravis Marketing released a new poll showing Independent candidate Greg Orman holding a solid seven point lead against his the Republican incumbent. However, despite his seemingly stable lead in the race, the model still only gives Orman a 68% chance of victory.  This is mostly due to the fact that Kansas is a deep red state and there are still quite a few undecided voters.

Overall, despite a lot of news and polls out today, the race only shifted about a point in the Democrats' direction.  Republicans still have a 57.8% chance of winning in November.  

Senate Update - 2 October

Not a whole lot to update today.  Only a couple new polls out, both from states that are already leaning Democratic.  Thus, the model didn't change much since yesterday. Republicans still have a 59.8% chance of taking control of the Senate in 2015.  

The first poll was from New England College showing Democratic Senator Jeanne Shaheen tied with her opponent, Scott Brown.  However, only two weeks ago, the same polling agency found Shaheen ahead by eleven.  As others have pointed out, there certainly wasn't an eleven point swing in just two weeks, especially when there has been no significant news or gaffes from either side.  More likely, the race is somewhere in between, which is where we had it anyway.  Still Shaheen's chances dropped to 75%.

The other poll today was also likely an outlier.  Mitchell Research gave Michigan Rep. Gary Peters a thirteen point lead against his rival, Terri Lynn Rand.  Other surveys have shown the race a bit tighter, but it is clear that Peters certainly has a strong advantage in the state. His chances of winning are now at 97%.