Leading up to the 2016 U.S. presidential election, the majority of polls had Democrat Hillary Clinton edging out Republican Donald Trump. When Trump won, criticism of this quantitative method of predicting elections swiftly took shape. As one Republican strategist noted on election night, “Tonight data died.”
Not so, according to a new study led by Northeastern network scientist David Lazer. The study found that national election polls remain a strong indicator of election outcomes. The researchers’ statistical models, which leaned heavily on late polling data and current economic conditions, correctly predicted up to 90 percent of such direct executive elections. The findings will be published Friday in the journal Science.
“We found that elections were reasonably predictable, but not overwhelmingly so,” said Lazer, Distinguished Professor of Political Science and Computer and Information Science. “The bottom line is that polls do surprisingly well across varied applications across the world, but they’re not magic. I think the 2016 U.S. election is exemplary in that way.”
With their study, Lazer and his team set out to test whether global models of election prediction—that is, models that could be applied to an executive election anywhere in the world, not just one specific country—could be successful. While Lazer noted that election forecasting has been around for decades, his team’s approach to take a global perspective was “demonstrably novel.”
“The bottom line is that polls do surprisingly well across varied applications across the world, but they’re not magic. I think the 2016 U.S. election is exemplary in that way.”
The team developed statistical models to assess all executive elections across the globe since 1945. Specifically, they looked at whether the incumbent party’s candidate—or the chosen successor of the current chief executive—won the subsequent election or if a candidate from an opposition party won.
The researchers developed statistical models using data from nearly 500 elections. They then performed “live tests” to predict a number of upcoming elections, all of which fell between 80 and 90 percent accuracy in predicting the outcome. For example, they correctly predicted 10 out of 11 elections (90.9 percent) in Latin America in 2013 and 2014, and correctly called 29 out of 36 elections (80.5 percent) across the globe.
Other findings:
- Lazer noted that surprisingly economic factors, such as unemployment rate and gross domestic product, weren’t as strongly predictive in elections globally as they expected—despite the fact that they are predictive in U.S. elections alone.
- Polling data, meanwhile, were “powerfully predictive.” In one test that looked at 96 elections, a model using only up-to-date polling data was found to be 86.3 percent accurate—and 87.9 percent accurate in elections where the incumbent was running
- Incumbent candidates were more likely to be reelected; however, that advantage did not carry over to a candidate in the incumbent’s party when the incumbent himself wasn’t running.
Lazer co-authored the paper with Ryan Kennedy, associate professor at the University of Houston, who spent a year as a visiting researcher in the lab of Lazer and Stefan Wojcik, a former postdoctoral researcher who is now a data scientist at the One Earth Future Foundation in Colorado.
All of the countries the researchers examined hold direct executive elections based on popular vote—except for the United States, which uses the Electoral College system. As a result, Lazer said his team based its prediction models on vote share to ensure a true apples-to-apples comparison. This means that in U.S. presidential elections, the models were technically predicting which candidate would win the popular vote. Ultimately, the models correctly predicted a popular vote victory for Clinton, which she indeed garnered, but overestimated her margin of victory of the popular vote by 5 percent.
In addition to the latest polls and economic factors, the team’s statistical models included factors such as whether one candidate was an incumbent or member of the incumbent party, the level of democracy in the country, and whether the country had good relations with the United States. The researchers found that if a country had strong relations with the U.S., it increased the probability the incumbent would win. They wrote that a possible reason for this is some scholars suggest good relations are accompanied by aid flows and political support that could impact an election.
The researchers asserted in their paper that they “have shown that a global election prediction system is viable, and such efforts are likely to become more successful in the future, as the internet continues to expand access to information on elections around the world and polling techniques become more widespread.”
The results, they wrote, underscored that neither public opinion nor structural factors, like economic growth, are ever going to be perfect predictors of election outcomes. Rather, these factors can “provide a generally accurate representation of likely election outcomes and help us overcome the many biases associated with human ‘gut feelings.’ We predict that reports of the death of quantitative electoral forecasts are greatly exaggerated.”