If you’ve ever taken part in the armchair sport of fantasy football and found yourself at the top of your league’s standings at the end of the season, a new MIT study suggests your performance — however far removed from any actual playing field — was likely based on skill rather than luck.
Those looking for ways to improve their fantasy game will have to look elsewhere: The study doesn’t identify any specific qualities that make one fantasy player more skilled over another. Instead, the researchers found, based on the win/loss records of thousands of fantasy players over multiple seasons, that the game of fantasy football is inherently a contest that rewards skill.
“Some [fantasy] players may know more about statistics, rules of the game, which players are injured, effects of weather, and a host of other factors that make them better at picking players — that’s the skill in fantasy sports,” says Anette “Peko” Hosoi, associate dean of engineering at MIT. “We ask, does that skill have an impact on the outcome of the [fantasy] game? In our analysis, the signal for skill in the data is very clear.”
Other fantasy sports such as baseball, basketball, and hockey also appear to be games of skill — considerably more so than activities based on pure chance, such as coin-flipping. What ultimately do these results mean for the average fantasy player?
“They probably can’t use our study to assemble better sports teams,” says Hosoi, who is also the Neil and Jane Pappalardo Professor of Mechanical Engineering. “But they can use it to talk better smack when they’re at the top of their standings.”
The team’s findings appear this week in the Society for Industrial and Applied Mathematics Review. Hosoi’s co-authors are first author Daniel Getty, a graduate student in MIT’s Department of Aeronautics and Astronautics; graduate student Hao Li; former graduate student Charles Gao; and Masayuki Yano of the University of Toronto.
A fantasy gamble
Hosoi and her colleagues began looking into the roles of skill and chance in fantasy sports several years ago, when they were approached by FanDuel, the second largest company in the daily fantasy sports industry. FanDuel provides online platforms for more than 6 million registered users, who use the site to create and manage fantasy teams — virtual teams made up of real players of professional sports, which fantasy players can pick and draft to their fantasy team. Players can pit their team against other virtual teams, and whether a team wins or loses depends on how the real players perform in actual games in a given day or week.
In recent years, the question has arisen as to whether fantasy sports are a potential form of online gambling. Under a federal law known as the Unlawful Internet Gambling Enforcement Act, or UIGEA, online players of games such as poker are prohibited from transmitting across state lines funds won through gambling activities using the internet. The law exempts fantasy sports, stating that the game is not a form of betting or wagering.
However, the UIGEA was not drafted to alter the legality of internet wagering, which is, for the most part, determined by individual states. As fantasy sports — and fantasy football in particular — have grown more popular, with prominent ads on commercial and cable television, a handful of states have questioned the legality of fantasy sports and the companies that enable them.
Gambling, of course, is defined as any money-exchanging activity that depends mostly on chance. Fantasy sports would not be considered a form of gambling if it were proven to be more of a contest of skill.
“That is the question that FanDuel wanted us to investigate: Have they designed the contest such that skill is rewarded? If so, then these contests should be classified as games of skill, and are not gambling,” Hosoi says. “They gave us all of their data, and asked whether we could determine the relative role of skill and luck in the outcomes.”
Tests of skill and chance
The team analyzed daily fantasy competitions played on FanDuel during the 2013 and 2014 seasons, in baseball, basketball, hockey, and football. In their analysis, the researchers followed guidelines laid out originally by economist and “Freakonomics” author Steven Levitt, along with Thomas Miles and Andrew Rosenfield. In a research paper they wrote in 2012, the economists sought to determine whether a game — in this case, poker — was based more on skill than on chance.
They reasoned that if a game were more skill-based, then a player’s performance should be persistent. It might be good or bad, but it would remain relatively constant over multiple rounds.
To test this in the context of fantasy sports, Hosoi’s team looked at the win/loss record of every fantasy player in FanDuel’s dataset, over one season. For each (anonymized) player, the researchers calculated the fraction of wins the player experienced over the first half of the season versus the second half. They then represented each player’s performance over an entire season as a single dot on a graph whose vertical and horizontal axes represented the win fraction for the first and second halves of the season, respectively.
If a given fantasy sport were based more on skill, then a individual player’s win fraction should be approximately the same — be it 90 percent or 10 percent — for the first and second halves of the season. When every player’s performance is plotted on the same graph, it should roughly resemble a line, indicating a prevalence of skill. On the other hand, if the game were one of chance, every player should have around a 50 percent win fraction, which on the graph would look more like a circular cloud.
For every fantasy sport, the researchers found the graph skewed more linear versus circular, indicating games of skill rather than chance.
The researchers tested a second hypothesis proposed by Levitt: If a game is based on chance, then every player should have the same expected outcome, just as flipping a coin has the same probability for landing heads versus tails. To test this idea, the team split the fantasy player population into two groups: those that played a large number of games, versus those who only participated in a few.
“Even when you correct for biases, like people who quit after losing a lot of games in a row, you find there’s a statistically higher win fraction for people who play a lot versus a little, regardless of the [type of] fantasy sport, which is indicative of skill,” Hosoi says.
The last test, again proposed by Levitt, was to see whether a player’s actions had any impact on the game’s outcome. If the answer is yes, then the game must be one of skill.
“So we looked at how the actual playing population on FanDuel performed, versus a random algorithm,” Hosoi says.
The researchers devised an algorithm that created randomly generated fantasy teams from the same pool of players that were available to the FanDuel users. The algorithm was designed to follow the rules of the game and to be relatively smart in how it generated each team.
“We ran hundreds of thousands of games, and looked at the scores of actual fantasy players, versus scores of computer-generated fantasy players,” Hosoi says. “And you see again that the fantasy players beat the computer-generated ones, indicating that there must be some skill involved.”
Sports on a spectrum
To put their findings in perspective, the researchers plotted the results of each fantasy sport on a spectrum of luck versus skill. Along this spectrum, they also included each fantasy sport’s real counterpart, along with other activities, such as coin flipping, based entirely on chance, and cyclocross racing, which hinges almost entirely on skill.
For the most part, success while playing both fantasy sports and real sports skewed more toward skill, with baseball and basketball, both real and virtual, being more skill-based compared to hockey and football.
Hosoi reasons that skill may play a relatively large role in basketball because the sport encompasses more than 80 games in a season.
“That’s a lot of games, and there are a lot of scoring opportunities in each game,” Hosoi says. “If you get a lucky basket, it doesn’t matter too much. Whereas in hockey, there are so few scoring opportunities that if you get a lucky goal it makes a difference, and luck can play a much larger role.”
Hosoi says the team’s results will ultimately be useful in characterizing fantasy sports, both in and out of the legal system.
“This is one piece of evidence [courts] have to weigh,” Hosoi says. “What I can give them is a quantitative analysis of where [fantasy sports] sit on the skill/luck spectrum. It’s mostly skill, but there’s always a little bit of luck.”
This research was supported, in part, by FanDuel.
