Navigating the treat-filled holiday season is challenging, but the new year brings opportunities to make improvements to your daily routine — hence the shiny, new gym membership. Unfortunately, resolutions are often short lived. While numerous studies have tried to identify the silver bullet to get people to make long-lasting change in their exercise routines, most interventions have had little success.
A group of researchers from multiple institutions, including five from Carnegie Mellon University, have taken a different approach to this problem. Through the University of Pennsylvania’s Behavior Change for Good Initiative, they decided to forgo a single, stand-alone study and opted for one massive project that includes thousands of participants.
This megastudy was set at one field site, 24 Hour Fitness, and recruited 61,293 participants from the organization’s membership. The participants were randomly put into different treatments, or arms, of the study. The value of the megastudy is that it is easier to compare results from one treatment to the next.
“Science has done this work piecemeal in the past, and you can’t compare across studies because so many things are different,” said Gretchen Chapman, department head and professor of social and decision sciences at CMU. Chapman contributed to this study. “For a megastudy, every participant is recruited in the same way so you can compare apples to apples to check for behavioral change.”
Participants were placed into one of 54 treatment arms for the 28-day study. One arm in the megastudy was the control where members were thanked for their participation but did not receive any tips or and simply earned a small lump sum payment for participation. All participants in the other arms received science-based tools to improve their exercise performance, but the details differed from arm to arm, such as reminders and/or incentives (ranging from $0.11 to $1.79 toward an Amazon gift card) for visiting the gym or performing other behaviors (e.g., responding to text messages or changing their workout schedules).
“A key goal of this project was to demonstrate what could be accomplished by bringing dozens of academics together from different institutions and disciplines to generate innovative ideas for testing in a tournament,” said Katy Milkman, a professor at the University of Pennsylvania, co-director of the Behavior Change for Good Initiative, and lead author on the study. “I’m so glad we were able to draw on talent from a diverse set of institutions to design the interventions tested in this megastudy.”
Twenty-four of the 53 treatment arms tested created behavior change that was significant.
The treatment arm that showed the greatest success was the one that offered participants roughly a dime if they returned to the gym for a planned workout following a missed visit they had scheduled at the start of the study. This dime was in addition to the standard incentive of roughly a quarter paid for each workout. Future research could focus on this arm of the study to find out what exactly was the most effective part in getting people to return to the gym.
“Megastudies accelerate the pace of scientific research and increase comparability, making it easier for policy makers to understand what ideas should be deployed at scale to solve pressing real-world problems,” said Milkman. “We hope the same approach can be used to generate key insights about other pressing problems too, like how to motivate voter turnout, reduce energy use and so on.”
Because repeated behavior change is difficult, Chapman notes that the brevity of the intervention, only 28 days, may not have been adequate to solidify the new habit for the long-term.
Despite their limitations, megastudies have methodological advantages. This study takes place in a real-world setting with people attending an actual gym and not going into a research lab. In addition, the study has a large sample size that increases confidence in the results.
“Exercise is hard to change and this change is hard to endure,” said Chapman. “The results from this study help us frame the discussion between physical activity and chronic health in new an interesting ways.”
Chapman, Samantha Horn, George Loewenstein, Silvia Saccardo and Joachim Talloen at CMU were joined by colleagues from the University of Pennsylvania, the University of Chicago, Princeton University, Harvard University, Northwestern University, the California Institute of Technology, Arizona State University, University of California, Los Angeles, Stanford University, New York University, Rutgers University, University of Michigan, San Francisco State University and Dartmouth College on the study entitled, “Megastudies Improve the Impact of Applied Behavioral Science,” published in the Dec.16 issue of the journal Nature.