Social-media commentary regarding television shows and commercials contains valuable data that can be used by networks and advertising agencies to improve programming or reach desired audiences. Analyzing that data, however, can be a costly, complex and unreliable process.
In 2008, MIT’s Deb Roy and his former doctoral student Michael Fleischman PhD ’08 began developing algorithm-based technology that could track and analyze social-media comments about television content. They began commercializing this novel technology, which they dubbed “Social TV analytics,” through their startup, Bluefin Labs.
Over the next five years, as social media rose in popularity, the importance of Bluefin’s technology as a data-analysis platform became clear: The startup quickly gained big-name clients — such as CBS, Fox, Procter & Gamble, Pepsi and various advertising agencies — and eventually caught the eye of Twitter.
In February, Twitter acquired Bluefin, and its technology, for a reported $100 million, which would make Bluefin Twitter’s largest acquisition to date. With Bluefin’s technology, the social-media giant aims to introduce new capabilities to benefit both advertisers and users, Roy says.
“It was a very positive outcome,” says Roy, an associate professor of media arts and sciences, who at different times served as Bluefin’s CEO, chairman and chief scientist. “Twitter is obviously a very important company, and we now have an opportunity to have a significant impact on its future operations.”
While Bluefin no longer functions as an independent entity, its technology lives on: Most former employees, including Roy (now Twitter’s chief media scientist and an MIT faculty member) and Fleischman (Bluefin’s former president), are further developing the technology for Twitter — which plans to keep Bluefin’s operations in Boston and start growing its own presence in the area.
“For MIT, it’s exciting to see a piece of basic research grow out of the lab and have real impact,” Roy says.
Language grounding
Bluefin’s core technology has its roots in 15 years of research Roy conducted to discover how language connects to the world — something Roy calls “language grounding.”
The research began with Roy’s PhD work in the late 1990s at MIT on developing robots that could learn language, modeled after how children learn language. In 2005, Roy initiated and led the Human Speechome Project (HSP) in the MIT Media Lab. The project aimed to understand how a child learns language by amassing and analyzing a digital record of the first years of a child’s home life.
Roy’s team developed an embedded home-video recording system to capture Roy’s son’s early life, from birth to age 3, accumulating 240,000 hours of audio and video recordings. Roy and the HSP team then developed machine-learning algorithms to uncover relationships between spoken language and context as captured through video.
Years later, Fleischman, as Roy’s student, steered the research toward TV analytics for his dissertation. After realizing its commercial potential, Roy and Fleischman won a $100,000 Small Business Innovation Research grant from the National Science Foundation — which triggered Bluefin’s creation.
Today, their technology uniquely blends machine and human analysis to capture millions of social-media comments posted daily about TV programs and commercials on Twitter. It analyzes what content drove the most conversation, the sentiment behind and topics of those responses, and the characteristics of the commenters. The co-founders call the resulting organized view of these TV-oriented Tweets the “TV genome.”
As part of Twitter, the Bluefin technology will help advertisers better connect TV with Twitter and enable new TV-related user experiences, Roy says. “As Twitter’s massive collection of servers move hundreds of millions of tweets around each day, our analytics can understand what’s happening in the TV world outside Twitter and ground Tweets in TV context,” Roy says. “It’s about grounding language in the real world at scale.”
‘At the intersection of big and fast’
At the time of its acquisition, Bluefin — then located in Kendall Square — had gained roughly $20 million in funding, hired 50 employees and attracted dozens of clients, including most U.S. major TV broadcast and cable networks and many of the world’s largest advertising agencies.
In 2012, Bluefin was named one of Technology Review’s 50 disruptive technology companies and helped ABC News analyze social-media comments during the presidential campaign.
Roy attributes Bluefin’s rapid success, in part, to the recent intersection of social media with mass media. As an example, he refers to the news coverage of the recent Boston Marathon bombings, where TV reporters were reading tweets on air.
“The biggest medium humans have ever created is television; the fastest growing medium we’ve ever seen is social media,” Roy says. “So if you can understand what’s happening at the intersection of big and fast, there’s huge potential value and interest. It’s fair to say Bluefin developed a technology that was unique and ahead of everyone else in understanding this intersection.”
Big ideas, big risks
But in order to grow a company around such a novel technology, the Bluefin team had to take risks. When the company gained its first partner, the NFL, it sank all its money into pushing its technology to work in a live setting. This was Bluefin’s biggest and earliest risk, Roy says, and even more perilous during 2008’s financial crisis.
“No one knew what it meant to link social media with television,” Roy says. “It was like placing a bet to see if we could really find a market for the technology. There would be no second chance.”
That risk paid off, as did others, Roy says, because the Bluefin team remained confident in its “big idea” — language grounding. “Bluefin was a way to bring this intelligence to machine processing and put it to work,” he says. “And we knew if we found the right application, it could pay off handsomely.”
After finding success, Roy’s advice to fledgling high-tech entrepreneurs: “Stick to your big idea, take big risks, remain agile in channeling your big idea to whatever opportunities present themselves.”
