New research reveals that smart TVs are continuously monitoring their screens, capturing snapshots as frequently as every 10 milliseconds – even when you’re using them as computer monitors. This surveillance extends far beyond traditional TV watching, affecting how you’re targeted with ads across all your devices.
Published in Proceedings of the 2024 ACM on Internet Measurement Conference | Estimated reading time: 4 minutes
Behind the sleek interface of modern smart TVs lies a sophisticated tracking system called Automatic Content Recognition (ACR), which operates much like the popular Shazam app does for music. According to research conducted by computer scientists at the University of California, Davis, University College London, and Universidad Carlos III de Madrid, this technology is far more pervasive than most consumers realize.
The study, led by UC Davis associate professor Zubair Shafiq and graduate student Yash Vekaria, focused on two major TV manufacturers: Samsung and LG. Their investigation revealed that these smart TVs capture and process screen content at remarkably short intervals, creating “fingerprints” of everything displayed on screen.
“These smart TV ad platforms have this vast profile of every single TV customer, and the ads on every TV get personalized based on that profile,” explains Shafiq in the study.
The research team discovered that the TVs’ tracking capabilities extend across multiple content types, including traditional broadcast television, internet-delivered channels, and even devices connected via HDMI cables. While streaming services like Netflix and Prime were exempt from this surveillance (likely due to separate agreements), the monitoring of HDMI-connected devices means your TV is watching even when you’re using it as a computer monitor.
Perhaps most concerning is how this information feeds into cross-device advertising. The collected data enables advertisers to track viewing habits with remarkable precision. As Vekaria explains: “They do a match against the database to figure out what exact piece of content that user is streaming at this point in time. When they do this over a period of time, they can infer that, say, this person watches NFL from 9 to 12 p.m., but they generally watch news in the afternoon.”
While manufacturers do include information about data collection in their documentation, the descriptions are notably vague. Users must opt in to ACR during initial setup, but the process of opting out later is considerably more complex. The researchers also found similar tracking patterns in TVs sold in the United Kingdom, though with some variations due to different regulatory environments.
The implications extend beyond current smart TV technology. Companies are developing even more advanced monitoring systems, such as Microsoft Recall, which combines ACR with AI to analyze screen content across various devices. As Shafiq notes, “Things are trending in the wrong direction, it seems, from a privacy perspective.”
Glossary
- ACR (Automatic Content Recognition): A technology that automatically identifies content displayed on screen by creating and matching digital fingerprints against a database.
- FAST (Free Ad-Supported TV): Traditional TV-like channels delivered over the internet with advertisements.
- Cross-device tracking: The practice of monitoring and connecting user behavior across multiple devices for targeted advertising purposes.
Test Your Knowledge
How often can smart TVs capture snapshots of screen content?
Smart TVs can capture snapshots as frequently as every 10 milliseconds.
What types of content do smart TVs typically monitor in the United States?
They monitor linear TV, FAST channels, and HDMI-connected devices, but not screencast content and OTT streaming services.
How does ACR technology create content “fingerprints”?
It batches captured snapshots of audio or video over a time interval (such as one minute) and uses an algorithm to generate a unique fingerprint representing that content.
What are the key differences between ACR implementation in the UK versus the US?
While data collection rates are similar, UK models have more restrictions on ACR data collection for FAST content due to different agreements between manufacturers and copyright owners.
Enjoy this story? Subscribe to our newsletter at scienceblog.substack.com.