If you’ve scrolled through social media lately and felt like a sudden, massive wave of opinion just crashed over a controversial topic, you might’ve been caught in an AI swarm. You wouldn’t have known it, though. Unlike the clunky botnets of 2016, these are invisible armies of artificial personas with distinct personalities, persistent memories, and a terrifying ability to adapt to your emotions in real time.
A new policy forum published in Science warns that we’re no longer dealing with simple repetitive messaging. These malicious AI swarms use slang, mimic organic posting rhythms, and tailor their arguments to exploit your specific cultural background. They’ll argue both sides of an issue simultaneously, all while taking orders from a single central command. It’s a level of deep infiltration that makes previous information warfare look like child’s play.
This isn’t just a theory. Researchers including Daniel Schroeder at SINTEF Digital and Nick Bostrom have analyzed pro-Kremlin networks like “Pravda” and found something startling. These operations aren’t even built for humans to read; they’re built for other machines. They flood the web with hundreds of identical articles across obscure domains to ensure that when the next generation of AI is trained, these fabricated narratives become part of the machine’s “truth.”
From Human Labor to Machine Scale
In the past, disinformation required massive staff. The 2016 Russian Internet Research Agency operation needed hundreds of humans just to reach a tiny fraction of users. AI’s deleted that constraint. Today, a single adversary can deploy thousands of autonomous agents that coordinate with minimal oversight, running thousands of micro-experiments per hour to see which lies stick the best.
The researchers define these swarms as sets of AI agents that maintain persistent identities and coordinate toward shared goals while varying their tone. This fusion of scale and adaptation creates a “synthetic consensus.” When you see a dozen different voices—some left-leaning, some right-leaning—all agreeing on a specific narrative, your brain triggers a “wisdom of crowds” response. You’ll assume it’s a grassroots movement, when it’s actually a coordinated illusion.
“A malicious AI swarm is a set of AI-controlled agents that (i) maintains persistent identities and memory; (ii) coordinates toward shared objectives while varying tone and content; (iii) adapts in real time to engagement, platform cues, and human responses; (iv) operates with minimal human oversight; and (v) can deploy across platforms.” – Science Policy Forum, Researchers
The danger goes beyond mere persuasion. Swarms can be used for “synthetic harassment,” where thousands of AI personas launch tailored abuse at a journalist or politician until they retreat from public life. Because the swarm adapts to the target’s responses, it’s nearly impossible for moderators to distinguish the attack from genuine public anger until the damage is already done.
The Fight for the Epistemic Substrate
Perhaps the most chilling warning in the Science report is the concept of “LLM grooming.” By flooding the internet with fake chatter, swarms contaminate the training data for future Large Language Models. When these models are retrained, the lies “calcify” into the model’s weights. We’re essentially poisoning the well of knowledge that the entire future of AI will drink from.
So, how do we fight back? The researchers argue that we can’t simply build “good” swarms to counter the “bad” ones. The attention economy rewards outrage and fear—the primary tools of the manipulator—while pro-social actors are bound by ethical rules against deception. This creates a permanent asymmetry that favors the liars.
Instead, the report suggests structural changes. We need an “AI Influence Observatory”—a global network of academics and NGOs to track these swarms in real time. We also need “swarm detectors” that look for patterns of coordination rather than trying to judge what’s “true” or “false.” By focusing on whether a behavior’s authentic, we can sidestep the messy politics of censorship while still protecting the public square.
The window to act’s closing. With major elections on the horizon, the researchers urge platforms to implement “stress tests” for persuasion and for governments to move from voluntary promises to financial consequences. If we don’t learn to see the invisible armies today, the consensus of tomorrow might be nothing more than a ghost in the machine.
Science: 10.1126/science.adz1697
ScienceBlog.com has no paywalls, no sponsored content, and no agenda beyond getting the science right. Every story here is written to inform, not to impress an advertiser or push a point of view.
Good science journalism takes time — reading the papers, checking the claims, finding researchers who can put findings in context. We do that work because we think it matters.
If you find this site useful, consider supporting it with a donation. Even a few dollars a month helps keep the coverage independent and free for everyone.
We are unleashing powerful technologies that overwhelm most people’s capacity to cope, let alone understand. Musk’s motto: “move fast and break things”. One of the things will be civilization.
I think it was Facebook and Zuckerberg that used “move fast and break things. But its seems to be a mantra the rest of the tech world has picked up.