On Aug. 4 in Las Vegas, seven computers will compete in the first all-machine cyber defense tournament, the result of a multiyear effort by the Defense Advanced Research Projects Agency to bring autonomy to the problem of making computers secure.
Mike Walker, program manager for DARPA’s Cyber Grand Challenge, discussed what the agency and the seven finalist teams set out to do and what the world will see during the final hours of a competition that eventually could deliver cybersecurity at network speeds by computers that look at all the bits, all the time, without human help.
“Today the comprehension [of] and reaction to unknown flaws in software is entirely manual,” Walker said during a recent media teleconference.
The best statistics indicate that when intruders have access to an unknown flaw and are using it to break into computers, on average they can use the flaw for 312 days before it’s discovered, and software vendors have about 24 median days to patch, he said. Both of those times are coming down, he added, but the amount of time it takes to discover, comprehend and react to an unknown flaw is about a year.
“We want to build autonomous systems that can arrive at their own insights about unknown flaws, do their own analysis, make their own risk-equity decisions about when to field a patch and how to manage that patching process autonomously,” Walker said, “and bring that entire … timeline down from a year to minutes or seconds.”
Bug Hunting
DARPA launched the challenge in 2013 and has so far spent $55 million on the effort. In October that year, it opened up a track for teams who wanted to submit a proposal and receive initial funding to compete, and an open track for anyone in the world who wanted to enter their own intellectual property without DARPA funding.
Walker said development and work on the challenge began in June 2014, and the qualifier stage for those who entered the competition ran until June 2015.
“At the end of the qualifier stage, we held a contest that was executed live on the internet for 24 hours,” he said, “where we gave 131 pieces of unexamined software to all competitors simultaneously and asked the machines to bug hunt those pieces of software in 24 hours and submit bug reports directly to DARPA.”
The results of the contest showed that, of the 590 known flaws in the publicly available software corpus, the machines mitigated 100 percent of them, Walker said, noting that no individual competitor achieved that result or even came close. Only by taking the best solution from each competitor in the field could it be achieved, he said, and all the teams learned from one another. Individually though, the machines successfully bug-hunted 73 percent of the challenges, he added, finding and proving at least one security-critical flaw in the software.
“We don’t require systems to write exploits, but they do have to prove vulnerability and gain very specific control of software and indicate that to a DARPA referee,” Walker said, adding that the goal is to create defenses that can prevent vulnerability from happening.
In Las Vegas, Walker said, he’ll be most excited to see the mix the machines decide to use of generic binary armoring, which doesn’t target specific bugs and is all over the program, slowing it down, and point patching, which very quickly fixes specific bugs but requires a lot of expertise.
“I will say that in all the results all of our machines released in 2015 as the result of our qualifiers, we did see point patching — very effective point patching written by an expert system,” Walker said, “and that was actually one of the reverse engineering tests that was most convincing” when he and his team were thinking about executing the second year of the Cyber Grand Challenge.
Stand and Compete
When the seven finalist teams meet in Las Vegas next month, the field of battle will be the Paris Hotel and Conference Center. The teams will compete in a cyber capture-the-flag event for nearly $4 million in prizes.
The machines themselves are DARPA-constructed high-performance computers with about 1,000 Intel Xeon cores and 16 terabytes of RAM. They’ll operate on an open-source operating system extension called DECREE — for DARPA Experimental Cybersecurity Research Evaluation Environment — built only for computer security research and experimentation.
What each team will do with its autonomous system, Walker said, “is program it with what we call a cyber reasoning system that they will eventually be disconnected from on the day before the grand challenge. And when they are disconnected from it, that cyber reasoning system will stand and compete entirely on its own, and they will be spectators to its victory or its defeat.”
The results will be open-source to the world as they happen, and every single piece of software the machines have written and will write will go on a public server in perpetuity, DARPA officials said.
Show Time
Walker said one thing that’s important to understand about the final event is that the compute time during which the event will happen and the audience time are different timescales.
On Aug. 4, the machines will compute the event for 10 hours without an audience, then at 5 p.m., Walker and his team will do a three-hour recap for the audience. But the live event and the rest of the computing will finish at the same time. “So the beginning will be a recap, but the end will be live, and that’s because a three-hour timescale for a live event was much more manageable,” he explained.
When the live event begins at 5 p.m., the audience in the 3,000-seat auditorium will watch a capture-the-flag competition among seven autonomous machines occur in rounds of about five minutes each, Walker said.
“We have a video we call an arena view that shows who’s proving vulnerability against who, whose software is broken, whose software is well defended, and it’s going to unfold as a graphical 3-D visualization, all driven by data occurring inside the game on screen,” he said.
Two announcers — one astrophysicist and one hacker — will talk the audience through the action.
“Then we have a second view called trace viewer that you can think of as a software microscope that is actually going to let people see what the structure of a good patch looks like, what the structure of a failed patch looks like, and what the structural feel of the software armor that these systems are constructing looks like,” he said. “You can see multiple samples from a single system and start to identify the visual field.”
The awards ceremony will take place the next day at 10 a.m.
A Seat at the Table
The Cyber Grand Challenge is co-located this year with the world series of hacking: Def Con, one of the world’s largest hacker conventions.
The day after DARPA’s event, Walker said, the autonomous system that wins the Cyber Grand Challenge has been challenged to play in a Def Con community capture-the-flag contest, a competition with at least two decades of history.
“You win a qualifying competition, where [that] has to be global entry open competition, and the winners of other competitions feed into Def Con capture the flag and earn a seat there,” Walker explained. “Teams fly in from around the world to play. It’s an annual contest, and this will be the first time that a machine will play at a table rather than a team of experts.
“That contest is actually post-DARPA’s involvement with the technology,” he added, “and could actually be considered the first step in the open technology revolution.”
