Computer scientists in the field of artificial intelligence have made an important advance that blends computer vision, machine learning and automated planning, and created a new system that may improve everything from factory efficiency to airport operation or nursing care.
And it’s based on watching the Oregon State University Beavers play football.
The idea is for a computer to observe a complex operation, learn how to do it, and then optimize those operations or accomplish other related tasks. In this project, the goal is for the computer to watch video of football plays, learn from them, and then design plays and control players in a football simulation or video game.
As it turns out, football is very complex, and computers struggle to see and understand plays a coach or even an average fan would find routine.
The findings of the new study were just published in AI Magazine, a professional journal of the Association for the Advancement of Artificial Intelligence.
“This is one of the first attempts to put several systems together and let a computer see something in the visual world, study it and then learn how to control it,” said Alan Fern, an associate professor of computer science at OSU.
“Football actually makes a pretty good test bed, because it’s much more complicated that you might think both visually and strategically, but also takes place in a structured setting,” he said. “This makes it quite analogous to other potential applications.”
Even everyday tasks that are simple for a human, Fern said, are a lot more complicated than they seem.
Consider driving home in your car, he said. What you actually have to do is walk to the parking lot, check for other traffic as you cross the street, select the correct key to put in the ignition, back it up, consider the anticipated traffic to plan a route home, slow down and move a little out of your lane to avoid the child wobbling down the street on a bicycle, make sure you have enough gas. And so on.
Then consider designing an OSU Beavers passing play, which is very fast-paced, designed to confuse the opponent, and based on complex rules; the ball could be thrown to any of several receivers and it still only works about half the time. For a computer that initially has no concept of pass routes and blocking, that’s difficult.
“Using football, we created learning algorithms that allow the computer to see the plays, analyze them and learn from them,” Fern said. “Ultimately these systems should be able to see what is happening, understand it and maybe even improve upon it.”
The work could have multiple applications. Control and logistics planning is hugely important in industry, and even small improvements in efficiency could save billions of dollars. Computer vision and controls might be useful in hospitals or nursing homes to help monitor patients and see who needs care. Large operations such as an airport offer multiple control challenges, or the military could use such approaches to improve supply chains for troops in the field.
The research has been supported by the National Science Foundation and the Defense Advanced Research Projects Agency of the U.S. Department of Defense. It is a collaboration of OSU and the Institute for the Study of Learning and Expertise in Palo Alto, Calif.
The research is still at a basic stage, the scientists said, but could have commercial applications within a few years. The new study outlines a clear “proof of concept” in action recognition, transferring that recognition into procedural knowledge, and adapting those procedures to new tasks, the scientists said in their conclusion.
“One thing I’d also like to do is return the favor to the football team,” Fern said.
“The study of these football plays is helping us to create intelligent computer systems,” he said. “When this is more fully developed, we should be able to actually apply it to football, maybe help coaches analyze an upcoming opponent, let the computer determine what they are doing and suggest a strategic nugget to the coach.”