Summary: In a significant advance for medical robotics, researchers have trained a surgical robot by having it watch videos of human surgeons at work. Using imitation learning, the robot successfully performed several fundamental surgical procedures with skill comparable to human doctors, potentially opening new possibilities for autonomous surgery.
Journal: Conference on Robot Learning, Munich, November 2024 | Reading time: 4 minutes
Learning by Watching
What if surgical robots could learn like medical students – by observing experienced surgeons at work? Johns Hopkins University researchers have made this possible, training a robotic system to perform surgical tasks simply by watching videos of human doctors.
“It’s really magical to have this model and all we do is feed it camera input and it can predict the robotic movements needed for surgery,” says senior author Axel Krieger, describing the breakthrough in robot training.
Teaching Robots to Speak Surgery
The research team used the da Vinci Surgical System, a widely used robotic platform, combining imitation learning with technology similar to that powering ChatGPT. But instead of processing text, this system works with kinematics – the mathematical language of robotic motion.
The training data came from hundreds of surgical videos recorded by wrist cameras on da Vinci robots during actual procedures. With nearly 7,000 such robots in use worldwide and over 50,000 trained surgeons, researchers had access to extensive real-world examples.
Adapting to Imperfection
While the da Vinci system is widely used, it has known precision limitations. The researchers overcame this challenge by teaching the robot to perform relative movements rather than absolute actions, improving accuracy.
“All we need is image input and then this AI system finds the right action,” explains lead author Ji Woong “Brian” Kim. “We find that even with a few hundred demos the model is able to learn the procedure and generalize new environments it hasn’t encountered.”
Unexpected Abilities
The robot demonstrated skills beyond its basic training. “Here the model is so good learning things we haven’t taught it,” Krieger notes. “Like if it drops the needle, it will automatically pick it up and continue. This isn’t something I taught it do.”
This approach marks a significant improvement over traditional methods, where programming even simple surgical tasks required extensive hand-coding. As Krieger explains, “What is new here is we only have to collect imitation learning of different procedures, and we can train a robot to learn it in a couple days. It allows us to accelerate to the goal of autonomy while reducing medical errors and achieving more accurate surgery.”
Glossary
- Imitation learning: A training method where AI learns by observing examples
- Kinematics: The mathematical study of motion and movement
- da Vinci Surgical System: A widely used robotic surgery platform
- Autonomous surgery: Surgery performed independently by robots
Quiz
- How did researchers train the surgical robot?
Answer: By having it watch videos of human surgeons - How many da Vinci robots are currently in use worldwide?
Answer: Nearly 7,000 - How long does it take to train the robot using the new method?
Answer: A couple of days - What unexpected ability did the robot demonstrate?
Answer: Picking up dropped needles and continuing the procedure
Enjoy this story? Get our newsletter! https://scienceblog.substack.com/
