Ulysses Balis, M.D., clicks a mouse to identify a helicopter in a satellite photo of Baghdad, Iraq. With another click, an algorithm that he and his team designed picks out three more choppers without highlighting any of the buildings, streets, trees or cars.
Balis isn’t playing war games. The director of the Division of Pathology Informatics at the University of Michigan Medical School is demonstrating the extreme flexibility of a software-tool aimed at making the detection of abnormalities in cell and tissue samples faster, more accurate and more consistent.
In a medical setting, instead of helicopters, the technique, known as Spatially-Invariant Vector Quantization (SIVQ), can pinpoint cancer cells and other critical features from digital images made from tissue slides.
But SIVQ isn’t limited to any particular area of medicine. It can readily separate calcifications from malignancies in breast tissue samples, search for and count particular cell types in a bone marrow slide, or quickly identify the cherry red nucleoli of cells associated with Hodgkin’s disease, according to findings just published in the Journal of Pathology Informatics.
“The fact that the algorithm operates effortlessly across domains and lengths scales, while requiring minimal user training, sets it apart from conventional approaches to image analysis,” Balis says.
The technology — developed in conjunction with researchers at Massachusetts General Hospital and Harvard Medical School — differs from conventional pattern recognition software by basing its core search on a series of concentric, pattern-matching rings, rather than the more typical rectangular or square blocks. This approach takes advantage of the rings’ continuous symmetry, allowing for the recognition of features no matter how they’re rotated or whether they’re reversed, like in a mirror.
“That’s good because in pathology, images of cells and tissue do not have a particular orientation,” Balis says. “They can face any direction.” One of the images included with the paper demonstrates this principle; SIVQ consistently identifies the letter A from a field of text, no matter how the letters are rotated.
How it works
In SIVQ, a search starts with the user selecting a small area of pixels, known as a vector, which she wants to try to match elsewhere in the image. The vector can also come from a stored library of images.
The algorithm then compares this circular vector to every part of the image. And at every location, the ring rotates through millions of possibilities in an attempt to find a match in every possible degree of rotation. Smaller rings within the main ring can provide an even more refined search.
The program then creates a heat map, shading the image based on the quality of match at every point.
This technique wouldn’t work with a square or rectangular-shaped search structure because those shapes don’t remain symmetrical as they rotate, Balis explains.
Why hasn’t everyone been using circles all along?
“It’s one of those things that’s only obvious in hindsight,” Balis says.
In testing the algorithm, researchers even used it to find Waldo in an illustration from a Where’s Waldo? children’s book.
“You just have to generate a vector for his face,” explains Jason Hipp, M.D., Ph.D., co-lead author of the paper — just as one would generate a vector to recognize calcifications in breast tissue.
A “game changer”
Hipp believes the technology has the potential to be a “game changer” for the field by opening myriad new possibilities for deeper image analysis.
“It’s going to allow us to think about things differently,” says Hipp, a pathology informatics research fellow and clinical lecturer in the Department of Pathology. “We’re starting to bridge the gap between the qualitative analysis carried out by trained expert pathologists with the quantitative approaches made possible by advances in imaging technology.”
For example, the most common way to look at tissue samples is still a staining technique that dates back to the1800s. Reading these complex slides and rendering a diagnosis is part of the art of pathology.
SIVQ, however, can assist pathologists by pre-screening an image and identifying potentially problematic areas, including subtle features that may not be readily apparent to the eye.
SIVQ’s efficiency in pre-identifying potential problems becomes apparent when one considers that a pathologist may review more than 100 slides in a single day.
“Unlike even the most diligent humans, computers do not suffer from the effects of boredom or fatigue,” Balis says.
Working together
Vectors can also be pooled to create shared libraries — a catalog of reference images upon which the computer can search — Balis explains, which could help pathologists to quickly identify rare anomalies.
“Bringing such tools into the clinical workflow could provide a higher level of expertise that is distributed more widely, and lower the rate at which findings get overlooked,” Balis says.
Following the publication of this first paper presenting the SIVQ algorithm, the team has a number of research projects nearing completion that demonstrate the technology’s potential usefulness in a number of basic science and clinical applications. These efforts involve collaborations with researchers at the National Institutes of Health, Mayo Clinic, Rutgers University, Harvard Medical School and Massachusetts General Hospital.
SIVQ may also help with the analysis of “liquid biopsies,” an experimental technique of scanning blood samples for tiny numbers of cancer cells hiding amid billions of healthy ones. Balis was involved with the development of that technology at Massachusetts General Hospital before he came to U-M and members of that research team are also involved in developing SIVQ and its applications.
Still, pathologists shouldn’t be worried that SIVQ will put them out of a job.
“No one is talking about replacing pathologists any time soon,” Balis says. “But working in tandem with this technology, the hope is that they will be able to achieve a higher overall level of performance.”
Disclosures: U-M is currently seeking licensing partners to bring this technology to market.
Balis is on the technical advisory board of Aperio Technology, although no resources of any kind from that source were utilized in the above studies.
Additional Authors: Jerome Y. Cheng, M.D., Informatics fellow, U-M Department of Pathology. Mehmet Toner, Ph.D. and Ronald G. Tomkins, M.D., Sc.D., both of Massachusetts General Hospital and Harvard Medical School.
Reference: “Spatially Invariant Vector Quantization: A pattern matching algorithm for multiple classes of image subject matter including pathology,” Journal of Pathology Informatics, Feb. 2011.
