Advertisement
Cleveland Clinic first to use augmented reality in this way
We loved seeing the ghostly image of Princess Leia plead for help in the first Star Wars flick.
Advertisement
Cleveland Clinic is a non-profit academic medical center. Advertising on our site helps support our mission. We do not endorse non-Cleveland Clinic products or services. Policy
Today, the same fascinating hologram technology has found its way into medicine. Cleveland Clinic is first to use augmented reality, or AR, in face transplantation.
Rather than create an artificial world like virtual reality, AR enhances the environment you see — with a computer’s help.
In the first total face transplant here in 2017, the computer was lodged in a headset surgeons wore.
It superimposed holographic images of hidden structures onto what remained of a gunshot survivor’s face — along with images of the donor’s anatomy — to guide the transplant team.
Their goal: to give the young woman, Katie Stubblefield, the opportunity to eat, swallow, speak and show emotion once again.
“It was like having X-ray vision,” says Frank Papay, MD, Chairman of the Dermatology & Plastic Surgery Institute and a leader of the surgical team.
“You put the AR headset on and saw the patient’s anatomy in 3-D. You could walk around the holographic image, and interact with it. As a surgeon, I’d never had this amazing perspective.”
It took more than a year of collaboration for the medical team and biomedical engineers, led by Karl West, MS, to program the unique applications for the Microsoft HoloLens®.
In any transplant, surgeons must join the donor’s blood vessels and nerves to the recipient’s so the graft will “take.”
In face transplant, they must join intricate facial vessels and nerves, and — since everyone’s facial bones are unique — carefully align skull, jaw and other bony structures.
To make this happen, AR helped the surgeons visualize structures they couldn’t see, and decide how much tissue and bone to remove from Katie and her donor.
Before the transplant, the biomedical engineers combed through data from computed tomography (CT) scans of Katie and the donor.
“We imported only critical information from the CT scans into the AR headset to create the 3D reconstructions — it’s surprising how much information CTs contain,” says West. “Then a special lens projected those holograms onto the surgeons’ goggles.”
Once a donor was found, West’s team had about 24 hours to generate similar 3D holograms of the donor’s anatomy.
“We worked all night to extract the most important information from the donor’s CTs,” he says.
Dr. Papay adds, “It really helped us see how the skull, bony structures and blood vessels fit together in different scenarios. Seeing blood vessels deep in the skull was especially critical.”
Advertisement
The transplant team overlaid the holographic images onto Katie’s face to adjust their approach throughout the surgery.
To help with visualizing the procedure, the transplant team created exact plastic replicas of the recipient’s and donor’s heads with a 3D printer, a process that takes many hours.
“We’ve used 3D-printed models, which are actual size, in all our face transplants. They’re good for visualizing larger structures,” says Dr. Papay.
“AR is much better at visualizing small structures, like blood vessels, and displaying different scenarios to guide our surgical strategy.”
Dr. Papay sees Cleveland Clinic’s pioneering use of AR in face transplantation as an innovation that has found its place.
“Face transplant is a natural application of this technology, and we’re just getting started,” he says.
AR is no longer futuristic technology “from a galaxy far, far away.” It’s advancing medicine right here, right now.
Advertisement
Learn more about our editorial process.
Advertisement
More than 20 organs and tissues can be donated, and one donor can help more than 80 people
Rest, hydration, protein and light exercise all play an important role in your healing
The field of uterus transplant is advancing
Clinical trial targets women with UFI
Healthy gut flora defeat Clostridium difficile infection
Gunshot wound survivor's recovery continues
The mental block is a brain-body disconnect that causes gymnasts to lose their sense of space on flips
The results aren’t great, and the risks — like vision problems — aren’t worth it
Focus on your body’s metabolic set point by eating healthy foods, making exercise a part of your routine and reducing stress
PFAS chemicals may make life easier — but they aren’t always so easy on the human body
While there’s little risk in trying this hair care treatment, there isn’t much science to back up the claims