We loved seeing the ghostly image of Princess Leia plead for help in the first Star Wars flick.
Cleveland Clinic is a non-profit academic medical center. Advertising on our site helps support our mission. We do not endorse non-Cleveland Clinic products or services. Policy
Today, the same fascinating hologram technology has found its way into medicine. Cleveland Clinic is first to use augmented reality, or AR, in face transplantation.
Rather than create an artificial world like virtual reality, AR enhances the environment you see — with a computer’s help.
In the first total face transplant here in 2017, the computer was lodged in a headset surgeons wore.
It superimposed holographic images of hidden structures onto what remained of a gunshot survivor’s face — along with images of the donor’s anatomy — to guide the transplant team.
Their goal: to give the young woman, Katie Stubblefield, the opportunity to eat, swallow, speak and show emotion once again.
An amazing perspective
“It was like having X-ray vision,” says Frank Papay, MD, Chairman of the Dermatology & Plastic Surgery Institute and a leader of the surgical team.
“You put the AR headset on and saw the patient’s anatomy in 3-D. You could walk around the holographic image, and interact with it. As a surgeon, I’d never had this amazing perspective.”
It took more than a year of collaboration for the medical team and biomedical engineers, led by Karl West, MS, to program the unique applications for the Microsoft HoloLens®.
A unique challenge
In any transplant, surgeons must join the donor’s blood vessels and nerves to the recipient’s so the graft will “take.”
In face transplant, they must join intricate facial vessels and nerves, and — since everyone’s facial bones are unique — carefully align skull, jaw and other bony structures.
To make this happen, AR helped the surgeons visualize structures they couldn’t see, and decide how much tissue and bone to remove from Katie and her donor.
From digital data to 3D
Before the transplant, the biomedical engineers combed through data from computed tomography (CT) scans of Katie and the donor.
“We imported only critical information from the CT scans into the AR headset to create the 3D reconstructions — it’s surprising how much information CTs contain,” says West. “Then a special lens projected those holograms onto the surgeons’ goggles.”
A quick turnaround
Once a donor was found, West’s team had about 24 hours to generate similar 3D holograms of the donor’s anatomy.
“We worked all night to extract the most important information from the donor’s CTs,” he says.
Dr. Papay adds, “It really helped us see how the skull, bony structures and blood vessels fit together in different scenarios. Seeing blood vessels deep in the skull was especially critical.”
The transplant team overlaid the holographic images onto Katie’s face to adjust their approach throughout the surgery.
3D printer adds another dimension
To help with visualizing the procedure, the transplant team created exact plastic replicas of the recipient’s and donor’s heads with a 3D printer, a process that takes many hours.
“We’ve used 3D-printed models, which are actual size, in all our face transplants. They’re good for visualizing larger structures,” says Dr. Papay.
“AR is much better at visualizing small structures, like blood vessels, and displaying different scenarios to guide our surgical strategy.”
The future is here
Dr. Papay sees Cleveland Clinic’s pioneering use of AR in face transplantation as an innovation that has found its place.
“Face transplant is a natural application of this technology, and we’re just getting started,” he says.
AR is no longer futuristic technology “from a galaxy far, far away.” It’s advancing medicine right here, right now.
