Locations:
Search IconSearch

Face Transplant Evolves With Amazing Hologram Technology

Cleveland Clinic first to use augmented reality in this way

Total Face Transplant Augmented Reality

We loved seeing the ghostly image of Princess Leia plead for help in the first Star Wars flick.

Advertisement

Cleveland Clinic is a non-profit academic medical center. Advertising on our site helps support our mission. We do not endorse non-Cleveland Clinic products or services. Policy

Today, the same fascinating hologram technology has found its way into medicine. Cleveland Clinic is first to use augmented reality, or AR, in face transplantation.

Rather than create an artificial world like virtual reality, AR enhances the environment you see — with a computer’s help.

In the first total face transplant here in 2017, the computer was lodged in a headset surgeons wore.

It superimposed holographic images of hidden structures onto what remained of a gunshot survivor’s face — along with images of the donor’s anatomy — to guide the transplant team.

Their goal: to give the young woman, Katie Stubblefield, the opportunity to eat, swallow, speak and show emotion once again.

An amazing perspective

“It was like having X-ray vision,” says Frank Papay, MD, Chairman of the Dermatology & Plastic Surgery Institute and a leader of the surgical team.

“You put the AR headset on and saw the patient’s anatomy in 3-D. You could walk around the holographic image, and interact with it. As a surgeon, I’d never had this amazing perspective.”

It took more than a year of collaboration for the medical team and biomedical engineers, led by Karl West, MS, to program the unique applications for the Microsoft HoloLens®.

A unique challenge

In any transplant, surgeons must join the donor’s blood vessels and nerves to the recipient’s so the graft will “take.”

In face transplant, they must join intricate facial vessels and nerves, and — since everyone’s facial bones are unique — carefully align skull, jaw and other bony structures.

To make this happen, AR helped the surgeons visualize structures they couldn’t see, and decide how much tissue and bone to remove from Katie and her donor.

From digital data to 3D

Before the transplant, the biomedical engineers combed through data from computed tomography (CT) scans of Katie and the donor.

“We imported only critical information from the CT scans into the AR headset to create the 3D reconstructions — it’s surprising how much information CTs contain,” says West. “Then a special lens projected those holograms onto the surgeons’ goggles.”

A quick turnaround

Once a donor was found, West’s team had about 24 hours to generate similar 3D holograms of the donor’s anatomy.

“We worked all night to extract the most important information from the donor’s CTs,” he says.

Dr. Papay adds, “It really helped us see how the skull, bony structures and blood vessels fit together in different scenarios. Seeing blood vessels deep in the skull was especially critical.”

Advertisement

The transplant team overlaid the holographic images onto Katie’s face to adjust their approach throughout the surgery.

3D printer adds another dimension

To help with visualizing the procedure, the transplant team created exact plastic replicas of the recipient’s and donor’s heads with a 3D printer, a process that takes many hours.

“We’ve used 3D-printed models, which are actual size, in all our face transplants. They’re good for visualizing larger structures,” says Dr. Papay.

“AR is much better at visualizing small structures, like blood vessels, and displaying different scenarios to guide our surgical strategy.”

The future is here

Dr. Papay sees Cleveland Clinic’s pioneering use of AR in face transplantation as an innovation that has found its place.

“Face transplant is a natural application of this technology, and we’re just getting started,” he says.

AR is no longer futuristic technology “from a galaxy far, far away.” It’s advancing medicine right here, right now.

Advertisement

Learn more about our editorial process.

Related Articles

Healthcare provider in gloves holding hand of patient in hospital bed
April 2, 2024/Transplant
Being an Organ Donor Means You Could Save Many Lives

More than 20 organs and tissues can be donated, and one donor can help more than 80 people

Doctor with patient showing new kidney for transplant.
September 29, 2022/Transplant
11 Kidney Transplant Recovery Tips

Rest, hydration, protein and light exercise all play an important role in your healing

Stomach Pain
May 30, 2019/Digestive
You Won’t Believe How This Works: Fecal Transplant

Healthy gut flora defeat Clostridium difficile infection

Katie Stubblefield After First Total Face Transplant
August 15, 2018/Transplant
A Young Woman’s Total Face Transplant: Her Road to Recovery

Gunshot wound survivor's recovery continues

A person's back, covered in moles and freckles, with their hand reaching over their shoulder
July 22, 2024/Skin Care & Beauty
What To Expect During a Full-Body Skin Cancer Screening

During an annual exam, your provider will check for any moles or spots that have changed in size, color or shape

Male standing on beach with hands behind his head, staring into distance and exhaling
July 22, 2024/Mental Health
Mental Health in Athletes: Breaking the Stigma

A more open conversation on athletes and their mental health needs is overdue

Trending Topics

Female and friend jogging outside
How To Increase Your Metabolism for Weight Loss

Focus on your body’s metabolic set point by eating healthy foods, making exercise a part of your routine and reducing stress

stovetop with stainless steel cookware and glassware
5 Ways Forever Chemicals (PFAS) May Affect Your Health

PFAS chemicals may make life easier — but they aren’t always so easy on the human body

jar of rice water and brush, with rice scattered around table
Could Rice Water Be the Secret To Healthier Hair?

While there’s little risk in trying this hair care treatment, there isn’t much science to back up the claims

Ad