August 14, 2018/Transplant

Face Transplant Evolves With Amazing Hologram Technology

Cleveland Clinic first to use augmented reality in this way

Total Face Transplant Augmented Reality

We loved seeing the ghostly image of Princess Leia plead for help in the first Star Wars flick.

Advertisement

Cleveland Clinic is a non-profit academic medical center. Advertising on our site helps support our mission. We do not endorse non-Cleveland Clinic products or services. Policy

Today, the same fascinating hologram technology has found its way into medicine. Cleveland Clinic is first to use augmented reality, or AR, in face transplantation.

Rather than create an artificial world like virtual reality, AR enhances the environment you see — with a computer’s help.

In the first total face transplant here in 2017, the computer was lodged in a headset surgeons wore.

It superimposed holographic images of hidden structures onto what remained of a gunshot survivor’s face — along with images of the donor’s anatomy — to guide the transplant team.

Their goal: to give the young woman, Katie Stubblefield, the opportunity to eat, swallow, speak and show emotion once again.

An amazing perspective

“It was like having X-ray vision,” says Frank Papay, MD, Chairman of the Dermatology & Plastic Surgery Institute and a leader of the surgical team.

“You put the AR headset on and saw the patient’s anatomy in 3-D. You could walk around the holographic image, and interact with it. As a surgeon, I’d never had this amazing perspective.”

It took more than a year of collaboration for the medical team and biomedical engineers, led by Karl West, MS, to program the unique applications for the Microsoft HoloLens®.

Advertisement

A unique challenge

In any transplant, surgeons must join the donor’s blood vessels and nerves to the recipient’s so the graft will “take.”

In face transplant, they must join intricate facial vessels and nerves, and — since everyone’s facial bones are unique — carefully align skull, jaw and other bony structures.

To make this happen, AR helped the surgeons visualize structures they couldn’t see, and decide how much tissue and bone to remove from Katie and her donor.

From digital data to 3D

Before the transplant, the biomedical engineers combed through data from computed tomography (CT) scans of Katie and the donor.

“We imported only critical information from the CT scans into the AR headset to create the 3D reconstructions — it’s surprising how much information CTs contain,” says West. “Then a special lens projected those holograms onto the surgeons’ goggles.”

A quick turnaround

Once a donor was found, West’s team had about 24 hours to generate similar 3D holograms of the donor’s anatomy.

“We worked all night to extract the most important information from the donor’s CTs,” he says.

Dr. Papay adds, “It really helped us see how the skull, bony structures and blood vessels fit together in different scenarios. Seeing blood vessels deep in the skull was especially critical.”

Advertisement

The transplant team overlaid the holographic images onto Katie’s face to adjust their approach throughout the surgery.

3D printer adds another dimension

To help with visualizing the procedure, the transplant team created exact plastic replicas of the recipient’s and donor’s heads with a 3D printer, a process that takes many hours.

“We’ve used 3D-printed models, which are actual size, in all our face transplants. They’re good for visualizing larger structures,” says Dr. Papay.

“AR is much better at visualizing small structures, like blood vessels, and displaying different scenarios to guide our surgical strategy.”

The future is here

Dr. Papay sees Cleveland Clinic’s pioneering use of AR in face transplantation as an innovation that has found its place.

“Face transplant is a natural application of this technology, and we’re just getting started,” he says.

AR is no longer futuristic technology “from a galaxy far, far away.” It’s advancing medicine right here, right now.

Learn more about our editorial process.

Related Articles

Katie Stubblefield After First Total Face Transplant
August 14, 2018/Transplant
A Young Woman’s Total Face Transplant: Her Road to Recovery

Gunshot wound survivor's recovery continues

Healthcare provider in gloves holding hand of patient in hospital bed
April 2, 2024/Transplant
Being an Organ Donor Means You Could Save Many Lives

More than 20 organs and tissues can be donated, and one donor can help more than 80 people

Doctor with patient showing new kidney for transplant.
September 28, 2022/Transplant
11 Kidney Transplant Recovery Tips

Rest, hydration, protein and light exercise all play an important role in your healing

A person's emotional storm raining down on someone else
April 22, 2024/Mental Health
Vicarious Trauma: What To Do When Others’ Distress Impacts Your Well-Being

Absorbing others’ trauma can take a big toll on your own mental health

Caregiver and child with doctor in kid-friendly pediatrician office
What’s the Best Age for Your Kid To Start Growth Hormone Treatment?

This therapy may be recommended at any age, but it’s commonly started before puberty

Moisturizer being applied to older hands
April 22, 2024/Skin Care & Beauty
How To Make Your Hands Look Younger

To help keep your mitts feeling and looking their best, moisturize, exfoliate, wear sunscreen and eat a healthy diet

Person eating tiny amount of bland food, daydreaming about larger portion of better food, amidst silhouette of crowd eating
April 19, 2024/Mental Health
The Intersection of Obesity and Eating Disorders

Having overweight and disordered eating is a high-risk combination that often gets dismissed or overlooked

Healthcare provider and patient talking in exam room
The Link Between Smoking and Bladder Cancer

Puffing on cigarettes is the leading cause of bladder cancer

Trending Topics

Person in yellow tshirt and blue jeans relaxing on green couch in living room reading texts on their phone.
Here’s How Many Calories You Naturally Burn in a Day

Your metabolism may torch 1,300 to 2,000 calories daily with no activity

woman snacking on raisins and nuts
52 Foods High In Iron

Pump up your iron intake with foods like tuna, tofu and turkey

Ad