Amber Hickman |
Medical iSight has collaborated with the BBC to demonstrate how the Microsoft HoloLens 2 can be used to create 3D visualisations of the human body for a new TV programme, Your Body Uncovered.
During the show, the patient, a consultant and the host all wore HoloLens 2 mixed reality headsets so that they could see 3D visualisations of the patient’s organs at the same time. This enabled the consultant to better describe the patient’s medical issues, while allowing the patient to gain a better understanding of how their body functions.
Consultants were shown how to use simple voice commands and hand gestures, such as pinching or pulling, to manipulate the 3D images. Spatial anchors were used in the room as landmarks for the headsets to ensure that all three saw the same digital replica at the same time.
“At the beginning we looked at virtual reality, but that wouldn’t work because it’s hard to translate what people are seeing through the headsets to people watching on TV at home,” said Patrick Furlong, executive producer of Your Body Uncovered. “We wanted to show patients interacting with their body and experiencing it. Philip showed us a digital replica of a skeleton using HoloLens and we all thought it was amazing. We realised we could use HoloLens to film everything we needed to, we could even flip the visor up and see patients’ faces and emotion.”
The patients that appeared on the show had all suffered from different conditions, including cancer, Covid-19 and strokes. Some had been told they needed surgery but did not understand what that would entail, and some were not sure if they wanted to undergo a procedure.
“If someone had an issue with their spine, a consultant would usually show them a generic model of a spine or draw something on a pad of paper,” said Dr Philip Pratt, chief scientific officer at Medical iSight, who consulted with the production company throughout the project. “But now they could see their own body in fantastic detail.”
To enable audiences watching the show at home to experience the same point of view as the patients and consultants, producers attached a HoloLens to the camera so that it could move around the studio. They then added a digital replica of the organ in the post-production stage.
“When some people think of mixed reality, they question whether the digital content can connect with people on an emotional level, but this really did,” said Furlong. “It shows that technology doesn’t have to be cold and clinical, it can have a human application, too.”