AR and VR in healthcare use cases include training surgeons, providing "bucket list" experiences for hospice patients and managing pain relief, as a series of healthcare and tech leaders highlighted at the Giant Health Event 2018 in London
The Microsoft HoloLens is a good example of AR and VR in healthcare
From training surgeons to helping patients nearing the end of their lives tick off items on their bucket list, the use of AR and VR in healthcare could make a profound impact on people’s lives.
The technologies – augmented reality and virtual reality to give them their full names – created by digital healthcare start-ups and tech giants are increasingly being adopted by healthcare providers.
Their potential was showcased at the first day of Giant Health Event 2018, a medtech conference being held at Chelsea FC’s Stamford Bridge stadium in London this week.
Professor Shafi Ahmed, a surgeon who also supports scale-up digital healthcare companies around the world, believes this reality-bending innovations – alongside other tech like artificial intelligence (AI) – have made this the “most exciting time to be working in medicine”.
“I’ve been in medicine for 25 years and never been happier because we finally see the convergence of technology – whether it’s AI, blockchain, wearables or AR and VR – all coming together at the same time.
“We’ve never had that before so I’m excited about the future and can see the energy to make healthcare more affordable, accessible and equitable.”
While plenty has been said about AI, it is the immersive technologies that Oculus VR’s young founder Palmer Luckey called “the most important technology in the history of humanity”.
With that in mind, we look at some of the ways AR and VR are being used in healthcare to save and transform lives.
What is AR and VR?
AR adds digital elements to a real-world setting, often by using a smartphone’s camera.
Mainstream examples of augmented reality experiences include Snapchat lenses and the Pokemon Go mobile game, which places animated characters in a player’s live view.
VR, on the other hand, is a total immersive experience that shuts out the physical world.
Using a VR headset, it transports users into a new “reality” of real-world and imagined environments.
Another emerging technology is mixed reality (MR), which combines both and is perhaps most commonly associated with the Microsoft HoloLens goggles.
Training surgeons with VR
Across the world, there are five billion people with no access to safe or affordable surgery.
Just to achieve a status quo requires another 2.2 million surgeons but – including university, core surgical training and learning a specialism – it can take 15 years to become fully-qualified in countries such as Britain.
At the Royal London Hospital where Dr Ahmed teaches, there are about 300 medical students in each year group.
A critical part of their education involves shadowing surgeons at work but, with only two students are allowed inside an operating theatre at any one time, this causes a logistical problem.
It prompted digital healthcare start-up Medical Realities to come up with a solution that uses VR in surgical training.
By installing numerous cameras at different angles around an operating theatre to create a 360-degree view, medical students can wear a VR headset to immerse themselves in what can be a high-pressure environment to gain vital experience and feel more comfortable.
The footage is made available on a downloadable app and anyone with a VR headset can access a VR operation.
Medical Realities co-founder and CEO Steve Dann says: “For the first time ever, that lack of being able to have proper surgery is killing more people than infectious diseases, with 18 million people dying around the world as a result.
“We’re trying to save lives by teaching medical students how to be surgeons, which is particularly important in developing countries.”
In April 2016, Dr Ahmed – a partner in Medical Realities – performed the first ever live 360-degree surgery, which was watched by 55,000 people online.
Matt Leatherbarrow is CTO of Medical Realities’ digital media production subsidiary Amplified Robot, which specialises in the 360-degree filming.
He says: “When you think there were two students allowed in the operating theatre versus 55,000 getting a similar experience, it has a big impact on education and training.
“In most cases, it’s actually a better experience because students will sit at the back of the room before they are brought forward by a surgeon but with VR, you can stand next to the surgeon.”
The Medical Realities VR platform also features quizzes on what the students’ have seen and learned during the VR experience, plus they can bring up key parts of the anatomy for reference over the virtual environment.
Other companies working in the field include Osso VR, which is focused on training surgeons for orthopaedic and spine therapies.
VR for pain relief
One of the first medical uses of VR was using the technology as a clinical tool for pain management.
In 1996, University of Washington researchers Hunter Hoffman and David Patterson came up with the idea of using immersive technology to relieve pain through the means of distraction.
The university’s HITLab and Harborview Burn Center, in Seattle, developed SnowWorld as the first virtual world for reducing pain – specifically aimed at burn patients, such as wounded soldiers.
Patients become immersed in a computer-generated icy adventure where their task is to throw endless amounts of snowballs at penguins to shift their attention away.
Mr Dann says: “Burn patients experience some of the worst pain imaginable and it’s particularly painful two or three times a day because the nurses have to change dressings.
“This is excruciatingly painful, no matter which drugs they give the patient, so they use VR as a distraction technique.
“But it’s not just a distraction because VR is taking you to another reality, and the trick is to make the patient think that is their reality.”
Results have shown burn patients experience between 35% and 50% less pain when immersed in VR.
The University of Washington explains how pain perception has a strong psychological component.
“The same incoming pain signal can be interpreted as painful or not, depending on what the patient is thinking,” it says.
“Pain requires conscious attention. The essence of VR is the illusion users have of going inside the computer-generated environment.
“Being drawn into another world drains a lot of attentional resources, leaving less attention available to process pain signals.”
The technology has been rolled out to also help patients experiencing “phantom limbs” – an often painful sensation that amputated arms or legs are still attached – while Mr Dann references a Mexican doctor who uses VR rather than an anaesthetic for removing non-benign tumours.
HoloLens leads AR in healthcare
Microsoft HoloLens is the first AR headset that creates holograms of digital content within a real-world environment.
At $3,000 (£2,300), consumer take-up has been slow but it is already being adopted in healthcare.
Medical students can bring up a realistic image of a brain or heart in a classroom to rotate, enlarge take apart and examine in-depth in front of their eyes, while surgeons can use holograms to prepare for complex operations.
Mr Dann says: “You can take any anatomical part to understand how it works.
“It improves students’ confidence in learning the anatomy and allows people to fail, which is the best way to learn but not something you can do too much in medicine.
“Having a 3D understanding of something is much more powerful in how you navigate it in your mind.”
Dr Ahmed at Medical Realities, meanwhile, has used the HoloLens to bring up holograms of other doctors based in other parts of the world while performing a surgery to discuss procedures as if they were all in the same room.
Some digital healthcare companies have also began to integrate HoloLens in their technology.
One of these is CAEHealthcare’s VimedixAR, an ultrasound simulator that enables healthcare professionals to see how an ultrasound beam cuts through human anatomy in real-time.
Its CAE LucinaAR programme is a wireless childbirth simulator that provides 3D holograms of a fetus and also prepares clinical learners for childbirth.
Treating hologram patients with AR
British multinational education firm Pearson went one further by creating holograms of human patients through its HoloPatient mixed reality platform.
Trainee doctors, nurses and paramedics can interact with AI-equipped virtual “standardised patients” using gestures, gaze and voice so they can learn about various clinical scenarios.
The technology – based on real-life simulation exercises with actors – works with the HoloLens and places the 3D digital patients inside the room where the user is based.
After being launched in May, it has been used across the world and has become one of Pearson’s most popular training products.
Mark Christian, global director in immersive technology, says: “The applications themselves aren’t transformative because using a standardised patient isn’t a new concept for teaching.
“But we’re providing the toolsets so teachers can do things they’ve never been able to do before, and it can be done cheaper and at scale.
“A teacher can change the scenario how they want and lead the students down different pathways.
“As the AI develops and we can have more natural conversations with avatars, we’ll embrace that.”
Ticking off bucket list wishes with AR in palliative care
It’s not just potentially life-saving benefits that AR and VR offer, but they can also improve quality of life too.
For many hospice patients suffering from terminal illnesses, their remaining days can be lived out bedbound.
But immersive technology has been trialled to help them tick off “bucket list” wishes, such as swimming with dolphins, go skydiving or running with wild horses – all in a virtual environment.
A pilot programme at the Royal Trinity Hospice, in Clapham, South London, has equipped patients with VR headsets to provide the experiences after working with film production company Flix Films.
In a video, one patient who used the experience says: “There’s something about life, especially when you’re coming to the end of it, where you want to recapture memories but it’s not practical.
“The complete exhilaration and joy I felt is something I haven’t had for a long time – joy isn’t something that’s part of my life now but for a moment I had it.”
Letizia Perna-Forrest, head of patient and family support at the hospice, admits the hospice charity was initially reticent about using the technology because it was untested.
There were also concerns it could lead to withdrawal symptoms after the VR experiences were finished once patients return to “real life”.
But Ms Perna-Forrest says: “We have patients who are bedbound and their world has become very small as a result of their terminal diagnosis.
“This has shown that hospices aren’t a place where people die, but where there’s life.
“Ultimately, what we’re aiming to understand is how VR therapy can benefit people from a physical, as well as psychological, point of view.
“And, actually, the preliminary results are fantastic. We’ve reduced patients’ pain, shortness of breath, anxiety and depression. We’ve even made someone less incontinent.”
It’s not just hospice patients, either. In 2016, Sarah Ezekiel – who had been diagnosed with motor neurone disease 16 years earlier and was paralysed – was able to swim with dolphins in a virtual setting after saying it was one of her regrets not to have done so when she was able-bodied.
Flix Films managing director Leon Ancliffe says: “When you give someone the chance to experience something beyond their physical ability, it’s the most incredible gift.”
The VR technology has also been used to train palliative care staff to address high staff turnover, as well as staff at prisons and Moorfields Eye Hospital in London.