Augmented reality (AR) and virtual reality (VR) are concepts we’ve heard of, but only seen in films such as Terminator, Minority Report and Mission Impossible. In this article, Tom Lovelock describes how AR/VR will impact our lives.
A recent article by Wired titled ‘AR Will Spark the Next Big Tech Platform – Call it Mirrorworld’ [1] has inspired my thoughts recently about augmented reality (AR) and medicine. The article describes an emerging “mirrorworld” existing as an extra layer over reality itself. It imagines uses of this in different industries, but neglects to discuss how this could exist in medicine.
The data necessary for a medical mirrorworld has been steadily accumulating for many years. How will this develop? How close are we to seeing this in use in medical education or in surgery? Using my experiences working with head & neck consultants as a Learning Technology Officer at Keele University Anatomy & Surgical Training Centre (KASTC), I hope to share some insight.
“Want to see a patient’s pre-surgery scans? Simply view them overlaid onto their body, anatomically sized and placed”
Imagine the information contained in a lifetime of patient records: notes, scans, x-ray, procedures, prescriptions, etc. Now imagine this overlaid onto a patient whenever and wherever they interact with a medical professional wearing an AR headset. Want to see a patient’s pre-surgery scans? Simply view them overlaid onto their body, anatomically sized and placed. Need to see scans from six months ago? Scroll back through their ‘history’ to experience any changes. Need to know their current medication and dosage? It could appear on demand during an examination of the patient as the AR equipment recognises keywords in the discussion taking place. This may seem like science fiction, but there are many signs of a near future containing tools like this.
Surgical training delegates of all levels being guided remotely by
Faculty via Proximie Tele-Surgical Platform. (Image courtesy of Keele University).
Microsoft Hololens 2. (Image courtesy of Microsoft.)
We can already view CT and MR images in VR [2] and AR [3] using relatively affordable equipment, and with the relentless development of new display devices like Microsoft’s Hololens 2 [4], this will become more accessible, with higher fidelity, thus increasing the value of AR imaging to medicine. A key feature that is not currently common is collective viewing within AR, where a group of medical professionals could experience the same content from their own viewpoint.
At KASTC, we are always looking for ways of incorporating new anatomy resources in our teaching. My colleague, Mr Ajith George, has been incorporating AR into his teaching of surgical trainees for over a year, his tools and techniques developing with each teaching session, reflecting the rapid development of the tools available to him. Tools such as 3D4Medical Complete Anatomy 2019 [5], which gives excellent quality imagery and is currently made available to undergraduate medical students for their studies at Keele University School of Medicine. Mr George favours Primal Pictures 3D Atlas of Human Anatomy [6] for his postgraduate teaching, commenting that “primal is a premium product and has a premium price, which is worth it for the specialists.” The ability to quickly create 3D models of complex anatomical and surgical concepts is key to this development, and once the educators have more control over content creation, I feel that more widespread adoption will follow.
Concept art of surgeons interacting with anatomical images in mixed reality. (Image courtesy of Microsoft)
A blend of traditional and technology enhanced resources are used for anatomical teaching
at Keele Anatomy & Surgical Training Centre. (Image courtesy of Keele University)
Mr Ajith George teaching during a head & neck dissection course at Keele Anatomy
& Surgical Training Centre. (Image courtesy of Keele University).
Concept Art of Surgeons interacting with anatomical images in mixed reality (Image courtesy of Microsoft.)
We are also at the start of a change in the way mentoring and surgical coaching might take place. Proximie [7] is a platform that combines some of the features I’ve previously described as science fiction. It uses elements of AR, keyword recognition and a high level of interactivity to provide a telesurgical platform that can connect surgeons anywhere in the world. At KASTC we are going to be using this platform for surgical training, allowing groups of surgeons to connect and experience a cadaveric lab session remotely, with no geographical boundaries to participation. We are looking forward to demonstrating this platform and its benefits during live dissections at BACO 2020 in Birmingham. So, whilst a ‘mirrorworld’ of medical data is still a long way in the future for most, we are starting to experience the first pioneers of this technology. It is important that we engage with these developments to ensure that they meet their enormous potential to be a powerful tool for patients and professionals alike.
References
1. Kelly K. AR Will Spark the Next Big Tech Platform—Call It Mirrorworld.
www.wired.com/story/
mirrorworld-ar-next-big-tech-platform/
Last accessed July 2019.
2. The Body VR. Anatomy Viewer.
https://thebodyvr.com/anatomy-viewer/
Last accessed July 2019.
3. Virtual Surgery Intelligence.
https://vsi.health/en/
Last accessed July 2019.
4. Microsoft HoloLens 2: Partner Spotlight with Philips.
https://youtu.be/loGxO3L7rFE
Last accessed July 2019.
5. Complete Anatomy 2020. The future of anatomy is here.
https://3d4medical.com/
Last accessed July 2019.
6. Primal Pictures. The Leading 3D Antamoy Resource.
https://primalpictures.com/
Last accessed July 2019.
7. Proximie. Augmented reality technology empowering clinicians.
https://www.proximie.com/
Last accessed July 2019.
