Four Use Cases for Augmented Reality in Medical Education

Medical care is currently undergoing a revolution that changes the way we interact with our world. Augmented and virtual reality solutions are improving the pace of technical education, streamlining the sales process of bulky medical equipment, improving patient outcomes, and connecting experts in their field to initiates at any distance for guided procedures. The global pandemic of 2020 has highlighted the importance of limiting human contact in a highly connected world to stem the spread of disease. The use cases for augmented and virtual reality are many, some having yet to be realized, but many of which are already being employed to great effect, especially in medicine. In a recent medical education discussion, Doug Stephen, president of enterprise learning at CGS, shared four very tangible use cases around how modern technologies empower healthcare providers to safely and effectively care for their patients.

Use Case 1: Leveraging AR to Improve Success Rates in the Operating Room

Our first use-case takes place directly in the operating theater. Here, a head mounted tablet device utilizes Microsoft’s HoloLens technology to overlay the neurosurgeon’s visual field with the patient’s three-dimensional brain scans. As mentioned in the clip, it’s said that the best surgeons have excellent spatial abilities, and this technology puts perfect spatial illustration in the hands of every surgeon. The surgeon is then able to manipulate the digital imagery with hand gestures, sliding through scans for the desired section, and even overlaying the scaled image on the patient’s head to visualize the actual localization of lesions. The marriage of imaging and AR provides greater success rates for physicians, and better outcomes for patients.

 

Use Case 2: Off Site Training and the Digital Twin

As a direct result of the COVID-19 pandemic, employers have been exploring novel methods of training employees that reduce the need for on-site training. With augmented reality, training can occur in a personal office, living room, or even bedroom. [4:30] This allows for familiarization with equipment so that when on-site training does occur, the time necessary is significantly collapsed to only the essentials of technique and protocols. Often, medical devices cost thousands of dollars or more, and it can be impractical to provide access to every trainee. With the so-called “Digital-Twin,” an AR asset that mirrors the functionality of real equipment can be deployed to anyone with a smartphone, and through simple, guided click and tap interactivity, digital muscle memory can be built up that translates directly to skills in the field.

The example presented in the video is a HemoCue diagnostic device with a digital display that features real units and statistics that might be seen on the machine. The asset can be delivered in an app-less manner over the internet, or may be featured in an app environment for higher fidelity and offline access. Tracking of motions and gestures is also possible permitting an evaluation of training efficacy for better outcomes. Overall, this innovation provides an opportunity for off-site training, remote collaboration, and interaction with simulated real-world tools and environments.

 

Use Case 3: Expensive, Bulky Equipment, In Your Pocket

One of the biggest obstacles to the sale and presentation of medical equipment is the size and expense of most devices. An MRI machine, for example, would be impractical to carry around. Augmented reality permits the opportunity for a scale model to be instantly deployed in a room, and scaled to full size for evaluating fit within a potential workspace. In addition, the model features full functionality, with instructions and interactivity. This shortens the sale cycle and provides clinicians with tangible information on prospective devices and their value to invest in. Devices may be deployed remotely, such as in a virtual call, or used in more traditional in-person meetings.

Another feature of demonstrations made in this fashion is the relationship with cloud services that provide additional functionality to AR assets. TrueFidelity, for example, uses AI to improve the contrast and reduce noise in radiology scans to help clinicians make faster, more accurate decisions. This service can be deployed at a tap over deployed AR assets. A line of devices may also be deployed in tandem to evaluate the degree of improvement a facility might see from their implementation. Thus, the spatial relationship between document scanning, an MRI machine, and cloud services might be evaluated in a highly visual way for how throughput might be improved in a month or a year, and how much time might be saved by medical staff using these devices.

With AR-deployed equipment, potential investors are able to see how a machine costing millions will work, fit into their prospective space/labs/facilities, and see how doctors and nurses or technicians might walk around and interact with those machines. Much opportunity exists for engaging and immediately accessible statistics, measurements of accuracy, and other important information. Together, these features save time, money, and lives.

 

Use Case 4: Mixed Reality

The final use-case we’ll examine is with mixed reality applications, in which augmented and virtual reality technology can be paired for additional functionality. [14:00] AR can be used for a preliminary view, for example, available to anyone with a smartphone, followed by transfer into a VR room with potentially multiple people around the world working together simultaneously and communicating with immediate feedback.

The virtual phlebotomy application featured in the video offers a self-paced and self-directed AR lesson with a blood pressure cuff. It may be a solitary experience with simple callouts for instruction, or move into a VR environment for working with an instructor, or where technique can be assessed. Digital devices and environments can provide both visual and haptic feedback to aid in working with the digital objects. Rolling a vein, for example, might offer visual, vibration, and sound feedback, while new media will be able to provide ever more realistic experiences. Trainees need to talk to patients, work with computers, and collect patient statistics while performing the manual tasks of phlebotomy, as they would in a real blood drive. Everything can be tracked and scored to prove competency, and assets are entirely reusable. AR and VR mediums can have multiple users, such that an instructor might demonstrate to a class to replace traditional on-site engagement with a dummy or other simulation.

Everything can be practiced at a pace suited to the needs of every individual. This provides an advantage in getting people closer to the material than they might in a video, building digital muscle memory, providing familiarity with the technology and techniques, and feedback from virtual patients assists in teaching the routines of bedside manner. After a team has practiced remotely, they can come in for on-site training, moving at the same pace, and performing at the same time to get a lot more out of the experience for better outcomes in a safe environment. AR, VR, and XR really allow the healthcare industry to get the best out of their employees who become demonstrably highly effective in the workforce.

 

Necessity is the mother of invention. The time and costs of creating material and the hardware to support AR uses cases in medical settings is rapidly going down, and quality is improving. These innovations provide real value, not only for the sake of demonstration, but offering a genuinely better solution for greater efficiency. With the advent of such powerful technologies, it is only a matter of time before their implementation becomes mainstream. As such, early adoption is incentivized by the potential for a distinct advantage over those slower to innovate. Between better outcomes in the operating room, modalities for off-site training, a faster and more tangible sales process, and mixed reality engagement for interaction at a distance, the benefits to this technology are exceptional. Reach out to us at CGS and Teamwork AR to learn more about how these technologies can better serve your industry.