GE Healthcare Systems
Mixed reality helps GE Healthcare train engineers to service complex equipment
GE Healthcare Systems - Mixed Reality HoloLens 2
GE Healthcare manufactures many of its most complex medical equipment, such as CT and MRI scanners, in China. The maintenance of these sophisticated equipment requires that service engineers undergo an intensive training and certification process. Working with Microsoft MRPP partner Adjacentech Corporation, GE Healthcare developed a mixed-reality training system that uses HoloLens 2 to improve training efficiency and expedite knowledge retention.
GE Healthcare in China manufactures costly and sophisticated medical equipment, such as CT (Computerized Tomography) and MRI (Magnetic Resonance Imaging) scanners, which make modern medical diagnostics possible. These complex devices, which are used by healthcare facilities around the world, require highly trained engineers for maintenance and repair. GE Healthcare has been exploring new training models and applying new technology to improve training effectiveness and efficiency.
“Engineer technical competency is the key to offering good service to the customer,” says Hui Gao, Head of Technical Operation, GE Healthcare China Service. “Training is one of the most important ways to build competency. The traditional methods of training proved to have some limitations when it comes to helping the engineers master the complex medical equipment, so we were very willing to try any new technology to improve the training.”
GE Healthcare turned to Microsoft MRPP (Mixed Reality Partner Program) partner Adjacentech Corporation, an expert on mixed reality development, to create a new training approach. “GE Healthcare has more than 1,000 engineers in the field in China. They need to be able to troubleshoot for customers, and if they’re new to the company, they need to learn quickly.” says Keyan Li, Chief Scientist at Adjacentech.
Adjacentech and GE Healthcare worked together to re-imagine training using the Microsoft HoloLens 2. These simulated training modules would give engineers accurate step-by-step guidance, along with holographic 3D visualizations overlaid atop real-world objects, such as medical equipment, to help them understand signal flow or critical maintenance procedures on extremely complex machines, including the Voyager MRI scanner and the Revolution CT scanner.
As an example of a real-world challenge, each Revolution CT scanner has 52 “detector modules” that receive the attenuated X-ray beam. These detector modules are very delicate components. “They’re also expensive,” says Keyan. “And without good training, it’s very easy to damage a part during service procedures.”
The Voyager MRI also has an intricately wired system cabinet, analogous to a stack of servers in an IT center. Even in a hands-on environment, it’s difficult to understand the connections and how different pieces link together. “Sometimes the theory is not easy for students to understand. And the signal flow—how the signal goes from this port to another port to this circuit board to another board—is difficult to show,” explains Gao.
To show GE Healthcare a new approach, the Adjacentech team demonstrated the HoloLens 2’s capabilities and showed GE Healthcare how Azure’s mixed reality components blend the physical and digital worlds. These components include Azure Spatial Anchors, Azure Remote Rendering, and Azure Object Anchors. With these tools, developers can create interactive 3D visualizations with a high degree of detail.
Azure Spatial Anchors enables spatially aware mixed reality solutions. With ASA, GE Healthcare can make virtual content persist in the real world. For example, a technician can always have step-by-step guidance appear next to the pertinent medical device in the classroom every time a new user that puts on a HoloLens 2 stands in that room. Spatial anchors also make it easy for multi-user experiences, so users wearing HoloLens 2 devices in the same room see the same visualizations, regardless of their location in that room.
Azure Object Anchors uses the hardware on the HoloLens 2 to detect objects in the room, then allows high-fidelity 3D objects rendered via Azure Remote Rendering to lock and anchor to those, such as GE’s medical equipment, and overlay it on a physical object.
Azure Remote Rendering leverages the power of the cloud to render complex 3D objects in high-fidelity on a local device such as a HoloLens 2. For GE Healthcare, this gives a HoloLens 2 user detailed 3D visualizations of medical equipment. The user can zoom in or out, manipulate the image, and more.
Azure AI and Cognitive Services provide audible AI-based narration that further helps the GE trainee follow step-by-step training.
After a successful pilot, GE Healthcare proceeded with two use cases. The first was for the Voyager MRI system cabinet training. Here, the training program overlays holographic schematics and animated connection diagrams onto the real cabinet. HoloLens 2 can identify and locate the components and ports in the real system cabinet when a trainee stands in front of it. The overlay, together with AI-based narration, functions as a virtual 1:1 coach to speed up the learning process. The second use case was for detector module replacement in the Revolution CT scanner. With a high-fidelity holographic model rendered in HoloLens 2, training is no longer limited to a dedicated classroom. An engineer can observe and learn the process of detector module replacement anywhere, without any risk of damaging sensitive components.
GE Healthcare has seen substantial benefits from deploying these mixed reality solutions and other new technology within the training. In the two current use cases, an average of four hours of training are saved per trainee. With reduced training hours, trainees get more hands-on lab time to practice, which can further deepen their understanding and application of key knowledge and service procedures. GE Healthcare Training Facilities house more than $20 million in medical equipment in China. The risk of incidental equipment damage can be greatly reduced as more mixed reality and HoloLens 2 solutions are used as a method of hands-on lab training.
GE Healthcare believes it has only scratched the surface of mixed reality’s potential. “This is the future,” says David Xiao, Instructor Manager for GE Healthcare. “I think this technology will have a very broad use in a wide range of industries.”
Adds Wei Meng, MRI Instructor for GE Healthcare, “We believe mixed reality may enhance traditional training and give students a faster, easier way to learn the technology. Virtual reality is typically video-based, but with mixed reality and HoloLens 2, we can provide a true interactive and immersive experience.”