Visualizing the Brain in 3D

The brain is more challenging to understand than heart, kidney, skin, and other organs¹. Unfortunately, its complex interconnecting structures and interactions have no close similarity to common, everyday experiences or models². Cutting open a brain reveals globs of gray and white with very little of its amazing complexity or vast potential. Medical-related learning, such as the pathways and circuits relevant to addiction, are certainly not obvious or easy to understand.

Students need to develop an internalized, spatially relevant 3D map of brain structures and their connections ³. Poor 3D comprehension can lead to errors, ⁴ misunderstandings, or doubt that a scientific basis of a brain-based disorder exists. To better understand student needs, we surveyed 36 students and 47% noted that health professional school introduced a new challenge in learning 3D structures, while 50% described it as initially very difficult.

Currently, students use 2D diagrams, dissections⁵, radiological scans ^6,7, textual descriptions⁸, 3D visualizations represented on a 2D screen⁹, glossaries ^10,11 and slides^12–14 to visualize brain neuroanatomy. 3D neuroanatomic renders delivered on a 2D screen can be partially successful^15–17in providing spatial relationship abilities ^18–20 compared to standard 2D objects^1,21,22. But these tools still do not reach the full potential of 3D immersive environments to enhance spatial skills and develop 3D maps. Learners need a powerful tool to visualize the brain and time-efficiently build an internal model of the brain. One does not exist. We asked 36 students about existing educational tools and forty-seven percent (47%) of the respondents to the survey disagreed that existing tools were adequate and 80% agreed that they wished there were better tools to understand 3D structures.

Enabling Technology

Virtual reality training engages the learner in environments not feasible in the real world, such as immersion inside body systems²³ used in surgery training ²⁴.

Older VR solutions²⁵ were bulky and expensive. Prior solutions using head-mounted displays suffered from a lag between head movement and visual display, leading to a lack of presence²⁶[per scales to measure presence^27,28] and often to severe motion-sickness side effects.²⁹ Room-sized technologies such as Cave2³⁰ and C6³¹ could produce an amazing experience ^32–34 at an equally amazing cost.

Oculus Rift™ VR renewed enthusiasm for consumer VR technology.³⁵ Oculus took advantage of Moore’s law alongside improvements in high-resolution small screens utilized in the latest generation of iOS and Android smartphones. For the field of VR, the ability to build a lightweight VR device connected to a computer ³⁶ was revolutionary. The Oculus VR experience compared favorably³⁷ to the room-sized C6 simulator ³¹, a multi-million dollar room with displays surrounding the user. The 3D stereoscopic VR world of today’s headsets responds promptly to head movements, fills the entire field of vision, and smoothly generates images that create a convincing alternate reality where the user can look up, down, and behind them.

These headsets allow a near-perfect representation of reality in terms of low lag and a full field of view.³⁸ The realism that is possible with the device evokes “real world” responses³⁹ that one simply does not see when folks watch a video or 3D images displayed on a 2D screen.

The technological potential application of immersive virtual reality to medicine is growing rapidly⁴⁰. Headset VR technology has already been used to explore pain control⁴¹, surgical repair of hernia⁴², and similarly complicated objects for engineering training⁴³.

The NIH BRAIN Initiative and the component Human Connectome Project have highlighted technologies that depict the brain’s gray and white globs in stunning images. Yet, as glorious as these images are, they are still displayed on a 2D screen. Although diffusion tensor MRI and images produced by tractography are an improvement on depictions of the brain based on standard MRI and diagrams based on dissection and other imaging techniques, they are still not immersive.

3D Immersive Learning Environment

In medical education, 3D graphics delivered on a 2D display are more efficient than textbooks ⁴⁴ and standard 2D objects ^1,21,22. However, full immersion in a VR environment can enhance learning even more. Immersion involves a complex interaction between the psychological experience of feeling “present” within the learning model, representational fidelity, and learner interaction.^45,46 The immersive learning experience provides advantages over routine education or even 3D models displayed on a 2D screen including superior memory, transfer, and motivation.^47–50 Immersive learning allows for a great connection to the materials being presented⁵¹ and a sense of presence [per the Temple Presence Inventory.^27,28,46].

Spatial Memory

Spatial memory is essential to survival and in the past helped our species understand things like caves, hunting grounds, and collections of people.^17,52 In modern-day, when faced with something new, we quickly and efficiently create internal 3D models simply by looking around and walking through an environment.^53–55

Graphical representations (in 3D models delivered on a 2D display) enhance understanding of complex spatial relations associated with neuroanatomy when compared to 2D representations.^1,21,56,57 Early work in the spatial memory field explained that construction of spatial memory and cognitive maps depends on consistent input from a variety of information for both rewarded (goal-directed) and non-rewarded (exploratory) purposes and that virtual reality may enhance the effect further^58(p459). Virtual environments do indeed confer spatial knowledge of the components and refine the understanding of how these components interact.²³ Our immersive 3D experience does not require the spatial manipulation skills inherent in visualizing a 3D object on a 2D display¹, and thus is likely to be even more effective than 2D-delivered 3D. The “supernatural” ability to instantaneously transport or zoom in/out of different scales may further enhance spatial learning ^59(p279).

Internalized 3D models stored in spatial memory are strong, potentially explained by the uniqueness or novelty⁶⁰ of the experience. The subsequent spatial map may cause understanding to persist longer than visualization based on images or textual descriptions.

Augmenting Reality

Augmenting reality with labels and guides further improves spatial memory acquisition^16,61 and ensures the learner does not become “lost” within the brain. The background, too, is made to help orient the learner regarding their current perspective.

With cognitive abilities engaged, learners utilize the layout and landmark orientation elements of the VR experience to develop a cognitive map that includes the overall layout, the objects, their meaning, and purpose.^62,63 Such a map organizes, structures, and stores memories of associations and concepts in durable memories that are more easily retrieved.^64,65 The internal map, with its relationships among elements in a space, also translates to a better understanding of relationships among concepts.^66,67

Potential Solution

To support immersive learning of the brain’s structures, we are creating VR Brain Exploration, an immersive 3D brain simulation that future medical professionals explore via the new generation of virtual reality headsets. Students will have a tool to build an internal 3D representation of the brain that yields improved understanding of the brain’s complex structures and the network of neural pathways compared to standard 2D-delivered training. The improved internalized brain map may, in turn, result in increased interest and confidence in the medical basis of addiction and other brain-based illnesses, as well as the effect of substances on brain circuits and neurotransmitters. An improved map will aid the understanding of new knowledge of the brain as it arises. Our work can also guide the exploration of immersive VR into additional medical topics and target different learning styles.

Our proposed 3D virtual reality VR Brain Exploration confers understanding of the complex neuroanatomy and neurophysiology of the brain and its processes and helps students develop a complete model of the brain. We look forward to developing this immersive VR experience so that learners can go into the brain and get a real understanding of anatomy from the inside out.

Image Credits

1. MRI T2 Brain axial image: GNU Free Documentation License
2. ClinicalTools / HealthImpactStudio
3. DTI Brain Tractographic Image Set per Aaron Filler, MD, PhD, GNU Free Documentation License

References