What Happens to Your Brain in Virtual Reality?

A man and a woman wear virtual reality headsets while gaming.

In late 2017, when Professor Robert Duncan (GC/York; Psychology, Biology/Behavioral Sciences) first began trying to design a virtual reality video game, he realized that no one really understood on a deep level how virtual reality worked.
He knew that game companies had a decent understanding of perception, and of the basic physiological mechanisms behind it. But what about presence? That’s a term that gets used a lot in virtual reality, and is generally defined as the feeling of being located in a particular world.
Yet the concept of presence is much more complicated and difficult to describe, Duncan realized. Duncan is a visual neuroscientist who spent much of his career developing new functional magnetic resonance imaging (fMRI) techniques. He decided to take a closer look at the neural mechanisms that are responsible for what psychologists call bodily self-consciousness.
Bodily self-consciousness is formed by two components. One is embodiment: the perception that you inhabit some kind of corporeal form. This form, however, can be abstract — maybe you feel like a fly on a wall, or a plume of smoke — and it can also be manipulated through sensory input. It is an illusion: You believe that you are in your body only because all of your senses are telling you that. (A quick note for anyone who has had an out-of-body experience: It was probably the result of inaccurate sensory input, Duncan says.)
The second component of bodily self-consciousness is how scientists define presence: the illusion of being located somewhere. Joining these components together, bodily self-consciousness means that you are both embodied and present somewhere. Duncan’s model posits that embodiment is a prerequisite for presence. “It was pure logic that made me come to this conclusion: You can’t be somewhere unless you are something,” he says. “Because being something is a necessary part of being somewhere, it’s been very difficult for people to tease these two constructs apart.”
To test this idea, Duncan and a doctoral student, Evan A. Owens (Psychology), designed a series of experiments in which they manipulate presence and embodiment, separately, in a virtual reality game. The game is played within an MRI scanner, so the researchers can see which neural networks are responsible for the two components.
So far, it has been challenging. As Duncan notes: “How do you design a game that focuses on movement when you can’t move in an MRI?” To get around this problem, he plans to track subtle movements of the player’s feet, which will be just outside of the scanner opening. Another challenge: Because metal can’t be used within an MRI environment, Duncan had to design his own virtual reality presentation system using entirely non-metal parts.
Virtual reality might seem a long way from Duncan’s most cited work: researching neuro-degeneration in the visual system from glaucoma. Yet a short time before he joined CUNY, he had an experience at a game developers’ conference that inspired him. “I saw this great intersection of designers, artists, technologists, and programmers,” he says. “There was a lot of positive energy.” Duncan decided he would develop video games for educational purposes.
He opened a lab in which students could make games, which later expanded into a larger initiative called Transformative Games. “I’m not trying to create game developers out of these students,” says Duncan, who was also the director of undergraduate research at York College. “It’s basically a course in ‘how to science.’ By designing a game, they can learn more about whatever interests them.”
Duncan didn’t consider himself a gamer before starting this research. “There’s a joke in game-design world: You can make games, play games, and have a life — but you can only pick two of the three,” he says. He points out, though, that most of us have played far more games than we realize. “Gaming is infused into our culture and into my generation, in particular” — a generation that grew up on the first wave of video games. And as he points out, even our analog lives are full of games. “People forget about board games, and field games like hide-and-go-seek, and sports,” he says. “Social dance is a game. Jazz music is a game.”
Perhaps most important: We learn from games. Duncan’s hope, as he moves forward with his virtual reality project, is that his game might reveal answers to a great mystery. “Your body is an avatar for your mind,” he says, “and we hope to find out how this works.”

Submitted on: APR 11, 2019

Category: Biology | Faculty | General GC News | Psychology