The head of US gaming company Valve Corporation says a future is fast approaching where video games will use data from people's brain signals to adjust the experience they get — and even a future where people's minds can be adjusted by computers.
Gabe Newell spoke to 1 NEWS about the future of brain computer interfaces (BCIs) — an area he and other Valve staff have studied for several years now — and talked about how Valve is working to put BCIs to use in the gaming sector.
Newell admits some of the ideas may seem incredible, and said some of the discussions he's having around BCIs are "indistinguishable from science fiction" — but according to him, game developers would be making a mistake by not investigating BCIs within the short-term future.
To help them to do that, Newell said Valve is currently working on an open-source BCI software project, allowing developers to begin to interpret the signals being read from people's brains using hardware like modified VR (virtual reality) helmets.
"We're working on an open source project so that everybody can have high-resolution [brain signal] read technologies built into headsets, in a bunch of different modalities," Newell said.
Valve has been working with OpenBCI headsets.
OpenBCI unveiled a headset design in November called Galea, designed to work alongside VR headsets like Valve's Index.
"If you're a software developer in 2022 who doesn't have one of these in your test lab, you're making a silly mistake," Newell said.
"Software developers for interactive experience[s] — you'll be absolutely using one of these modified VR head straps to be doing that routinely — simply because there's too much useful data."
That data will generally consist of readings from the player's body and brain, which can be used to tell if the player is excited, surprised, sad, bored, amused and afraid, among other emotions.
The readings can be used by developers to improve immersion and personalise what happens during games — like turning up the difficulty a bit if the system realises the player is getting bored.
Aside from just reading people's brain signals, Newell also discussed the near-future reality of being able to write signals to people's minds — to change how they're feeling or deliver better-than-real visuals in games.
He said BCIs will lead to gaming experiences far better than a player could get through their "meat peripherals" — as in, their eyes and ears.
"You're used to experiencing the world through eyes," Newell said, "but eyes were created by this low-cost bidder that didn't care about failure rates and RMAs, and if it got broken there was no way to repair anything effectively, which totally makes sense from an evolutionary perspective, but is not at all reflective of consumer preferences.
"So the visual experience, the visual fidelity we'll be able to create — the real world will stop being the metric that we apply to the best possible visual fidelity.
"The real world will seem flat, colourless, blurry compared to the experiences you'll be able to create in people's brains.
"Where it gets weird is when who you are becomes editable through a BCI," Newell said.
At the moment, people accept their feelings are just how they feel — but Newell says BCIs will soon allow the editing of these feelings digitally, which could be as easy as using an app.
"One of the early applications I expect we'll see is improved sleep — sleep will become an app that you run where you say, 'Oh, I need this much sleep, I need this much REM,'" he said.
Another benefit could be the reduction or total removal of unwanted feelings or conditions from the brain, for therapeutic reasons.
Some people who use VR headsets suffer from vertigo due to the mismatch between what they're seeing and what their body is sensing — but Newell said that, right now, BCIs have advanced to a point where that vertigo could be suppressed artificially, and that "it's more of a certification issue than it is a scientific issue".
Despite several achievable applications for BCIs, Newell said he's hesitant to pause Valve's progress and turn one into a consumer product, when the speed of research is so high.
"The rate at which we're learning stuff is so fast that you don't want to prematurely say, 'OK, let's just lock everything down and build a product and go through all the approval processes, when six months from now, we'll have something that would have enabled a bunch of other features."
Valve is also contributing to projects developing synthetic body parts in exchange for expertise.
"It turns out game engines are really useful, because they simulate a lot of the information you need in order to create a simulated hand for people," Newell said.
"You can iterate software faster than you can iterate a prosthetic, so we give them a framework in which they can do research and work with patients."
In case you were wondering, a Valve-brand cybernetic limb is probably off the table for now.
"Valve is not in the business of creating virtual prosthetics for people," Newell said.
"This is what we're contributing to this particular research project, and because of that we get access to leaders in the neuroscience field who teach us a lot about the neuroscience side."
On the subject of prostheses, Newell said there are some interesting questions to answer around developing artificial limbs.
"As soon as you do that, they say, 'Oh, well can we give people a tentacle?' Our brains were never designed to have tentacles, but it turns out brains are really flexible."
Neuroplasticity is a term which refers to the ability of our brains to re-learn how to operate the body when something changes.
Neuroplasticity also takes place when we learn to use tools — an example would be a builder using a hammer for so many years that it feels like a natural extension of their body.
Newell gave a personal example of neuroplasticity. He underwent two cornea transplants in 2006/07, and after the surgery a ghost image of some objects was produced in the field of vision between his eyes due to the change in colour perception.
The condition went away in a few weeks as his brain readjusted to the new input being received from his eyes.
So the future of BCIs sounds interesting, but what about the darker side?
Newell briefly mentioned that BCIs could potentially be used to cause people physical pain — even pain beyond their physical body.
"You could make people think they [are] hurt by injuring their tool, which is a complicated topic in and of itself," he said.
Game developers might harness that function to make a player feel the pain of the character they are playing as when they are injured — perhaps to a lesser degree.
Like any other form of technology, Newell says there's a degree of trust to using it, and that not everyone will feel comfortable with connecting their brain to a computer.
He says no one will be forced to do anything they don't want to do, and that people will likely follow others if they have good experiences, likening BCI technology to cellular phones.
"People are going to decide for themselves if they want to do it. Nobody makes people use a phone," Newell said.
"I'm not saying that everybody is going to love and insist that they have a brain computer interface. I'm just saying each person is going to decide for themselves whether or not there's an interesting combination of feature, functionality and price."
There will also be a heavy onus on developers to ensure their BCI products are rigorously tested and are secure from breaches.
"There's nothing magical about these systems that make them less vulnerable to viruses or things like that than other computer systems," Newell said.
"Right now, you have to trust all your financial data, all of your personal information to your technology infrastructure, and if the people who build those people do a bad job of it, they'll drive consumer acceptance off a cliff.
"Nobody wants to say, 'Oh, remember Bob? Remember when Bob got hacked by the Russian malware? That sucked - is he still running naked through the forests?' or whatever. So yeah, people are going to have to have a lot of confidence that these are secure systems that don't have long-term health risks."