Video games are not only the most popular form of entertainment today, they are also the most dynamic, with storytelling conventions, game mechanics, and user experiences (UX) all evolving with ongoing technological innovation. Adeia is a pioneer in interactive media and human-machine interfaces (HMI). Our team of scientists and engineers have been actively building on that legacy, inventing new ways to interact with game content and improve the game-playing experience.

Recent examples include…

• The development of more sophisticated techniques to not only correct input errors, but to optimize gameplay inputs – think of it as auto-correct, but for gaming;
• Adopting electromyography (EMG) sensors for input, leading to faster reaction times when gaming;
• Diegetic messaging – minimizing the intrusiveness of real-world messages by embedding them in the game in ways that appear intrinsic to the game.

Minimizing gameplay errors

Mashing the wrong button is an existential issue for video game players. We propose a new error correction methodology that will greatly minimize player frustration, including lowering barriers for less experienced players.

The industry has already developed several ways to prevent negative game results caused by motor error – pushing the wrong button – the most common of player mistakes. These existing techniques are predicated on hardware configuration and input probability. They compare actual inputs with expected inputs. For example, it is easy for players to accidentally push the button situated next to the one they intended to push. The error-correcting mechanism identifies the mistake and activates the input that would be expected given the circumstances.

This approach is undeniably useful, but it has become insufficient as games have become more complex, and as game controllers have added more inputs and more types of inputs – buttons, levers, touchpads, and even motion-sensing. Where once any button was probably a toggle, now any given button, lever, or touchpad might do something different when halfway depressed than it does when fully depressed, or when double-clicked, or when used in conjunction with other buttons.

These increasingly complex control schemes naturally take longer to learn and master. To compound the issue, control schemes vary from one game to the next.

As a result, new players are increasingly reluctant to invest time learning a new control scheme for a game they don’t even know if they like yet. This tends to lead to gameplay errors, and when players experience too many errors, they’re apt to churn – to not return to the game.

Exacerbating the situation, game mechanics tend to get more complex throughout a game, and less forgiving of errors, which can make it hard even for players who are adept but play too sporadically to develop the mastery required to complete modern games.

We are developing techniques that go beyond button-selection probabilities and factor in actual gameplay dynamics. The processes involved include:

• Identifying player errors by comparing player input to a list of potential inputs (which could be defined contextually and/or manually by developers).
• Scoring each potential input, including the player input, by its relevance to game context as well as potential benefit to the player.

If a player input falls below a threshold relevance or benefit, the mechanism would automatically select a more suitable input and activate it.

This method not only addresses mechanical input errors, it also provides subtle assists to gameplay. As a practical matter, these mechanisms would kick in most often for players most likely to need and appreciate the support: new gamers, casual gamers, and players new to any particular game. That said, they would still be a useful, if infrequent, boon for seasoned gamers too.

Electromyographic input

Experienced gamers prize any advantage they can get. We are working to enable the use of EMG sensors with modern game systems, which will lead to a reduction in reaction times significant enough to make a difference between in-game success and failure.

It takes a certain amount of time between a gamer pressing a button and the result of that button press being shown on screen. The electronics industry consequently works hard to make each new generation of game processors and displays progressively faster, and to improve communications network performance, all to minimize the latency, but such improvements are coming in progressively smaller time increments and bought at increasingly higher cost.

Nor is there much that can be done to improve human reaction times, the time it takes for an impulse formed in the brain to manifest as a physical action. The time between impulse and, for example, a button push, is roughly 250 milliseconds.

But it turns out you don’t have to shorten human reaction times. We’ve developed an alternative that can shave tens of milliseconds off that 250ms figure – a significant amount in the context of gaming. It relies not on speeding up human reaction times, but by measuring reaction impulses early.

EMG sensors measure the electrical impulses produced during muscle contractions. They provide a compact and low-powered way to detect the muscle activity that precedes motor actions such as a mouse click or button press. EMG is upstream of behavior and therefore provides a faster measure of user intent. When directly comparing mouse clicks to EMG, researchers were able to detect the initiation of a mouse click with EMG 70ms faster than the actual click was registered.

Adeia researchers are developing a methodology and techniques that rely on EMG sensing, alone or in combination with other sensors (proximity sensors, force sensors measuring grip) to anticipate which button the player intends to push and activate it.

EMG has yet to be adopted for gaming, but several prominent consumer electronics companies (Apple among them) are performing R&D on EMG and its potential applications. Gamers, eager for any advantage, are notably early adopters of new technology.  

Diegetic messaging

Games are designed to be immersive. But we all still need to keep track of what’s going on in real life (IRL). A message from the real world (e.g., a text, a DM, a notification) can impinge on the sense of immersion, however, crimping enjoyment of the game. Furthermore, a player fully immersed in a game might miss an important notice.

So, we devised methods of injecting messaging directly into the game, not as intrusive pop-up windows or banners, but as a seemingly intrinsic part of the game – to make it diegetic. [A diegetic interface is one that naturally fits in with the game world, rather than being artificially inserted. For example, a compass indicator panel showing direction at the top of the screen is not diegetic, but the player's character raising their hand to check a compass they are holding in-game is diegetic.]

Using the approach we developed, it is possible to deliver messages in a manner that seems natural to the game. It is possible, for example, to have a non-player character (NPC) convey the incoming message. Did Door Dash just deliver dinner IRL? Have an NPC announce: “Halt! I propose we suspend our hostilities. Let us feast now and return to battle with bellies full and vigor renewed.”

Adeia and gaming

We love gaming, and we love that we are in a position to a) develop technologies that can improve the gaming experience, and b) work with other companies in the gaming space to help realize these and the many other gaming inventions we are working on. Call us. Let us know if we can help improve your games, your game platforms, or the gameplay experience for your customers.

‍