Video games typically utilize vision and audio separately, with both mechanics sticking to their own sides. However, some games blur the line, enhancing the vision with audio, and the audio with vision. Fighting games are one such example where audio cues are just as important as the visual ones.
Then there are those games that take the audio and represent it in a visual format, turning an auditory treat into a visual feast. This is where audio is represented not only as a sound, but as an effect or mechanic on-screen in the form of vision.
Here are some of our favorite examples of games that have visualized audio.
The Last of Us
The first cab off the rank is The Last of Us, Naughty Dog’s critically acclaimed post-apocalyptic title from 2013. In The Last of Us, Joel and Ellie must try and survive against the infected through any means necessary. To do this, one of the skills Joel and Ellie can use is the Listen Mode.
When Listen Mode is activated, Joel and Ellie can see the silhouette of an enemy through walls and objects. Any enemy that is making a noise, moving, or even just standing still and breathing can be detected this way. Normally, games only give this information through audio, but in The Last of Us, this audio is represented visually.
This list wouldn’t be complete if we didn’t mention the titan that was Guitar Hero. Known for letting gamers experience the thrill of rocking out without having to learn an instrument, Guitar Hero was the, excuse the pun, hero of many parties.
It’s a simple visualization, but one that has stuck with gamers for a long time. Each beat of the music, each string on the guitar, though played through the audio track, was also represented by a simple button on-screen.
As these buttons flew by, it was up to you to hit them in time. Even if you didn’t have an ear for music, Guitar Hero gave you the visual clues to help you become a rock god.
The Legend of Zelda: Breath of the Wild
The Legend of Zelda: Breath of the Wild brought us back to Hyrule, but it’s a different place than when we last visited. With bigger mountains, deeper valleys, and challenging dungeons, Hyrule has evolved into something spectacular.
One of the newest additions in Breath of the Wild is Link’s ability to sneak. Remaining silent and slinking up on an enemy will grant Link the chance to score a critical attack. Thankfully, there’s a way to tell how noisy Link is being.
The way Breath of the Wild shows Link’s noise is through a Noise Meter. This little circle, with its purple line, spikes and jitters whenever Link is running around or attacking, but drop to a crouch or stand still and the line will become calm.
This visual representation of Link’s audio is helpful, as it lets you judge at a glance how much noise Link is making. Sometimes all you want to do is avoid fighting enemies and get to loot they might be protecting, and with the Noise Meter showing you sound, it makes the task all the more easier.
Every Extend Extra Extreme
Another game where music plays a huge role in its mechanics is an old Xbox 360 game called Every Extend Extra Extreme, also known as E4. This game was originally released as an Arcade Game where players controlled a ship that could set off explosions, destroying enemy shapes, causing a chain reaction to bloom across the screen.
E4 was set to a thumping soundtrack that increased in beats-per-minute the longer you survived, and with this increase in BPM came more intense groupings of enemies for you to explode. As these explosions occurred, they would add their own sounds to the music, further increasing the rave. You were also rewarded with higher multipliers for setting off an explosion on a beat.
All these elements mingled together to create a visual banquette of sounds and explosions, like looking at a radial musical soundwave made up of tiny supernovas. Each beat of the music was met with a visual pulse that your reactions would recognize as a sign to set off another explosion. Even the musically challenged could match what they saw on screen to the music they could hear.
In Perception, players take on the role of Cassie, a blind woman, who seeks to uncover the secrets of a large – and haunted – mansion in Gloucester, Massachusetts. As far as games that represent audio in a visual format, Perception is unprecedented.
Cassie, being completely blind, cannot see the world as someone with working eyesight would, instead, she uses echolocation to sense the world around her. Echolocation is the ability to make or hear a noise and hear how it resonates around the environment, bouncing off surfaces. This would indicate how far an object is from the source of the sound.
Players are able to tap Cassie’s cane to send out a shockwave of sound that hits walls, beams, tables, and other objects, painting a brief picture of the room she inhabits. This is one of the most unique ways of showing how sound works and how someone with the ability to use echolocation might be able to “see” the world around them.
Each noise in the environment also creates a splash of color that washes over everything before disappearing. The twist is that, in order to “see”, you need to make noise, but making noise attracts the Presence, the monster haunting the mansion.
Crypt of the Necrodancer
In a similar fashion to Every Extend Extra Extreme, Crypt of the Necrodancer also utilizes music to affect the gameplay. As a rogue-like dungeon explorer, Crypt of the Necrodancer is different each time you play, with every room you enter populated by different creatures and monsters you must defeat.
Interestingly, the game visualises the music you hear by rewarding you if you attack on the beat. So the music doesn’t simply add atmosphere, it plays a direct hand in your movement. Attacking on the beat creates this visual flow that is lifted by the music, with each successful strike and attack being rewarded with attack bonuses.
As the game progresses, the music becomes more intense, requiring more effort to match the beat and reap the rewards. Though it’s a simple idea, the execution creates a visually impressive performance as you character seemingly dances with the soundtrack.
Friday the 13th – Jason can detect noise
Finally, the latest online craze is Friday the 13th: The Game, where one person gets to be Jason while the remaining seven players fill the shoes of scared Counselors trying to escape. Audio is harnessed in multiple ways in Friday the 13th: The game, with Jason and the Counselors experiencing the audio through different visual expressions.
For Jason, any time a Counselor makes a sound, it’s as if a beacon has been set off or a sonar shot. A white circle pulses on the screen, marking the direction the noise originated. Though you might be too far to hear it yourself, the noise is translated visually through this manner.
When Counselors become scared, they express their fear through cries, whimpers, and shrieks, but to Jason, these sounds of fear literally paint the town red. Each Counselor’s fear is a like a bright neon sign, pointing Jason directly to them.
As for the Counselors, whenever Jason teleports or shifts around the map, they might not be able to hear where he lands, but thankfully there is a visual representation associated with this. Whenever Jason gets close enough to be heard after relocating, the screen will fracture like an old VHS tape. Sometimes a Counselor will be too busy planning with the other Counselors to hear Jason nearby, so the visualization of this audio acts as a helpful prod!
Though games typically keep their audio and visuals separate, a few games offer a visual representation of audio. Whether it’s a noise bar, increased character movements, or colors pulsing on-screen, these games like to give you more tools to work with! What games can you think of that show their audio in a visual way?