VR Collab #6 – Understanding the dynamics required for sound design for video games

In this post, I will reveal some insights I have gained through analysing the process of implementing audio in video games.

First of all, it is essential to understand the video game scenario. In terms of means of production, there are two types of multimedia:

• Non-Linear Media: Not presented sequentially or chronologically. In this type of multimedia, the interaction of its user is necessary, like video games or even web pages examples of this.
• Linear media: in this type of multimedia, information is retained or observed in a continuous sequence. It is not a format in which one can interact. Typically, these presentations begin at a predetermined point in the same way they end. A clear example of linear media is a Powerpoint presentation or a film.

So, where does audio implementation come in? A practical concept called 3D emission is the basis of all sonic magic. Basically, the sound designer is presented with a world where he has to sound. He has to geolocate several small speakers with different, programmable sounds. These speakers are called 3D emitters. For the emitters to be activated, you need an “audio listener”. By audio listener, you mean a virtual pair of ears that picks up the sounds from 2D and 3D emitters. What is the difference between 2D and 3D transmitters?

2D: the audio output comes from the headphones/audio listener. It means no spatialisation of sound and no geolocation in XYZ.

3D: means that volume and panning are modified depending on the distance of the audio listener from the transmitter.

Room tones, ambiences, and music are usually programmed with 2D sounds. However, the creative point of these two parallels is the possibility of transitioning from 2D to 3D. A classic example of this type of transition can be found in the mythical game GTA V (and other versions). When a player breaks into a car, before getting into it, you can hear the sound of the radio spatialised. But when the player gets into the car and starts driving it, the radio is programmed with a 2D sound.

How do you trigger a sound?

There are several ways to do this:

• By pressing a button
• Collision
• Values (numbers)
• Animation notifiers
• Proximity

In this way, the sound designer has to find a way to reach the programmer to achieve his goals since they run different languages in their development processes.

According to Sam Hayhurst, game programmer, the first thing that unravels in conjunction with the sound designer is “what are we trying to achieve together?” “Everything I do later on is shared via google docs, where I get feedback from part of them (sound designers).” In this document, the information is written as follows: when x happens, sound y is played. Hayhurst also mentions that active dialogue between the two is necessary and avoids group chats as much as possible, which can be a slowing factor in the audio programming process.

Succinctly here’s what a sound designer should ensure when collaborating with a programmer:

1. Explain precisely and in simple terms what is intended concerning the sound event.
2. Make it clear when you want to make changes to specific audio.
3. Clarify whether it’s a prototype that might be thrown away or if it’s a feature that’s here to stay.
4. Not being afraid of asking.
5. Speak face to face or call because lots of information can be lost in a text
6. Take the time to get to know each other and build a stronger working relationship.
7. Don’t tell the programmer how to do their job.

How do sounds get into games?

To put sounds into a game, we have two options: either importing them directly into the platform (unity, unreal engine, etc.) or through a tool called “audio middleware”. Then we create “audio events”, also called “sound cues”, and put the sounds we want in them. These events act as containers, and these can have an unlimited number of sounds. However, there are playback instructions. Imagine the following scenario: A character has 20 different step sounds, and we as sound designers can define if the playback is random, we can modulate the pitch and, also, the volume. But this is just the surface of it. You can also modify the following parameters: obstruction/occlusion; rooms/portals; attenuation; switches; states; RTPCs.

So. What is the difference between using audio middleware and integrating sounds directly into the engine?

Middleware tools like Fmod and Wwise are 3rd party tools which sit between the game engine and the audio hardware. Learning audio middleware is a daunting task, but understanding the basics is enough to understand the rest of the features.

There is usually the role of audio programmers in bigger productions, which they’re underrated figures. Game engines like Unity or Unreal Engine are more limited. However, EPIC GAMES, the developers of Unreal Engine, have put a lot of work into improving audio editing on the platform. Other publishers have their own engines. Ubisoft works with Snowdrop and EA with Frostbite Engine.

Implementation is not the last step in the process of sounding a game. It must be taken into account that:

• No matter how good the sound is, the player will never hear it if it’s not in the game.
• The way the sound will play back usually shapes the sound’s design and the production pipeline.
• Early on, collaborating with the design and programming teams to create audio systems for specific gameplay mechanics will ensure that the audio group can deliver their best work and won’t simply be left behind.