Sound Design Showreel

Game Audio Innovation: Can Games Go That Little Bit Further?

There are many technical advancements that are being studied which will change the way game audio is approached. Ideas such as procedural audio and generative music are areas surrounding this research into the advancement of game audio. They look into creating real time audio assets whilst the game is running, using analysis and physical modeling as a way to recreate the sounds. Although this leads to a list of positives (non-repetition, reduced memory footprint and more control), it also comes with its negatives (CPU usage, difficult to debug). This is why it is currently being looked into, as a way to get around the cons and begin implementing this approach into games. We can see the generative music approach already happening in some games, one example being Spore, where renowned composer Brian Eno has implemented a generative and reactive music system which cooperates with the games events.

Creative Innovation

Although technology is one area that guarantees to innovate the way games operate, I want to discuss how games can be furthered aesthetically and inspire people on an emotional level. Game audio/games themselves have experienced a substantial increase in opportunities due to new developments that allow complete freedom in approach. As new technologies arise, so do new approaches to sound in games. It is evident that many games are leaning towards the Hollywood style of production, and with the industry growing bigger and bigger each year, hiring the London Symphony Orchestra to play the score seems a reasonable idea. But then again, is it entirely necessary? Can games instead go artistically further than those forms of media they base themselves upon?

It takes a lot to create something so sonically innovative in games, that it can change our musical preferences and expectations. Typically, people presume that the more time and funds a sound designer or composer has, the closer the audio will reside to these assumptions. This is usually true in most cases, as more money equals more opportunities to enhance the production (Such as hiring musicians and orchestras). However, I personally like to think opposite of this approach can yield greater results, when approached carefully….

Limbo

Limbo (2010), an arcade game from Playdead studios is renowned for it’s innovative and refreshing approach to both game and audio design. The Audio Director for the game is ‘Martin Stig Anderson’ who prior to Limbo, worked in the field of acousmatic music. His experience in this genre of music is present in the game, he utilises the audio in a unique way to fortify the aesthetics of the game. By only using samples usually associated with sound design, alongside the software on his computer, the unique score for limbo was born.

‘For me it has a much bigger psychological impact when you turn a naturalistic soundscape into abstraction by making your sound effects play as music rather than adding some traditional background music.’

-Martin Stig Anderson

Limbo definitely boasts the idea of less is more. The plot itself is completely ambiguous and left for the player to make sense of; therefore this style of music goes hand in hand with the style of the game.

3mins40 –Example of musical approach

What really strikes me about its sonic approach is how the boundaries between music and sound design have been blurred to create a completely immersive atmosphere, as reiterated by Anderson:

‘What I found interesting in relation to audiovisual media was that soundscape and acousmatic music together embraces the entire continuum between representational and abstract sound, in this way dismissing the traditional dividing line between sound design and music’

-Martin Stig Anderson

This approach undoubtedly comes from principles associated with acousmatic music and when its incorporated into a game that shares similar intentions, it delivers outstanding results. It may be down to the nature of the game itself as to why it is so effective, but maybe the idea of merging the sound design and music into one medium could be taken into other games. In my opinion, it helps the game to feel like a finished product on a very professional scale.

7:12 – Sound Design/music

This approach to a soundtrack can also be seen in many independent and arthouse films, arguably due to the budget for production that they have. The blurring of sound design and music is musical device that can be used to create ambiguity within the on screen visuals. An example of this effect in PI:

Limbo is a recent game, like many others, that has reminded game developers that games have the power and resources to be regarded as artistic or something completely innovative. They can now communicate emotional ideas on a much larger scale than a film, creating a completely new kind of immersive media output that inspires both those who play games and also those who design them.

The Last of Us

Wow, this game looks cool. Liking the number of gameplay possibilities there appears to be, there's another playthrough which shows a different approach to gameplay here. The dialogue and events are different in each version, with the player able to interact with almost anything. It seems that the disk is ready to stream one of many animations/events at all times and it looks good!

Realistic audio and reverb, sounds good!

Adaptive Musical Scores as a Functional Device in Games

In response to 'The Year I Gained Courage to Ignore Video Game Music'

The above article, in a nutshell, discusses one mans personal opinion that suggests music in a video game can be seen as superfluous. The writer of the article talks about how his feelings towards game music changed in 2010, where he opted to multitask during his gaming sessions and alternatively listen to a podcast or his own music on his iPod, instead of the non-diegetic music featured in the game. Red Dead Redemption acts as the basis for discussion and it was the game that triggered realisation. He talked about how RDR's score, not that it is a bad one, can be viewed as 'an optional garnish' to the aesthetic beauty of the game. The writer does however demonstrate his appreciation of how the musical soundtrack enhances a particular moment in the game. The below clip is the moment in which the player first steps foot/hooves on Mexican soil.

The song is 'Far Away' By 'Jose Gonzalez', it is only heard the once in the entire game, and in my opinion is an effective and refreshing approach to utilising modern game soundtracks. The thought that a player may turn the music off in the game, or listen to audio from an alternative source provokes a slight cause for concern amongst game developers. The intended emotions and narrative direction can become lost without the music present to evoke them.


Music as a Functional Device

This idea that music in games is superfluous, can be looked at in an opposing light, in some cases the audio is a functional device to help guide the player through the game. Take the game L.A. Noire as an example, where its music acts as a device in the crime cases to guide the player towards succession of a case. Within the first mission in the game, the HUD displays comments about how the player should use sound to their advantage (around 6 minutes in).

As shown above, the game gives indicators to listen to the audio and all these was devised using FMOD. The chimes are used to indicate an area of interest for the player, so often when they step past a clue in the case, the chimes are triggered. The music also fades when the player has located all of the clues, which demonstrates its functional use in-game to support the narrative. Without the sound, the player would lose a sense of direction within the game and its intentions would appear to be disjunct and unclear.

Games that are based in an open world environment, such as LA Noire, can use music or sound to guide the player through the narrative and supply them with audible information that allows them navigate through the game successfully. The Assassins Creed series is a another prime example in which adaptive music also serves a functional purpose within the game. In a stealth game, where the NPC's state is not always clear, music can act to inform. When the player is roaming the environment freely, they are able to do whatever they like, for example at any time they may choose to attack an NPC. The music or absence thereof can be used to inform the player if they have been spotted by an enemy guard or other game events. Playing the game with the absence of music could easily be done, but the quality of gameplay would significantly decrease as the player isn't receiving the intended information planted by the creators of the game. Also worth mentioning is the multiplayer mode in AC3, the player relies on sound in order to locate their target and pursuer, whispers indicate your pursuer is nearby and heartbeats indicate that your target is close. Although these are more sound effects, they share the same purpose.

To wrap up, if the music in a game acts to aid the gameplay in an informative way, then it is a necessary garnish to include whilst playing. I can however see it from another point of view, I
if it doesn't serve as functionality and its repetitive in its nature, then it could ultimately lead to distraction and a negative disposition to the game. One possible solution to this problem could be the inclusion of a generative music system, so that the music heard is never the same twice.

The Sound of L.A. Noire

Good read about audio implementation in L.A. Noire complete with some picture maps outlining some interesting stuff!

The Sound of L.A Noire

TED Talks: Gaming

Two inspirational and interesting TEDtalks about gaming - Effects on human health and Reward Systems:

1. Jane Mcgonigal on 'The game that can give you 10 extra years of life'. 
www.ted.com/talks/jane_mcgonigal_the_game_that_can_give_you_10_extra_years_of_life.html

2. Tom Chatfield on '7 ways games reward the brain' 
www.ted.com/talks/tom_chatfield_7_ways_games_reward_the_brain.html

Cited from Talk two by Tom Chatfield:

We evolved over hundreds of thousands of years to be satisfied by the world in particular ways; and to be intensely satisfied as a species by learning and problem-solving. Perhaps the most amazing thing about the virtual arenas that games create is that we are now able to reverse-engineer that, and to produce environments that exist expressly to tick our evolutionary boxes and to engage us.

When it comes to games themselves, the “fun first” principle is an absolute: before anything and everything else, a game must be fun. Not everything can be made into a game, though; and it’s simply misleading to think of games as potential solutions to all our ills. So I’ve come up with seven larger ideas for motivating and engaging people on the basis of observing many games’ stunning power as engines of human engagement:

1. Using an experience system. This is something that Jesse Schell has talked about brilliantly in this last year, and that is actually being done in places like Indiana University. Don’t have grades, for example: give students an avatar or a profile that levels up steadily based on things like attendance and performance. Everything should count in some way towards this precisely-measured, steady individual progression: a far more intimate, involving and nuanced way of measuring progress over time than most conventional means. 

2. Multiple long and short-term aims. You break something down into many parallel tasks. You don’t just to say to someone, do 5,000 sums, or 100, or even 50: you create a whole spectrum of larger and smaller objectives that help people take take ownership of their progress, and keep them feeling they are progressing and succeeding – as well as targeting particular sets of skills. 

3. You reward for effort. People should be credited for everything they try and do. Don’t punish failure. Instead, reward and reinforce, and make everything count towards a clear measure of progress. As I’ve said elsewhere, one of the most profound transformations we can learn from games is how to turn the sense that someone has “failed” into the sense that they “haven’t succeeded yet.” 

4. Rapid, clear, frequent feedback. This is absolutely central to all forms of learning and engagement. With many of the most intractable problems in the world today, like global warming and pollution, it can be almost impossible to learn or understand something when consequences and feedback are distant from causes. Showing a clear link between things, and allowing people to experience this experimentally, allows learning to take place: you need to be shown and to experience exactly how an action plays out, what it caused, whether your attempt worked or not. 

5. Uncertainty. This is the real neurological gold mine so far as gaming is concerned. Dopamine elevates when you get a little prize for doing something, but what really lights up the brain is the unexpected reward: the one that couldn’t be predicted. And so the right amount of well-calibrated uncertainty can create intense engagement in all manner of tasks. 

6. Windows of enhanced attention. This is about using the emerging field of neurological modelling to identify those moments when attention and memory are enhanced in the brain by an elevated dopamine level, and putting learning into them – literally dropping the nugget of fact into those few seconds when attention is elevated. It’s early days here, but the potential of the field is vast. 

7. Other people. If games should remind us of one crucial aspect of our evolutionary natures more than any other, it’s that reward is not just money or personal achievement points; and it’s not just solitary individuals slumped in front of screens: it’s the intense validation of doing something in comparison and in collaboration with others. 

Collective engagement can be transformed by the unprecedented laboratory that virtual worlds offer for observing group psychology and motivation; from analyzing Guild structures in games to exploring how the public visibility of participants’ levels of achievement can encourage both competition and collaboration. This is, for me, perhaps the most thrilling area of all.

Sound propagation and the 'Stealth' Genre

'Sound propagation' is the term used to explain the behavior of sound in an environment. When creating a game, we want to accurately reproduce the acoustics of the virtual environment as it would sound in reality. This is no only so that the player can have an understanding as they relate it to the real world, but also to help increase immersion into the game. Accurate reproduction of sound propagation in games consists of multiple aspects, such as Reflection (Reverberation), Refraction (Re-direction of wave) and Attenuation/spatialisation (Where the sound is situated and its drop-off properties).

Reflection and Refraction
See the link below and select 'sound from air to water' to get a basic idea of how it propagates:

www.phys.hawaii.edu/~teb/java/ntnujava/propagation/propagation.html

Essentially, this behavior can be reproduced through using the correct reverb. In UDK, it would require too much CPU to run an accurate propagation model within a map. So to prevent this problem, it provides pre calculated reverb effects in which you can choose from. You can apply one of these presets which seems appropriate using the [Reverb Volume].

Attenuation and spatialisation

Spatiailisation is basically the realisation of the sound in a 3D environment, it allows a sound to emanate from 3D space. A lot of the things we hear in games are off screen due to our field of vision and spatialisation of sounds helps inform us of our surroundings. Attenuation and sometimes filtering when designing games can give a sense of distance from the object to the player. There are many types of distance algorithms to consider when designing attenuation and in games, these are:

• Linear - Unrealistic in the real world, but can be used for crossfading between looped audio and room tones.
• Logarithmic - For sounds more suited to be heard at a closer distance.
• Inverse - Good for loud and distant objects. Quiet at max radius but can be heard from far away.
• LogReverse - Loud at the max radius.
• Natural Sound - Realistic attenuation curve.

In UDK, these algorithms can be selected using the 'Attenuation' node.

Sound Propagation in the stealth genre

Accurate reconstruction of sound propagation is crucial in more genres of gaming than others. One example of this is the stealth genre, in which the player relies on sound for the most part to receive information on NPC whereabouts and the geometry of the environment. 'Dishonored', a recent release from Bethesda Softworks looks into this idea carefully:

http://vimeo.com/45113835#at=0

The sound changes when the player is in another room and the NPC is behind a door, also when the NPC walks in to the same room, the sound informs the player of this through adding clarity to the audio.

Another game which springs to mind is 'Thief', a similar game to Dishonored which used the same idea of accurate and carefully considered sound propagation. It seems essential in stealth games to accurately reproduce the behavior of sound dependent on the geometry of the game.

A final example taken from a fun 2D game called 'Mark of the Ninja' demonstrates the importance of sound within the stealth genre:

Sound is so important to the flow of the game,that it is emphasized through the ability to see it.

Aspects taken from:
The Game Audio Tutorial - (Book by Richard Stevens and Dave Raybould)
The Propagation of Sound - http://www.jhu.edu/virtlab/ray/acoustic.htm
UDK Documentation -http://udn.epicgames.com/Three/SoundCueReference.html