Spatial Awareness: How sound design can enrich immersive experience

Unlike “traditional” interactive media, immersion is crucial when designing VR and AR experiences. Since there is a higher degree of involvement in immersive media, the users or guests subconsciously expect to have a more exclusive, personal and intimate connection between themselves and the world they are in. Any feature that disconnects them may lead to an immersion break and make them feel the world they are in is fake which will hugely downgrade the quality of their experience.

Audio is a one of the key elements that can establish or destroy immersion in an experience in our subconscious minds. In order to create a more immersive audio experience, we have to make sure that it stays as close to our perceptual hearing experience in our daily life as possible. To this end, we have focused on two features in our design, spatial audio features and variability of the soundscape.

Spatial Audio Feature: 

We introduced spatial audio features to make sure that when the users move around in the experience, the audio would behave like an actual audio source. In other words, we designed the audio features to correlate directly with where a user is in the virtual space. There are several Spatial Audio Plug-ins with a great HRTF setup that helped us to realize such features. Here is a video example to compare the audio with and without this spatial audio feature. 

Variability of the Soundscape: 

When we are generating or perceiving noises in reality, it is exceedingly unlikely to create or hear the exact same sounds even when an action is repeated in an identical manner, and thus our subconscious preferentially expects to hear similar yet different sounds each time. As a result, if identical sounds are repeated across actions and experiences, users may lose their suspension of disbelief and instead feel they are in a simulated and programmed world. In order to increase variability in the game’s sound experiences, for each repetitive sound we designed an array of audio files each containing several subtle variants, which the game system randomly picks from and plays each time that sound type is triggered. On top of that, we also added on a pitch variation, which creates another layer of randomization.  

By these methods outlined above, we feel we have achieved a dynamic playspace that allows our young game players to suspend their disbelief and truly enter the augmented reality created by Little Pizzeria. It is in these small ways that we create a learning environment for children that feels fun rather than pedantic, as research has shown again and again that the deepest and longest term learning comes from immersion and high engagement scenarios.