News / 26.09.2018
Tips For Producing Audio Content For VR Applications
Mark Jones, Audio Engineer, REWIND
As with most applications of interactive audio, the objective of sound in virtual reality is to make the experience as immersive as possible through replicating reality. Though stylistic aspects may inform the actual sound design (for example, you are unlikely to use sci-fi-style sounds in a fantasy setting) the behaviour of a sound in the virtual environment must reflect the behaviour of sound in the real world.
Here are my five tips for producing audio content for VR:
TIP 1: Use sound to expand the environment
Most mainstream VR headsets have a very limited field of view, creating a kind of “tunnel vision”, but this can be compensated for by using sound to “expand” the environment beyond what can be seen. This can be accomplished by using multiple sounds at a variety of distances from the player, in any position, and applying the following techniques:
– Separating very distant sounds – such as city traffic or music from a nightclub – to cover the full 360 degrees around the vertical axis. This will help to “fill” the environment.
– Placing nearby subtle ambient sounds just out of sight, such as the dull rumble of machinery on the other side of a wall.
– Placing “exposed” ambient sounds on visible objects, such as static from an untuned radio. These will always be the most prominent ambient sounds, but should not be overly distracting.
TIP 2: Use sound to draw the player’s attention
With a limited visual field, audio plays an enhanced role in allowing the player to locate objects they can’t see directly in front of them. Sound can be used as a powerful ‘attractor,’ grabbing the player’s attention and drawing them towards a certain object or location. For example:
– Loud sounds can prompt the player to immediately look in the direction of the sound, it is not necessary to deafen the player with something that is obnoxiously loud, the sound merely has to stand out.
– Continuous sounds can be used to remind the player of the presence of an object.
– Sounds with exaggerated volume attenuation and doppler effects can emphasise the impression that the object is moving. For example, a vehicle driving towards the player.
TIP 3: Be aware that the player is the centre of the experience
In a typical game displayed on a 2D screen, the in-game player character is not in the same spatial location as the player. In VR, this is not the case, as the player is always in the centre of the experience, and the sound moves around them. Using knowledge of acoustics, we can deliver a convincing, realistic experience by using three core components:
– Realistic attenuation (the drop in volume of a sound over distance). This is not a linear relationship, and most audio engines will include a realistic attenuation curve that is used as a default.
– Binaural spatialisation (a system that simulates the physical properties of a human head in the game’s 3D environment). In the real world, a noise behind a person will sound different from an identical noise in front, binaural spatialisation simulates these properties of human hearing in the virtual environment.
– Reverberation (echoes of a sound as it reflects from the surfaces in a space). A good reverberation simulator will take into consideration the materials these surfaces are made from. For example, a room built from solid stone will echo more than a room covered in fabric.
The major game engines such as Unreal and Unity are typically capable of natively supporting these core features without additional plugins being required, though audio plugins such as Steam Audio or Google Resonance Audio can be used to provide additional functionality such as reflection-based reverberation simulation.
TIP 4: Reinforce a sense of 3D space by emphasising spatial sounds
In order to help reinforce the feeling that the player is in an expansive 3D environment, the spatial nature of sounds can be emphasised through several means:
– Separation: Physically separate similar sounds by placing them beyond the audible range of each other. If there are two objects in the same room that should be producing similar sounds, create distinct variations of the sound. If two sources are producing the same sound (especially if they are widely-separated), the player will lose the ability to precisely locate that sound.
– Isolation: Position dissimilar sounds in proximity, preferably with different key frequencies. This will give each sound enough “acoustic room” in which to be heard.
– Verticality: Binaural audio allows the player to more easily locate sounds in the vertical axis, and as such, it is advisable to position sounds above and below the player. Otherwise, the lack of vertical separation between sounds will give the impression that the player is on an isolated 2D plane.
TIP 5: Ambisonics can be used for background ambiences
In some cases, there may be a need to limit the number of discrete sounds being used in a scene. In this case, ambisonic audio (a multi-channel audio format that changes to reflect the direction the player is facing) can be used. Here are three useful points to consider:
– Pre-rendered background ambient loops can be used to fill out a space, as ambisonic audio files can combine an unlimited number of discrete audio sources in a 3D space into a single file. This approach is best used in situations in which the player is unable to move, as the ambisonic audio cannot compensate for changes in the player’s location relative to the “location” of the audio.
– Spatialisation is not as precise as a mono audio source, so ambisonics aren’t ideal for sounds close to the listener. These sounds could be split from the ambisonic mix and played back separately.
– As ambisonic audio is pre-rendered, it is particularly useful for 360-degree videos or other non-interactive VR content. Ambisonic audio can theoretically combine every sound in an entire experience into a single file – useful for linear media on platforms with limited performance, such as mobile phones.
A final point to be made is that it is important to trust your instincts over any set of rules: if something feels wrong, then it probably is. In one of my projects, I had a sound attached to an object that the player could throw. A realistic attenuation just didn’t feel right, so I made it into an unrealistic linear attenuation, and fixed the issue. This is a creative process that is informed by a knowledge of acoustics; not dictated by it!
We have been working with Great Ormond Street Hospital Children’s Charity (GOSH Charity) over the past few months on an engaging VR tool to highlight the impact of GOSH Charity funding on the hospital. We are delighted to see it launch today, 14th February – GOSH’s 167th birthday! The VR tour brings Great Ormond Street […]
Eye-tracking technology has been around for years in medicine and aerospace but is quickly gaining a new lease of life thanks to VR. The heavy-hitters of the VR world are incorporating eye-tracking capabilities into their future headsets and are projected to grow the eye-tracking market. Market analysts forecast the global eye-tracking devices market to grow at a […]
Virtual reality has been around for a long time. It’s been used commercially by large aerospace and automotive organizations in CAVEs and powerwalls for over 30 years. Now, as VR is becoming more accessible and reliable, it is being rapidly adopted across a whole range of industries, from architecture to tourism. In the current climate […]
Just weeks after British retailers suffered major losses during the festive period, new research shows that 39 percent of Britons would “not care” if all high street stores closed down. The study, carried out by Awin, also revealed that 68 percent of recipients did most of their shopping online. Many cited ease and cheaper prices […]
Sol Rogers Founder/CEO, REWIND – Self-driving cars are an inevitability and will unlock the potential for cars to transform beyond a simple means of transport. In the not too distant future, drivers will find themselves with a great deal of free time in-car. Rather than having to focus on driving, time can be spent on working, […]
REWIND Part Of Winning Audience Of The Future Consortium Set To Redefine Audience Participation In Esports
We are delighted to announce we are part of a winning Audience of the Future consortium – Weavr. Working with ESL; Cybula; Dock 10; Focal Point VR and the University of York, we will be transforming the experiences of millions of fans in esports, but also in traditional sports broadcast and live entertainment more broadly. […]
2018 was a jam-packed year for those working in the immerse industry. We saw hits like Beat Saber and Astro Bot getting people excited about VR content and hardware releases such as Oculus Go and HTC VIVE Pro making it easier for people to try out VR. We’re excited about what’s in store this year. Here […]
It’s not often we take a step back and review. So, as the end of 2018 draws closer, I’ve taken a moment to do just that. From major hardware news to AR taking to the stage in spectacular fashion, 2018 has been jam-packed. Here are my choices for the Top 10 Immersive Moments of 2018 […]
If AR is truly the future of mobile, then WebAR could be the key to unlocking its potential.