VisiSonics: Rendering Spatialized 3D Audio

The goal of AR and VR is to immerse the user and trick a person into thinking that they are experiencing a different reality. However, in most cases, audio is a neglected part of the AR/VR experience. This causes the user to fall out of the experience, or get sick. Humans use sound to provide them with full sphere awareness of the objects, geometry of the space and the materials making it. If the audio is rendered by a model that is not aware of these, they do not get immersed. Similarly, if the audio is captured in a way that does not retain this information, the audio reproduced is wrong. As Ramani Duraiswami, Founder and CEO of VisiSonics, explains, “In real life, you would pay attention to the action because there would be sound coming from the source of the action, even if it is behind you. But because the sound is not directional in a standard VR presentation, people often focus on the wrong things, and destroy the intent of the story teller.” VisiSonics has leaped onto the Augmented Reality (AR) and Virtual Reality (VR) audio scene, delivering incredible and accurate 3D audio based on sound scientific principles. “Our product portfolio ensures audio fidelity is consistent with the visual in gaming, in entertainment, and in other applications, enabling people to stay longer in the AR and VR experience without experiencing fatigue or sickness,” affirms Duraiswami.

VisiSonics technology is based on a scientific model of the propagation of sound from the source to the ears of the listener. The sound from the source travels in many different directions and interacts with the environment, and the direct sound and many copies arrive at the listener’s ears. Each of the copies interacts with the body of the listener, scatter from it, and finally reach the ear canal. The brain then processes the sound and identifies the sound source, its location, and the content. All of these change as the listener or the sources move through the environment.

Interestingly, humans have different shapes and sizes, and their ears vary significantly. VisiSonics can personalize the audio to each individual by personalizing the Head-Related Transfer Function (HRTF), which is a measurement of how the sound received at the ear canal is different from the sound that would have been received in the absence of the listener at the same point in space.