VisiSonics
Rendering Spatialized 3D Audio

The goal of AR and VR is to immerse the user and trick a person into thinking that they are experiencing a different reality. However, in most cases, audio is a neglected part of the AR/VR experience. This causes the user to fall out of the experience, or get sick. Humans use sound to provide them with full sphere awareness of the objects, geometry of the space and the materials making it. If the audio is rendered by a model that is not aware of these, they do not get immersed. Similarly, if the audio is captured in a way that does not retain this information, the audio reproduced is wrong. As Ramani Duraiswami, Founder and CEO of VisiSonics, explains, “In real life, you would pay attention to the action because there would be sound coming from the source of the action, even if it is behind you. But because the sound is not directional in a standard VR presentation, people often focus on the wrong things, and destroy the intent of the story teller.” VisiSonics has leaped onto the Augmented Reality (AR) and Virtual Reality (VR) audio scene, delivering incredible and accurate 3D audio based on sound scientific principles. “Our product portfolio ensures audio fidelity is consistent with the visual in gaming, in entertainment, and in other applications, enabling people to stay longer in the AR and VR experience without experiencing fatigue or sickness,” affirms Duraiswami.

VisiSonics technology is based on a scientific model of the propagation of sound from the source to the ears of the listener. The sound from the source travels in many different directions and interacts with the environment, and the direct sound and many copies arrive at the listener’s ears. Each of the copies interacts with the body of the listener, scatter from it, and finally reach the ear canal. The brain then processes the sound and identifies the sound source, its location, and the content. All of these change as the listener or the sources move through the environment.

Interestingly, humans have different shapes and sizes, and their ears vary significantly. VisiSonics can personalize the audio to each individual by personalizing the Head-Related Transfer Function (HRTF), which is a measurement of how the sound received at the ear canal is different from the sound that would have been received in the absence of the listener at the same point in space.

Our product portfolio ensures audio fidelity is consistent with the real world

The HRTF is unique to every person, and this measurement is incorporated into the software, ensuring the 3D audio is fully customized. “We are currently improving our solutions, enabling the user to take measurements within their environment and uploading those to the cloud, along with images of their head and ears to create a customized solution for the end-user,” says Duraiswami.

VisiSonics also offers their RealSpace 3D Audio Camera, which decomposes a sound at any point in space. Using this enables VisiSonics to capture 360-degree video of the scene on which the audio intensity map is overlaid. Customers in automotive and machinery use these to mitigate noise and vibration problems using the device. In addition the devices are used to capture audio for reproduction in VR using object-based representations and high order ambisonics.

VisiSonics’ customers include game and content developers, leading suppliers of VR glasses, and camera makers. One of VisiSonics’ customers is the start-up Vuze Camera, which created an astounding 3D video capture product. The camera had a unique shape, which made sound capture more challenging. VisiSonics designed a solution to capture audio with the use of microphones mounted on the surface of the camera. As a result, Vuze Camera has been able to develop a near-professional-level camera for less than $800, providing incredible value for the customer.

According to Duraiswami, their ultimate goal is to ensure their technology is available to everyone in the form of software packages that are licensed to leading content providers and makers of VR hardware and software.