Efficient Volumetric Scene-based audio with ZYLIA 6 Degrees of Freedom solution
What is the difference between Object-based audio (OBA) and Volumetric Scene-based audio (VSBA)?
The most popular method of producing a soundtrack for games is known as Object-based audio. In this technique, the entire audio consists of individual sound assets with metadata describing their relationships and associations. Rendering these sound assets on the user's device means assembling these objects (sound + metadata) to create an overall user experience. The rendering of objects is flexible and responsive to the user, environmental, and platform-specific factors [ref.].
In practice, if an audio designer wants to create an ambient for an adventure in a jungle, he or she needs to use several individual sound objects, for example, the wind rustling through the trees, sounds of wild animals, the sound of a waterfall, the buzzing of mosquitoes, etc. The complexity associated with Object-based renderings increases with the number of sound objects. This means that the more individual objects there are (the more complex the audio scene is) the higher is the usage of the CPU (and hence power consumption) which can be problematic in the case of mobile devices or limitations of the bandwidth during data transmission.
A complementary approach for games is Volumetric Scene-based audio, especially if the goal is to achieve natural behavior of the sound (reflections, diffraction). VSBA is a set of 3D sound technologies based on Higher-Order Ambisonics (HOA), a format for the modeling of 3D audio fields defined on the surface of a sphere. It allows for accurate capturing, efficient delivery, and compelling reproduction of 3D sound fields on any device (headphones, loudspeakers, etc.). VSBA and HOA are deeply interrelated; therefore, these two terms are often used interchangeably. Higher-Order Ambisonics is an ideal format for productions that involve large numbers of audio sources, typically held in many stems. While transmitting all these sources plus meta-information may be prohibitive as OBA, the Volumetric Scene-based approach limits the number of PCM (Pulse-Code Modulation) channels transmitted to the end-user as compact HOA signals [ref.].
ZYLIAs interpolation algorithm for 6DoF 3D audio
Creating a sound ambience for an adventure in a jungle through Volumetric Scene-based audio, can be as simple as taking multiple HOA microphones to the natural environment that produces the desired soundscape and record an entire 360° audio-sphere around devices. The main advantage of this approach is that the complexity of the VSBA rendering will not increase with the number of objects. This is because the source signals are converted to a fixed number of HOA signals, uniquely dependent on the HOA order, and not on the number of objects present in the scene. This is in contrast with OBA, where rendering complexity increases as the number of objects increases. Note that Object-based audio scenes can profit from this advantage by converting them to HOA signals i.e., Volumetric Scene-based audio assets.
To summarizing, the advantages of the Volumetric Scene-based audio approach affecting the CPU and power consumption are:
Zylia 6 Degrees of Freedom Navigable Audio
One of the most innovative and efficient tools for producing Volumetric Scene-based audio is ZYLIA 6 Degrees of Freedom Navigable Audio solution. It is based on several Higher Order Ambisonics microphones which capture large sound-scenes in high resolution, and a set of software for recording, synchronizing signals, converting audio to B-Format, and rendering HOA files. The Renderer can be also used independently from the 6DoF hardware – to create navigable 3D assets for audio game design.
ZYLIA 6 DoF HOA Renderer is a MAX/MSP plugin available for MAC OS and Windows. It allows processing and rendering ZYLIA Navigable 3D Audio content. With this plugin users can playback the synchronized Ambisonics files, change the listener’s position, and interpolate multiple Ambisonics spheres. The plugin is also available for Wwise, allowing developers to use ZYLIA Navigable Audio technology in various game engines.
Watch the comparison between Object-based audio and Volumetric Scene-based audio produced with Zylia 6 Degrees of Freedom Navigable Audio solution. Notice how the 6DoF approach reduces the CPU during sound rendering.
Volumetric Scene-based audio and Higher Order Ambisonics can be used for many different purposes, not only for creating soundtracks for games. This format is very efficient when producing audio for:
#zylia #gameaudio #6dof #objectbased #scenebased #audio #volumetric #gamedevelopment #GameDevelopersConference #GDC2021 #GDC
We are happy to announce the new release of the ZYLIA 6DoF Recording Application in version 1.0.0 for Linux and macOS. This application is a part of the ZYLIA 6DoF Navigable Audio solution (ZYLIA 6DoF VR/AR set). It replaces the command line toolkit for the recording and synchronization process. For your comfort, this application has a graphical user interface, so there is no need to use the command line anymore.
This application offers all features of the ZYLIA 6DoF Recording Toolkit such as:
Additionally, there are added few new features.
Configure you session.
Make the recording.
Synchronize raw audio files.
The command-line application will be also available but will not be further developed.
How to live stream a 360° video with Ambisonics audio on Facebook using Insta360 camera and ZYLIA ZM-1microphone
For this workflow you will require:
This project consists of 3 main parts:
A. Receiving a live 360° video from the Insta360 to OBS
B. Receiving Ambisonics audio from the ZM-1microphone to OBS.
C. Configuring Facebook for live stream of 360° video with Ambisonics audio
A. Receiving a live 360° video from the Insta360 to OBS
1. Turn On the Insta360 camera and connect your computer Wi-Fi to the corresponding Insta360 Wi-Fi network.
2. Simultaneously, connect your computer to your local ethernet using an ethernet cable. On your network preferences you should be connected with Ethernet and Wi-Fi.
On step number 4 you will need the IP address for the Insta360 Wi-Fi referred on your network preferences.
3. On your MacOS, open the application Local RTMP server (https://github.com/sallar/mac-local-rtmp-server).
This simple application will be used to receive a live stream from the Insta360 camera. It also provides you the local host address for OBS.
Start streaming from the Insta360. The Local RTMP server application icon should now turn red as an indication of receiving the stream.
5. Open OBS and setup your project:
6. To receive the video stream from the Insta360 to OBS:
After adding the source, you should be receiving a 360° video from the Insta360 into OBS.
B Receiving Ambisonics audio from the ZM-1microphone to OBS.
7. Connect your ZM-1 microphone to your computer.
Open ZYLIA Streaming application.
8. In OBS, add source select ZYLIA Streaming application.
You are now receiving 1st Order Ambisonics audio (4 channels) and a 360° video into OBS.
C Setup Facebook for livestreaming
On the Settings – Stream, enable “Broadcast this as a spherical video” and also “This broadcast has Audio 360 (ambix4)”.
10. In OBS press “Start Streaming”. You are now live!
We are happy to announce the new release of ZYLIA ZM-1 drivers for Windows 10 (v2.1.0). Version for Windows 7 will be released in the future.
Some of you, who came across the term ‘3D sound’, probably are wondering what it actually is?
Intuitively, we may understand it by analogy to 3D space, which as we know is made up of three dimensions: width, depth, and height. In real life, any position in space is characterized by these dimensions.
Now imagine, that you are outside, for example in a park, full of various sounds. You can hear children laughing on the playground behind you; a dog is barking on your left, a couple of people are talking while sitting on a bench in front of you – you can hear their voices more and more clearly when you move towards them. We may say that you are in the 3-dimensional sound space. When you move, the sound you hear changes as well, corresponding to the position of your ears (its intensity, direction, height and even timbre).
Reproducing this natural human way of hearing in the recording isn’t easy, however, we already have the technology which allows us to do it – it is binaural and Ambisonics sound.
The most common type of recordings – which you hear while listening to the music on your computer or watching TV –is stereo. You may also come across mono recordings but these have been outdated with the introduction of stereo mixing. Stereo sound basically allows you only to hear if the sound comes from left or right which is the main advantage over the mono-type of audio.
Bellow, you may find some great recordings that will allow you to hear the difference between these different formats. We have also summarized their characteristics in the bullet points, to present the information most clearly.
#zylia #binaural #ambisonics #3Daudio #surround #spatial #sound
New ZYLIA Streaming Application allows you to set up binaural and Ambisonics audio streaming on your Mac computer with no need off using Digital Audio Workstation.
It’s FREE for all current ZYLIA Ambisonics Converter owners. So available for users of the ZYLIA PRO Ambisonics and ZYLIA PRO Have it all! sets.
How can you use 3D audio streaming?
To set up binaural and Ambisonics audio streaming you will need the following prerequisites:
Learn everything about ZYLIA Streaming Application with the following video presentation:
In the video we cover:
We are happy to announce the new release of the ZYLIA Streaming Application (v1.0.1).
With this version, we fixed the problem with streaming via the OBS plugin, when OBS is in stereo mode.
We are happy to announce the very first release of ZYLIA Streaming Application v1.0 for macOS. This is our new solution for quick and easy setting-up live streaming of your music performance in the Ambisonics format. The core of this application is ZYLIA Ambisonics Converter that converts the ZM-1 multi-channel recordings into Higher Order Ambisonics (HOA).
With this tool, you can stream your audio content directly in HOA format (1st, 2nd, and 3rd order) or Binaural format by using an additional binauralization plugin. It can be easily configured with streaming software like OBS (Open Broadcaster Software) that allows combining your audio stream with a video stream and transmitting directly to most of the well-known multimedia platforms (e.g. YouTube, Facebook, Twitch, etc.). Also in addition to this application, we provide the beta OBS Plugin which is able to get direct output from ZYLIA Streaming Application, without using any virtual sound card.
ZYLIA Streaming Application will be available as an addition to all packages that contain the ZYLIA Ambisonics Converter plugin, eg. the ZYLIA PRO Ambisonics package.
#zylia #streaming #3Daudio #Ambisonics #binaural #live #music
We are happy to announce the new release of ZYLIA ZM-1 drivers for macOS. This driver supports the latest Apple computers with an M1 processor. As it is still kernel extension, you may need to Enable System Extension for M1 systems. This instruction is available on the Apple webpage and also here.
Also, all other ZYLIA Software is able to run on this kind of machine in Intel compatibility mode with Rosetta 2. ( https://support.apple.com/en-us/HT211861)
We are happy to announce the new release of ZYLIA Studio (v2.1).
With this version, we introduce the following features: