Zylia is building a future of immersive and fully navigable audio for Virtual Reality by creating an installation of 53 3rd order Ambisonics microphone arrays
Zylia introduced a six-degrees-of-freedom (6DoF) multi-level microphone arrays installation for navigable live recorded audio.
What does it mean? We are working on technology that gives people the possibility to listen to a concert or live performance from any point in the audio scene. With our technology you are able to record an audio scene from different points of the space – the center of the stage, from the middle of a string quartet, audience, or backstage. Audio recorded in such a way can be used together with virtual reality projections and allow the user to freely move around the space giving the natural experience of audio scene and the possibility to listen to it from different perspectives.
6DoF installation and test setup
The first step of test recordings was to install 9 3rd order Ambisonics microphone arrays on the same level and record musicians playing their performance. Such an approach allowed the listener to move around those 9 points and listen to their music from different perspectives. However, microphones placement on a single level introduced limitations in terms of audio resolution in the vertical plane.
Since we like challenges we decided to increase the number of microphones to 53 and build an installation on five different levels. It allowed us to freely move in every direction of the recorded scene in a truly immersive experience. The second idea behind this test setup was to check the limits of Ambisonics recordings in order to achieve a fully navigable audio scene. We placed the microphones arrays densely in the recorded scene and we received a spatial audio image of very high resolution.
We used 53 19-channel mic arrays – which gave us 1007 audio channels recorded simultaneously. Microphones were connected to a USB hub and the recordings were operated via a single laptop.
The audio recorded from each microphone array was converted to 3rd order Ambisonics using our ZYLIA Ambisonics Converter plugin (it can be done in real-time or offline). After the recording, we used our interpolation software. This software is a MaxMSP plugin, that generates 3rd order Ambisonics spheres based on the signal from all microphones in the position you are at the moment. When you put your headphones and VR headset and move around the space the algorithm in MaxMSP takes your position and interpolates 3D sound in the position you are at the moment.
We used 3rd order Ambisonics microphone arrays. It is important because the higher the order the more precision we get in the spatial localization of sound around the listener. We are able to recreate the sound with a very high spatial resolution which influences the audio quality - an extremely important aspect for listeners.
With this simple approach, you can record the natural audio scene for your VR/AR productions and use it right away without complicated work-flow in post-production. You can record live events and stream audio directly to the listener giving him the possibility to freely choose the position in this real-time recorded space for an ultimate immersive audio experience.
Cinematic trailers for VR, audio for games, live performances recording, domes with multi-loudspeakers installations
by Eduardo Patricio
Earlier this year (2019), during Easter, we recorded a string quintet concert* with 2 ZM-1 microphones. One in front of the ensemble and 1 behind it. Very simple and portable setup with powerful postproduction possibilities!
Here’s a quick example of the multi-track possibilities we have from the recordings:
How we did it
Using ZM-1 microphones basically allows you to have multi-track recordings from single mics.
Here’s what we did for this project in 5 steps.
1. We recorded the 19 channels from each ZM-1 directly onto DAW tracks on each laptop. The result was 2 uncompressed, 24-bit, 48KHz wave files.
Resulting multi-channel audio file being played back in Reaper.
2. After the recording, we loaded both multi-channel files onto a single DAW session, onto separate tracks. We synchronized the files manually using a clear transient sound (the classic "clap to sync").
Two 19-channel audio files on 2 DAW tracks.
3. We added ZYLIA Studio PRO plugin to the tracks containing the multi-channel files (one instance of the plugin for each). The plugin converts the recorded information into perfect sound spheres, from which you can extract sounds from any direction.
ZYLIA Studio PRO plugin interface before adding any virtual microphones.
4. We added virtual microphones, by clicking on the plus sign in the middle of the circle and defined their direction (in the horizontal and vertical planes) and polar patterns (width). Obs.: That means we basically set up a complex microphone arrangement after the recording, and one that can be tweaked or changed at any moment (!).
NOTE: If you don’t know yet how to work with virtual microphones and ZYLIA Studio PRO, you can check the following additional resources:
4A. From the main microphone recordings, 7 virtual microphones were used as seen in the following screen capture:
ZYLIA Studio PRO with 7 virtual microphones in place.
The virtual microphones Z1 – Z5 were narrow (shotgun-like) ones, pointing at each instrument and using S2 separation mode.
Z6 and Z7 form a wide stereo pair, pointing 45˚ up, to capture some of the overall sound and a great deal of reverberant sound.
4B. From the rear microphone recordings, 5 virtual microphones were used as seen in the following screen capture:
ZYLIA Studio PRO with 5 virtual microphones in place.
Detailed configuration of each virtual microphone:
Z1: 0˚ azimuth, 33˚ elevation, 60˚ width;
Z2 and Z5 form a stereo pair with a sum of 90˚ azimuth, pointing up (-45˚ elevation), and with 60˚ width;
Z3 and Z4 form a stereo pair with a sum of 80˚ azimuth, pointing down (-51˚ elevation), and with 60˚ width.
5. We directed each virtual microphone to individual tracks in our DAW and carried on with a regular basic mixing strategy (leveling, panning and EQing).
Section of DAW showing the tracks receiving signals from virtual microphones Z1-Z5 (Main ZM-1 microphone).
To sum up
by Pedro Firmino
In this article, we will give you an alternative way of recording and coloring your guitar tone using the ZYLIA ZM-1 microphone by taking advantage of multiple channel recording.
Please, remember that these are suggestions based on our own experience and might not be suitable for your specific situation. While trying our methods, make sure to listen and adapt to your instrument, environment, and musician.
When miking a guitar it’s important to understand that this instrument is not designed to be experienced at close distances making it more challenging to find a balanced positioning for recording, especially if you plan to use the ZM-1 to obtain several channels for posterior mixing. After all, it’s one of the ZM-1 strengths.
Here we will guide you to make the most out of the ZM-1 recording abilities.
Position the ZM-1 between the neck and the sound-hole.
At 25 cm distance, the results were quite flexible to work with, however, if you see clipping happening on the VU meter of ZYLIA Studio, reduce the gain on ZYLIA Control Panel or move the ZM-1 further from the sound source.
Record your take and take some time to listen to the results. If you’re happy with the tone of the guitar you may proceed to mixing stage, if not here are some options for you:
- increasing distance of the ZM-1 and the guitar for more room tone and less presence.
- positioning ZM-1 closer to neck, giving you more treble and percussive sounds.
- positioning the ZM-1 closer to the soundhole, provides more lower end and less treble.
- using another channel of the ZM-1 recordings to add brighter tone to the mix.
This last option shows one of the great advantages of recording with the ZM-1, which it will be mentioned next.
Separate the take and Mix your tone.
Using ZYLIA Studio or ZYLIA Studio PRO, you are able to get different tones from your instrument with a single recording due to the multiple channels. It can also be useful if you want to add some reflections from the environment.
After Separating the take, try using the Auto-mix feature. The tone resulting of this feature might surprise you, but if that’s not the case you are always able to mix the tone by controlling the different channel parameters in the MIXER window.
Are you looking for more low end in your tone? Simply adjust the level of the channel correspondent to the microphone closer to the sound-hole of the guitar (In this example Musician 2).
Or perhaps you want more room tone and a wider stereo image of your recording? Increase the levels of the back microphones and apply panning (Musician 4 and 5).
We are happy to share the ProSound News product review featuring ZYLIA hardware and software.
"Zylia Studio software is surprisingly good and is hands-down the easiest way I’ve encountered to record multiple musicians in a single room."
"Recording with the ZM-1 is an enjoyable process, and the quality is surprisingly good. Tracks are natural and smooth, and the isolation between instruments is impressive."
What would happen if on a rainy and cloudy day, during a walk along a forest path, you could move into a completely different place thousands of kilometers away from you? Putting the goggles on would get you into a virtual reality world, you would find yourself on a sunny island in the Pacific Ocean, you would be on the beach, admiring the scenery and walking among the palm trees listening to the sound of waves and colorful parrots screeching over your head.
It sounds unrealistic, but such goals are determined by the latest trends in the development of Augmented / Virtual Reality technology (AR / VR). Technology and content for full VR or 6DoF (6 Degrees-of-Freedom) rendered in real time will give the user the opportunity to interact and navigate through virtual worlds. To experience the feeling of "full immersion" in the virtual world, realistic sound must also follow a high-level image. Therefore, only each individual sound source present in virtual audio landscape provided to the user as a single object signal can reliably reflect both the environment and the way the user interacts with it.
What are Six Degrees of Freedom (6DOF)
"Six degrees of freedom" is a specific parameter count for the number of degrees of freedom an object has in three-dimensional space, such as the real world. It means that there are six parameters or ways that the object can move.
There are many possibilities of using a 6DoF VR technology. You can imagine exploring a movie plan in your own pace. You could stroll between the actors, look at the action from different sides, listen to any conversations and paying attention to what is interesting only for you. Such technology would provide really unique experiences.
A wide spectrum of virtual reality applications drives the development of technology in the audio-visual industry. Until now, image-related technologies have been developing much faster, leaving the sound far behind. We have made the first attempts to show that 6DoF for sound is also achievable.
How to record audio in 6DoF?
It's extremely challenging to record high-quality sound from many sources present in the sound scene at the same time. We managed to do this using nine ZYLIA ZM-1 multi-track microphone arrays evenly spaced in the room.
In our experiment the sound field was captured using two different spatial arrangements of ZYLIA ZM-1 microphones placed within and around the recorded sound scenes. In the first arrangement, nine ZYLIA ZM-1 microphones were placed on a rectangular grid. Second configuration consisted of seven microphones placed on a grid composed of equilateral triangles.
Fig. Setup of 9 and 7 ZYLIA ZM-1 microphone arrays
Microphone signals were captured using a personal computer running GNU/Linux operating system. Signals originating from individual ZM-1 arrays were recorded with the specially designed software.
We recorded a few takes of musical performance with instruments such as an Irish bouzouki (stringed instrument similar to the mandolin), a tabla (Indian drums), acoustic guitars and a cajon.
Unity and 3D audio
To present interesting possibilities of using audio recorded with multiple microphone arrays we have created a Unity project with 7 Ambisonics sources. In this simulated environment, you will find three sound sources (our musicians) represented by bonfires among whom you can move around. Experiencing fluent immersive audio becomes so natural that you can actually feel being inside of this scene.
MPEG Standardization Committee
Since the beginning, every prototype solution has been tested with musicians and sound engineers. Based on their feedback we have introduced changes in quality and functionality of microphones. What we have learned very early is that the quality of obtained recordings is crucial for musicians. No compromises are allowed in this matter.
Every detail of the ZYLIA ZM-1 reflects the cutting-edge technology and simplicity with elegant product design.