By Eduardo Patricio
In this post we present two videos in different formats, but edited from the same source material captured on the 20th of June 2018, at Barigui park (Curitiba, Brazil).
The audio was recorded with the ZYLIA ZM-1 3rd order Ambisonics spherical microphone array while the video was captured by a 360-degree camera (Gear 360).
Below, you can watch both videos and find some information on how to achieve the two different results, with focus on preparing the audio recorded with the ZM-1 microphone for each scenario.
Interactive, immersive video with full 3D sound
(media components: 360-degree video + Ambisonics audio)
Non-interactive video with fixed perspective 3D sound
(media components: Tiny planet” video + binaural audio)
The microphone and the camera were placed on a single camera stand with a small clamped extension arm (see picture below). Both devices were aligned vertically with a small horizontal offset. We made sure the microphone and the camera always had the same relative facing direction (front of the microphone aligned with the camera side where the recording button is found).
For scenario B, we used the video from Gear 360 in ‘tiny planet’ format and a binaural audio track.
Since, the source material is the same as the one from scenario A, we’ll list here only the steps that differ.
Scenario B steps:
Choosing binaural preset on ZYLIA Studio PRO in REAPER
#ambiencerecording #ambisonics #binaural #soundscapes #immersiveaudio #360recording
By Zaneta Lejwoda
ZYLIA Studio PRO is an innovative VST/AU plugin which allows you to separate sources from ZYLIA ZM-1 microphone recording in a real-time. It requires 64-bit DAW with at least 19 channels per track. You can create up to 24 "virtual microphones", adjust and automate parameters of them such as azimuth, elevation and polar pattern. Because of this multichannel processing you can't use ZYLIA Studio PRO directly within ProTools but there is a way to mix signals from „virtual microphones” in this DAW – you can use ReWire protocol.
ReWire allows you to connect two applications, where first is a master (it's called "ReWire Mixer") and second is a slave ("ReWire Device"). In this tutorial, we will use ProTools as ReWire Mixer and Reaper as ReWire Device. Reaper is a digital audio workstation, it handles up to 64 channels per track and it's perfect to use it with ZYLIA Studio PRO.
First of all, we need a host for ZYLIA Studio PRO plugin – Reaper (if you haven't installed it yet – now it's good time to do it).
Run Reaper, open Preferences – Plugins – ReWire/DX and enable the option "Check for ReWire mixer/device on startup (and automatically enter slave mode if present)", click "OK" and close application (Fig. 1).
Fig. 1: Reaper, ReWire configuration.
Next, open Pro Tools and make a new session – remember that ZYLIA requires 48 kHz sample rate (Fig. 2).
Create new tracks for all sources („virtual microphones”) you want to use in your mix.
Fig. 2: Pro Tools, audio settings.
Open Reaper, configure session and import your recording (you can use our template).
Now, the application should run automatically in the slave mode – you should be able to see "ReWire Slave (or just ReWire) on the menu bar, next to your audio settings.
Open Routing Matrix (View – Routing Matrix)
Delete send of Master Bus to ReWire Output 1(stereo) and send your virtual microphones' tracks to ReWire Mono Output - in this case it's Output 3 to 6 (Fig. 3).
Now, you can go back to Pro Tools.
Fig. 3: Reaper, routing matrix.
On previously prepared tracks insert Reaper, click on insert field, choose Multi-mono plug-in – Instruments – Reaper (Fig. 4).
Fig. 4: Pro Tools, Reaper inserting.
After doing that you will see the window with ReWire logo, in which you can choose ReWire Output (picture below). In this case, Z1 track in Reaper is connected to ReWire mono output number 3, so Pro Tools' Z1 track is assigned to ReWire Output 3 and so forth (Fig. 5).
Fig. 5: Pro Tools, choosing ReWire Output fo Z1 track.
That's all, now you should be able to hear all your virtual microphones in Pro Tools – just insert your favorite plugins and have fun mixing!
by Zaneta Lejwoda
In this article, we will show how to configure Ardour session to work with ZYLIA Studio PRO.
Briefly, we connect 19-channel track recorded with ZYLIA ZM-1 microphone with "output" tracks for virtual microphones (and leave out the connection to master) and then route "output" tracks to the master bus.
Now the most complicated thing – routing. Open Audio Connection Manager (Window → Audio Connections or Alt+P). In this window, you can see a matrix of Sources and Destinations.
By Zaneta Lejwoda
In this tutorial we will explain how to configure Reaper session to work with ZYLIA Studio PRO software.
Similar settings are applicable for more virtual microphones – just add more “output” tracks and set them up as above (remember to choose a proper audio input channel number).
It is possible to record directly from a ZM-1 microphone into a Reaper DAW.
Routing for recording is similar to a mixing setup. You should add track for 19-channel recording with plugged ZYLIA Studio PRO and “output” track for each virtual microphone. Remember to set up appropriate number of “Zylia Studio PRO” track’s channels and assign each “output” track’s audio input to corresponding virtual microphone output.
Input monitoring using Aggregate Device (available only for macOS users).
ZYLIA Studio PRO Reaper Template
By Jakub Zamojski & Lukasz Januszkiewicz
Recording and mixing surround sound becomes more and more popular. Among the popular multichannel representation of surround sound systems like 5.1, 7.1 or cinematic 22.2, especially worthy of note is an Ambisonics format, which is a full-sphere spatial audio technique allowing to get a real immersive experience of 3D sound. You can find more details about Ambisonics here (What is the Ambisonics format?).
Our previous blog post “2nd order Ambisonics Demo VR” described the process of combining audio and the corresponding 360 video into fine 360 movie on Facebook. Presented approach assumes using of 8-channel TBE signal from ZYLIA Ambisonics Converter and converts audio into the Ambisonics domain. As a result we get a nice 3D sound image which is rotating and adapting together with the virtual movement of our position. However, it is still not possible to adjust parameters (gain, EQ correction, etc.) or change the relative position of the individual sound sources present in the recorded sound scene.
In this tutorial we are going to introduce another approach of using ZYLIA ZM-1 to create a 3D sound recording, which gives much more flexibility in sound source manipulation. It allows us not only to adjust the position of instruments in recorded 3D space around ZYLIA microphone, but also to control the gain or to apply any additional effects (EQ, Comp, etc.). In this way we are able to create a fancy spatial mix using only one microphone instead of several spot mics!
Spatial Encoding of Sound Sources – Tutorial
In the end of July 2017, using ZYLIA ZM-1 microphone we have recorded a band called “Trelotechnika”. All band members were located around ZM-1 microphone, 4 musicians and one additional sound source – drums (played from a loudspeaker). During the post-production process, we applied ZYLIA Studio PRO VST plug-in (within Reaper DAW) on recorded 19-channel audio track. This allowed us to separate the previously recorded instruments and transfer them into the individual tracks in the DAW. Those tracks were then directed to the FB360 plug-ins, where encoding to the Ambisonics domain was performed.
“Spatial Encoding of Sound Sources” - a step-by-step description
Below, you will find a detailed description of how to run a demo session presenting our approach of recording and spatial encoding of sound sources. Demo works on Mac Os X and Windows.
After opening the session, you will see several tracks:
3. Separated signals from ZYLIA Studio PRO are passing to 5 individual tracks. You are able to adjust the gain, you can also mute or solo instruments, or you can apply some audio effects. A good practice is to use a high-pass filter for non-bass and low-pass for bass instruments to reduce a spill between them. We applied these filters to our session:
4. Spatialiser track – receives 5 signals from tracks with separated instruments. Spatialiser allows to distribute sound sources in desired positions in the 3D space.
a) Click on FX and choose FB360 Spatialiser.
d) Back to Spatialiser view. You will see an equirectangular picture and five circles with numbers. Each circle represents a sound source position in the space. By default, sources are located in the positions corresponding to the real positions of the instruments in the picture, but it is possible to adjust it by clicking on the circle and dragging it around the picture.
6. Now video is synchronized with audio. Adjusting the location of play-head in REAPER’s time line will affect the video’s time. Tap space bar to play audio and video. Rotation of the video in the player is tracked by the decoded and binauralized Ambisonics sound.
7. A good practice is to play video from the beginning of file to keep the synchronization. In some cases, it is necessary to close the VideoClient + VideoPlayer and load 360 video again to recover the synchronization.
8. Now you are able to rotate video across the pitch and yaw axis. Your demo is ready to run.
by Maciej Lenartowicz
Integrating new tools into existing work-flows may be a difficult task, especially when those tools introduce innovative features, such as “virtual microphones” in ZYLIA Studio PRO.
This article will guide you through the process of setting up a connection between ZYLIA Studio PRO (running within REAPER) and your DAW of choice (assuming it supports ReWire protocol).
Example session in REAPER with ZYLIA Studio PRO. Notice that REAPER runs in ReWire mode.
4. Now you can create a new project in REAPER or load existing one. Use ZYLIA Studio PRO as you would otherwise, i.e. create a new track for every virtual microphone, with just one exception – these new tracks won’t be sent to master track. To do so, enter “Routing” menu for a track in REAPER and disable “Master send” option.
To route audio form virtual microphone out of REAPER, you need to select “hardware” output for a track. These are the very same channels that will be available in Ableton. Click on “Add new hardware output...” menu and select one of the outputs from the list. It is advised to start from “ReWire Output 3” (for Z1 virtual microphone) since first and second outputs are usually associated with a master track
Ableton session set to receive audio from REAPER.
Author: Łukasz Kurzawski
The main aim of the test was to make a comparison of traditional multitrack recording techniques with recording using the ZYLIA ZM-1 microphone. I compared the both approaches in terms of offered recording capabilities, tone and sound quality. I have chosen acoustically challenging environment – music ensemble recorded in a church (see the Test details box for more details on this).
The traditional recording setup
For this type of recordings I usually use a traditional recording setup. It consists of 8 small diaphragm condenser microphones (4 stereo pairs).
You can check the result of the recording with the traditional setup by playing back the file Church_traditional_setup.wav, Samplitude screenshot from the recording is presented below (Fig. 1). It presents well the balance between the microphones: the first two tracks are the main microphones which dominate the overall mix (in red).
Fig. 1 Samplitude screenshot.
ZYLIA microphone in the middle
The whole ensemble was recorded also simultaneously with the ZYLIA ZM-1 microphone. The microphone setup was very easy – just one microphone, one stand and one USB cable plugged into the computer. The ZM-1 was set in the middle of the ensemble – very close to the stand of the main stereo pair (approx. 1 meter away from all musicians). See the picture below as a reference (Fig. 2).
Fig. 2 Ensemble recording.
I tested two heights for the microphone 1.7 m and 2.15 m. The stereo mixes are stored in the files Church_Zylia_170cm.wav and Church_Zylia_215cm.wav, respectively.
The resulting stereo mix from the ZYLIA microphone was good and on par with the recording using a traditional setup. I found that the height of 1.7m for the microphone was optimal for this recording. In this setup the aisle of the ZM-1 was closer to the sound sources. The height of 2.15m gave worse results in terms of tone and spatial feeling. This was interesting because in a traditional setup you can expect a better spatial feeling when putting the stereo pair higher.
Magic in ZYLIA Studio PRO
To be honest before the tests I was really sceptical about the tone and spatial effect of recording with ZYLIA but the end results surpassed my expectations and opened my mind for completely new recording capabilities. The recording process with ZM-1 was very straightforward and quick to setup but the whole fun and magic really started when I launched the ZYLIA Studio PRO VST plugin within Reaper. It is really incredible what you can do in the post production with ZYLIA Studio PRO. I started by creating 3 virtual microphones that were pointing at the locations of the instruments: Z1 – flute, Z2 – cello, and Z3 – harpsichord. See the below screenshot for the setup of virtual microphones (Fig. 3).
Fig.3 ZYLIA Studio PRO, VST plugin.
After that I have added an extra stereo pair of virtual microphones Z4 and Z5. It appeared that this pair gave an excellent result in terms of tone and panorama effect. It is really incredible that you can change the wideness of this stereo pair and its position in both horizontal and vertical pane. In this given recording setup the cello and the harpsichord are placed naturally lower than the flute, so by changing the elevation parameter of the virtual microphones you can be more selective on each instrument. An interesting effect was to change the aim of the Z4 and Z5 pair in the vertical pane by pointing it up 45 degrees. This setting resulted in a much deeper spatial effect without losing the detail of each instrument (this setting was used in the final mix). With such setup of Z4 and Z5 I was missing a better exposition of the harpsichord so I added a bit more of the Z3 (harpsichord virtual mic). The end result was a very good sounding mix – the best I have managed to create from this recording session (file Church_Zylia_VST_Z4Z5Z3.wav).
As a final touch I tried to add more space to the recording to reflect great acoustics of the church. In order to do that I have created another stereo pair of virtual mics Z6 and Z7 which was aimed at the space without sound sources. In this setting I missed a little bit of flute so I have added more of the Z1 (flute virtual mic) to the mix. The effect is recorded in file Church_Zylia_VST_Z6Z7Z1.wav. All the above mixes were made using the recording from the ZM-1 placed at the height of 1.7 m. Setting it higher at 2.15 m limits a bit the possibilities of changing the balance and space of the recording.
My personal view
Recording using ZYLIA microphone really changed my thinking about music recording. Using the mic and the plugin I have access to almost infinite number of additional virtual microphones on any side of the “magic ball”. On top of that I can point virtual microphones or stereo pairs up and down changing their elevation – this opened my mind in a completely new “vertical” way. These capabilities offer great flexibility and many extra options during post-production. In addition the ZYLIA ZM-1 can fix microphone placement mistakes or mask bad acoustic conditions of the recording venue. Something that is currently not possible, or not so easy, using traditional methods.
Overall I am very satisfied with the recordings and final mixes achieved using ZYLIA. The ZM-1 microphone together with the ZYLIA Studio PRO VST plugin opens up a whole new exciting range of opportunities for recording engineers and audio creators. What is really important is that you can do a lot of modifications in the post-production. As a sound designer you can decide on the spatial effect, the tone and balance of the instruments after the recording is made from the comfort of your studio by listening to the effects of your work in good listening conditions. The creators of the mic already announced addition of new options to the software like the possibility to change the width and characteristic of individual virtual microphones. This will give complete control over the recording/mixing process into the hands of sound designer and will open endless possibilities. Now it is only up to you how you are going to use them...
By Łukasz Januszkiewicz
Step two - On The Stage
- Microphone placement.
The main ZM-1 microphone was positioned approximately 20 cm below the camera.
Backup ZM-1’s were positioned on the right and on the left side of the main ZM-1 (distance of 4-5 meters). There was also one mic just right in front of the choir.
- The final setup
Recordings made with 6DOF Development Kit by Zylia
Read more ...
Step Three – Recording our perfect 360 movie
Step Four - Video And Audio Post-processing
*At this moment YouTube 360 supports only 1st order Ambisonics audio.
360 movie with 3D sound - The final effect
Video will be available soon.
How to turn a room into a recording studio?