By Eduardo Patricio
In this post we present two videos in different formats, but edited from the same source material captured on the 20th of June 2018, at Barigui park (Curitiba, Brazil).
The audio was recorded with the ZYLIA ZM-1 3rd order Ambisonics spherical microphone array while the video was captured by a 360-degree camera (Gear 360).
Below, you can watch both videos and find some information on how to achieve the two different results, with focus on preparing the audio recorded with the ZM-1 microphone for each scenario.
Interactive, immersive video with full 3D sound
(media components: 360-degree video + Ambisonics audio)
Non-interactive video with fixed perspective 3D sound
(media components: Tiny planet” video + binaural audio)
The microphone and the camera were placed on a single camera stand with a small clamped extension arm (see picture below). Both devices were aligned vertically with a small horizontal offset. We made sure the microphone and the camera always had the same relative facing direction (front of the microphone aligned with the camera side where the recording button is found).
For scenario B, we used the video from Gear 360 in ‘tiny planet’ format and a binaural audio track.
Since, the source material is the same as the one from scenario A, we’ll list here only the steps that differ.
Scenario B steps:
Choosing binaural preset on ZYLIA Studio PRO in REAPER
#ambiencerecording #ambisonics #binaural #soundscapes #immersiveaudio #360recording
We are happy to announce a release of ZYLIA ZM-1 drivers for Windows 7. Now our software is compatible with Windows 7 and 10.
We are thrilled to announce new versions of ZYLIA Studio (1.7.2), ZYLIA Studio PRO (1.3.1) and update of ZYLIA ZM‑1 drivers.
ZYLIA ZM-1 model 3E support
The most significant change is support for a new model of ZYLIA ZM-1. We added tailored spatial filters for a 3E microphone model to ZYLIA Studio and ZYLIA Studio PRO.
Source localization in ZYLIA Studio PRO
Source localization function is a new feature which will help you to find sound sources in the space. Thanks to that you will add a new virtual microphone in the right 3D position immediately.
To enable this function, press on the "Source localization" button in the left-upper corner of ZYLIA Studio PRO window. The localization algorithm works in real time. During playback of your 19-channel recording, estimated positions of the sound sources will appear on the VST/AU radar screen (blue icons). To add a virtual microphone in this position, simply click on the sound source icon.
ZYLIA Control Panel (macOS)
ZYLIA Control Panel now is bundled with ZYLIA ZM-1 driver for macOS operating systems. Thanks to this application you will be able to manipulate LED brightness of your ZYLIA ZM-1 device.
To dim LED of plugged ZYLIA ZM-1 microphone, open ZYLIA Control Panel application and by using the slider decrease brightness to the desired level (you can turn the LED off as well). After closing ZYLIA Control Panel you'll be ready to start your recording session. ZYLIA Control Panel saves the last used LED brightness level and restores it right after application start.
Additionally we fixed a few small bugs to improve stability of ZYLIA software.
Organizing rehearsals is much easier nowadays than it was in the past. Songwriters and musicians can easily rent a rehearsal room, join together and take part in the jam session.
But how to achieve a great rehearsal recording without spending too much time on technical issues? First, you need to know is what happens with the audio signal that goes to the recording device. The terms 'conversion' and 'converter' are crucial here.
Band rehearsal recorded with ZYLIA ZM-1.
Conversion of the audio into a digital form
The audio which comes from a microphone or directly from an instrument to the interface is actually a series of electric signals of varying voltage, which represent acoustic sounds. These signals are converted into a series of numbers (by the analogue-to-digital converter), which correspond to the specific values of the voltage. This process is called conversion, digitalization or sampling.
To be more detailed, digitalization is collecting a great number of snapshots (samples) of changing voltage values in equal time intervals. You can visualize this by comparing this process with a film stock which consists of many still photographs taken with the camera at the specific time frame.
Depending on the sampling frequency set, these snapshots are taken at different time intervals. For example, at 44.1kHz the analog-to-digital converter takes one sample every 1 / 44,100th of a second. It mins that it takes 44,100 samples representing the voltage every one second and converts them into appropriate numbers which can be stored in the digital form.
The audio signal recorded in this form, in order to be reproduced, must go ‘way back’ – so it needs to be converted into the analog signal, which will again be a variable electrical voltage. This process takes place in the digital-to-analog converter.
Audio bit depth
In digital audio ‘bit depth’ is the number of bits of information in each sample, and it directly corresponds to the resolution of each sample. To simplify, we can assume that the sampling frequency are values assigned in time and the bit depth are values assigned to the strength of the signal.
As it was already mentioned, in the 44.1kHz system, the audio-to-digital converter takes 44,100 samples per second. So, in the 48kHz system it is going to be 48,000, and so on. Increasing this value allows capturing more information about the sound which equals higher sound frequencies. This simply means that high-frequency instrument components can be better reproduced.
On mastering process, it is significant to operate on full band samples where low and high-frequency components are well heard. It is easier to mask some instruments by others in the same acoustic scene. Taking care of high sample frequency, you can guarantee your listeners’ natural timbre and feelings.
High sample frequency and bit resolution recordings are strongly recommended in compression, equalization and many additional audio effects. Standard configurations are 16 bits depth and 44,1k Hz sample frequency called CD standard. More advanced audio processing may require higher parameters.
If you neglect to configure high bit resolution a quantization noise will appear in your recording. It is an understandable effect since sound samples will be represented by an insufficient number of bits.
In audio mastering process, the quantization noise prevents engineer to operate on well-recorded sound frequencies. Low bit depth frequency components are much distorted than the high bit. When mastering operator mix rehearsals with noise, the final record contains strengthened noise, which may be onerous for the listener and difficult to accept in professional audio production.
Typically 3D audio is reproduced using whether multi loudspeaker setups (5.1, 7.1, 22.2, e.g. Dolby Atmos) or headphones. Therefore, there are also two main recording setups related to 3D audio recording:
There is also a third approach, recording of sound objects or folleys and use them in game engine mixers or 3D audio renderers.
MUSIC AND AUDIO PRODUCTION PROCES
Now you can divide audio production process into four main stages. Pre-production stage is the most engaging. It depends on many factors such as the characteristics of sound sources you are going to record, their position in relation to each other, acoustic conditions or simply your budget for equipment. It requires selection of microphones and their right placement in recording landscape. In many cases, it is connected with multiple test recordings you need to make and microphone placement changes. The more experienced audio engineer you are the less time it should take and the better results you will obtain.
The second stage – recording. It should be straightforward. Start recording, repeat it, record a few takes and that’s it. But what if you had just one chance to record a scene? Or you didn’t notice that there was something wrong with your microphone placement? You go home and then it shows up that your recordings are poor, that some strange noises from a background are noticeable. What can you do? You can repeat a whole recording day or try to mask it in the mixing process. In both cases, it equals additional hours of your extra work.
You never know what happens during a field recording. Being able to correct a “bad” microphone placement after the recording would be a great option!
Felix Andriessens, Sound designer, BERLIN
How to save some time and reduce the risk of bad microphone placement or unwanted noises in your recordings? You can simplify the first two steps by using a ZYLIA microphone array. The system is capable of recording a full 360˚ music scene and representing it in 19-channel ZYLIA format or B-format (Wikipedia). By using one of those formats it is possible to save the whole sound field information in a single recording which can be further (in post-production) decomposed into any number of Virtual Microphones. In this case, you are using one device for recording the whole 360˚ audio scene. You simply capture everything with a high spatial resolution.
HOW IT WORKS? THE TECHNOLOGY BEHIND ZYLIA ZM-1
ZYLIA is a 3rd order Ambisonics microphone array. That is, it records 360˚ audio in the high spatial resolution that further on can be decomposed into different audio layers that represent individual sound sources, instruments or background noises.
Instead of using multiple spot microphones you put ZYLIA in the middle of the audio scene. The recorded audio tracks are processed by the dedicated software which allows creating so-called Virtual Microphones. They provide an adaptive isolation of sound sources. As a result, you receive separated tracks of the recorded sound sources. The software allows for steering a Virtual Microphone at any direction in the 3D space. In post-processing, you can now easily manipulate with recording and select exact sound sources, ambient or room reflections. What is more, you can identify unwanted sounds and remove them from your recording. Virtual Microphones operate in two modes:
The main advantage of this approach is that all those adjustments may be done without repeating the pre-recording and recording step because it is possible to change a spatial characteristic of Virtual Microphone in the post‑production phase
3D AUDIO IN A NEW WAY
ZYLIA ZM-1 microphone array can be used in many applications. Let us name a few: binaural recordings, object recording, Ambisonics 360˚ or multichannel 5.1, 7.1 and 22.2.
Binaural recording is a well-known method of recording sound scene with two microphones, creating a 3D stereo sound sensation for the listener, he can actually hear the effect of being in the middle of the sound scene. This effect is often created using a technique known as "dummy head recording" but you can also prepare a binaural recording with ZYLIA microphone array. There are two options to achieve that effect with ZYLIA:
Fig. Ambient and binaural recordings with ZYLIA ZM-1.
Typical audio mix used for film or game production contains lots of sound objects that are further placed in the 3D space by using dedicated spatialization plugin. However, recording of each object separately is time-consuming and prone to errors such as capturing of unwanted noises.
By using ZYLIA ZM-1 it is possible to record a full 360˚ audio scene and then in the post‑production extract sound objects – instruments, dialogs, etc. Extraction process provides:
3rd order Ambisonics and 360 movies
Ambisonics audio (B-format) can be used directly in many audio and video players such as YouTube or Facebook 360˚. 360˚ players convert Ambisonics audio into binaural (two channel) signal. The higher the microphone order the better the spatial resolution (ability to extract and listen to point sources in the surrounding sound scene) and rotation effect during playback.
It is highly recommended to use the highest possible microphone array, e.g. 3rd order. The high spatial resolution provides a truly immersive experience where the listener is able to easily localize sound sources in the 3D space. On the other hand, using lower order microphones result in fuzzy sounds during binaural or loudspeaker playback.
360˚ audio comes usually with 360˚ videos. For that context, ZYLIA ZM-1 is often combined with a range of popular cameras e.g. Insta360.
3rd order Ambisonics plus objects
There are situations where Ambisonics sound is not enough to fully represents 3D audio space. Usually, it is a scene where there is a lot of sound sources which lead to problems with sound localisation. There are at least two solutions to solve this problem:
Cinematic audio and music production often requires recording a surround sound e.g. 5.1, 22.2 (Dolby Atmos). In that context, it is possible to use ZYLIA ZM-1 and ZYLIA Studio PRO plugin. The latter consists of surround preset demonstrated in the below picture.
Fig. ZYLIA Studio PRO.
A notable example of surround sound production is a cinematic trailer of First Aid Kit band recorded by Sony Music UK. During that production, Sony Music UK (Kris Winter) used a ZYLIA ZM-1 microphone attached to a standard filming equipment.
"Music is the only language in the universe that we can all speak and understand" says Basia Galaj our Brand Ambassador, sharing her thoughts in a short interview.
Zylia: Tell us why music is so important for you? When and how it all began?
Basia: Music is the only language in the universe that we can all speak and understand. Even when you don’t necessarily play an instrument or sing, you can feel it. Music makes people emotional, helps go through happy and sad times and that is why I feel like it is the most powerful weapon we have. It is very important to me because it brings hope that humans can unite and create no matter the differences.
When I was about a year old, my grandma started singing to me and playing me different recordings and albums. I would repeat those melodies after her. When she passed away, my mom and grandpa took over. Grandfather played me jazz in the car - lots of Ella Fitzgerald and Louis Armstrong; I still remember the cassettes he used! :-) My mom showed me Michael Jackson, Basia Trzetrzelewska, Kayah and classical music such as Bach and Chopin.
That lead to piano lessons at age of 5 and continues until day on :-)
Zylia: How and when did you firs hear about ZYLIA?
Basia: I heard about ZYLIA from my mentor and friend – Mikolaj Stroinski. It was during the summer of 2017. He called me and said: Basia, you need to meet these incredible guys, They created a mic... and he just kept going and going :-)
Zylia: How do you see the potential of microphone array technology in changing the way musicians record themselves?
Basia: I think and I really hope it will become a necessity for every musician or music passionate that would like to easily record themselves in a great quality. It has so much potential in many areas! It can only grow in the VR world and that is becoming a next step in the music industry. It makes life of bands much easier since it is using only one cable, not 19 for every mic capsule inside the mic.
Zylia: What are your plans for future? Any interesting projects on your mind?
Basia: I am the leader / organizer of the very first TED talk at Berklee that will be happening next semester – Fall 2018. I am hoping to grow my blog and YouTube channel; I have lots of new content coming up very soon! Over the summer I am going to be doing interviews with Polish World War II veterans, asking them about their experiences and memories and I am going to write music to their confessions and stories. Hopefully, I will be able to use ZYLIA mic to record the interviews :-)
Some extra thing that I do is a project I called – Throwback Thursday! Every Thursday I do a video featuring my friends, we play a song from our childhood or just something we have memories of. I put those videos on my YouTube channel and Facebook fan page :-)
Zylia: We definitely need to see them:-) Thank you for your time and good luck with new projects!
Born in Bydgoszcz, Poland, Basia entered the world of music at a very young age. In 2014, she got accepted into the University of North Texas, where she pursued a degree in Jazz Studies. She has had the privilege to share the stage with artists such as ten-time Grammy Award winner Bobby McFerrin.
Currently Basia lives in Boston, MA and attends the prestigious Berklee College of Music where she is pursuing a degree in Contemporary Writing and Production with a minor in Creative Entrepreneurship. On June 19th 2017, Basia released her debut EP called "Day By Day".
She is currently performing around East Coast with her band, playing her original music.
We are pleased to announce a new release of ZYLIA Studio 1.6.1.
This edition is a minor update to the previous release and it brings a lot of small changes and bug fixes.
The new icon of ZYLIA Studio
Auto-mixing by default
Second change is that we enabled the auto-mix function by default.
After you separate your recording into separate instrument tracks the auto-mixing feature will engage when you visit the mixer screen for the first time. Thanks to that you’ll immediately receive a balanced recording. You can always reset the adjustments using the “Reset adjustments” option in the mixer’s menu. You can also use the “Mix automatically” button to repeat the auto-mixing procedure.
Bug fixes and other improvements
Alongside those two changes, we fixed couple bugs regarding the user’s interface, including: better touch-pad scrolling on macOS, improved snack bar notification’s management and we fixed a minor bug in the auto-mix functionality that could lead to weird results in some band configurations.
Zylia: Tell us what is the story behind your project?
Yao: Last spring, I was 9 months away from graduating from Berklee College of Music, and the panic of post-graduation uncertainty was becoming unbearable. I was struggling to plan my career and I wanted to do something different. I spent a whole summer researching the ins and outs of spatial audio and decided to do my Senior Portfolio Project around my research. What I have found is that spatial audio is often found in VR games and films - recreating a 3D environment. It is rarely used as a tool for music composition and production. I saw my opportunity.
With the help and hard work of my team (around 60 students involved), we succeeded in creating ‘Unraveled’, an immersive 360 audio and visual experience, where the audience would find themselves at the center of all elements, being surrounded by choir, strings, synths and imagery. My role was the project leader, composer, and executive producer. I found a most talented team of friends to work on this together: Gareth Wong and Deniz Turan as co-producers, Carlos Del Castillo as visual designer, Ben Knorr as music contractor, Paden Osburn as music contractor and conductor, Jeffrey Millonig as lead engineer and Sherry Li as lead vocalist and lyricist. Not to mention the wonderful musicians and choir members. I am truly grateful for their hard work, dedication and focus.
‘Unraveled’ also officially kickstarts my company ICTUS, a company that provides music and sound design content specializing in spatial audio solutions. For immersive experiences such as VR, AR and MR, we are your one-stop audio shop for a soundscape that completes the reality. We provide music composition, sound design, 360 recording, mixing, mastering, post-production, spatialization and visualizing services tailored to your unique project.
We are incredibly humbled that 'Unraveled' has been officially selected for the upcoming 44th Seattle International Film Festival, which runs May 17 to June 10, and to have been accepted for the Art and Technology Exhibition at the Boston Cyberarts Gallery, from Saturday May 26 to Sunday July 1.
"Get a pair of headphones. Cave in somewhere quiet. Alone. Empty your thoughts and… Allow yourself to dive into something new. A pristine place that will truly disconnect you from the daily frenzy of life."
‘Unraveled’ has been officially selected for the upcoming 44th Seattle International Film Festival, which runs May 17 - June 10, with more than 400 films from 80 countries, running 25 days, and with over 155,000 attendees!
Zylia: Recording so many people at once must have been challenging. How did you organize this?
Yao: I worked very closely with Paden Osburn, the conductor and music contractor, to schedule, revise, coordinate and plan the session. Paden is a dear to work with, basically allowing me to focus on the music while she coordinated with the rest of the amazing choir members. We had developed a great workflow.
I also had many meetings with the team of engineers as well as many professors to figure out the simplest, most efficient way to record. It was indeed very challenging and stressful to pull off, but it was also one the most magical night of my life.
Behind the scene, photo by @jamiexu0528.
Zylia: Tell us more about the technical part of this project.
Yao: On October 27, 2017, we had a recording session of the choir parts with 40 students from Berklee College of Music. The recording was done using three ambisonic microphones (Zylia, Ambeo, TetraMic). We tried forging a 320 piece choir by asking the 40 students to shift their positions around the microphones for every overdub. We also recorded 12 close mic-ed singers to have some freedom spatializing individual mono sources.
The spatialization was achieved through Facebook360 Spatial Workstation in REAPER. Many sound design elements were created in Ableton and REAPER. The visuals were done in Unity. We basically created a VR game and recorded a 360 video of the performance. Carlos Del Castillo did an outstanding job creating an abstract world that had many moments syncing with the musical cues.
Zylia: What are your plans for future? Any interesting projects on your mind?
Yao: My long-term goal would be to establish my company ICTUS as one of the leading experts in the field of spatial audio. We are currently working on an interactive VR music experience called ‘Flow’ with an ethnic ensemble, GAIA, and the visuals are influenced by Chinese water paintings. The organic nature of this project will be a nice contrast to ‘Unraveled’s futuristic space vibe.
Another segment of the company is focused on creating high quality, cinematic spatial audio for VR films and games. We are producing a 3D audio series featuring short horror/thriller stories with music, descriptive narration, dialogues, SFX and soundscapes. Empathy is truly at the heart of this project, some of our stories will have a humanitarian purpose and we will be associated with many organizations that are fighting to end domestic abuse, human trafficking, rape, abuse and other violent crimes. We hope to bring more awareness and traffic to these causes with our art. Spatial audio is incredibly powerful, it really allows you to be in the shoe of the victims and without the visuals, I swear your imagination will go crazy!
In this short interview our Brand Ambassador Gene Torres describes his experiences with ZYLIA microphone and provides insights on recording his gigs and live performances.
Zylia: Gene, this is our second interview. Last time we talk you were testing a prototype of ZYLIA mic (the first interview). Now you are the owner of your own ZYLIA ZM-1 mic. Do you like it?
Gene: Hello, nice to chat with you again!
ZM-1, as far as I’m concerned, is a game changer in terms of recording rehearsals & gigs. I do like it, the possibilities are endless.
Zylia: How do you use ZYLIA recording studio?
Gene: I mainly use it for gigs & rehearsals. Jorge Sylvester a sax player and composer wants me to record a solo bunch of tunes that he wants to release!
>>Jorge Sylvester web page
Matter of fact, he just did a performance based on Eric Dolphy’s music. If you scroll down the above link, you can hear some of the show.
Performance of Gene Torres, Jorge Sylvester and Nora McCarthy doing The Re-Imagining Eric Dolphy show.
Zylia: What is so interesting for you in this system? Does it change the way you record your music?
Gene: Yes, it has changed the way I record not just my music but all the gigs that I do!!
I think that very noticeable is the fact that recording set-up time was cut by at least 50 minutes, definitely less set-up time is required. Within 10 minutes I’m up and running. When I had my two 8-channel computer interfaces there was at least 16 input cables that had to be plugged in. Not any more!
Recorded live performance. Rhonda Denet and the Silver Fox Trio. Mixed Company.
Zylia: You play with many NY musicians, what is their opinion?
Gene: Once, just out of curiosity, I counted the number of groups I was going to work with, either 6 months back or 6 months forward to a specific date... I counted up to 30 groups/bands!
Their opinion is they freak out cause the ZM-1 has the either Red or Blue led ring around it that glows, so they asked what the heck is that! :-) Once I explain what the ZM-1 is and what it does, they usually ask a million questions!!
Gene Torres, Andy Friedberg and Mike Campenni
Zylia: We are very happy that you decided to be our Ambassador and share with other musicians your knowledge and experiences using ZYLIA.
Gene: Once I received the ZM-1 I felt that the world of NYC needs to know what Zylia came out with and how good it is!!
Zylia: Do you have any plans for future? Any interesting gigs are coming?
Gene: I am going to record a solo recording of Saxophonist Jorge Sylvester when we both can work our schedules out!
I’m trying to finally release a CD as a band-leader. One of the things I want to record is a bunch of tunes with minimal direction but strictly improve. So the CD will have 1/3 originals, 1/3 covers but arranged, and 1/3 improve tunes!
Be on the lookout for it!