For game audio alumnus Giorgio Cortiana, the rapidly changing landscape of interactive audio is an irresistible playground to explore. We join Giorgio to discuss his time at dBs and the exciting project he’s been involved in amidst the pandemic.
“I get ideas all the time and the problem is, you only have one life. I wish I had the time to do them all, but I just have to choose. In my mind though, I'm not giving up something to go do something else, these things feed off of each other and become part of each other.”
Meet Giorgio, a recent graduate of our game audio degree and frequent collaborator with our in-house commercial studio dBs Pro. During his time at dBs, Giorgio’s passion for both sound engineering and interactive audio resulted in a number of incredible projects. Yet until now, we’d never had the chance to find out where it all began. As we join Giorgio for a virtual chat he reveals his introduction to music came at a very young age.
“When I was a baby, my older brother was playing piano and I think that filtered though and I started playing drums when I was very little. My mom told me that I was so small that even the baby drums were too big for me and they had to get the smallest drum kit they could find just so I could play it. I don't really have many memories about all of this; that's how early I got into drumming.”
Growing up in Northern Italy, he and his brother shared their passion for music, taking turns to monitor the radio for their favourite songs so they could record it onto cassette. When Giorgio was a teenager and his brother stopped playing piano, he went on to form a metal band with his friends and it was here that both the need and desire to learn more about sound engineering became clear.
“I realised that I loved both the musical and technical side of things, but in Italy, studying music technology is not really recognised the same way it is here in education, at least not at the time. I was lucky enough to get a place at probably the best place for sound engineering in Italy; APM. It’s an historical school based in the ancient Monastery of the Annunziata, in Saluzzo.
“It’s also where I met Marco Migliari, who is an engineer that used to work for Peter Gabriel at Real World Studios. It’s a funny story, but he's pretty much the reason why I'm in Bristol. He came to our engineering school to talk about game audio because he engineered the soundtrack for Halo 4 and that was the first time I heard about it. I started thinking, ‘Wow, this sounds really cool. It's nonlinear, it's always different - it's interesting!’ A few years later when I wanted to come to England to study, he was the one that mentioned Bristol to me and it just so happened that dBs had the perfect course for me.”
Let’s get interactive
With the fires stoked to pursue interactive audio and a place confirmed at dBs on our game audio course, Giorgio spent every moment creating, both independently and in collaboration with dBs Pro, which at the time had recently relocated to Bristol.
“I was harassing Jay [Auborn] (Creative Director at dBs Pro) and constantly emailing him to get me involved on a project. He finally gave me a trial doing some post-production on a video trailer for Joss Stone’s latest album, Project Mama Earth. It was really already mostly mixed and I only gave it a tweak, but Jay was really impressed, and from there I was involved in so many projects.”
Though his work at dBs Pro began in engineering, Jay quickly recognised Giorgio’s passion and aptitude for innovative audio projects, and as more of those projects came through the door, he was an obvious candidate to get involved.
“After Joss Stone, Jay sent me to Plymouth for a 360 audio project alongside Kanghyo Lee. Then I worked on U+I Stories, a project we did with Mayfield Manchester where we developed a binaural augmented reality app that allowed employees to relive the history of the building, and we made soundscapes for the space.
“Jay tended to give me all of these more experimental interactive projects that were related to my course, but he also brought me in when he was composing and needed someone to handle the sound engineering side of things, such as on the Sony Entertainment feature-length film, ‘In The Cloud’. That's how I ended up in Sofia, Bulgaria for the orchestra recording, which I was very fortunate to go to again for another project a couple of years later.”
Giorgio on location in Sofia (right), John Matthias and Giorgio recording in dBs Pro (right), Giorgio reviewing his parts for the all synth performance of 'In C' with Adrian Utley
The trend continued throughout his degree, and Giorgio's role within dBs Pro grew and grew. In November 2018, he joined several other students and staff for a very special all synth performance of Terry Riley's 'In C' that was curated by Portishead's Adrian Utley.
Giorgio also played a key role in 'Between Us' - an augmented reality project developed with Dr. Jane Grant, Coral Manton and fellow dBs student Phil Liford. Utilising the Magic Leap AR headset, Giorgio and Phil were at the forefront of developing and programming the soundscapes featured in the project - a formidable task considering the headset was still a prototype and not readily available outside of the US.
Alongside his involvement with dBs Pro, Giorgio was hard at work pushing the boundaries with his own work.
“During the first semester of my third year, I was thinking about what new technology I could utilise as part of my project for the innovation in sound module. My research led me to EEG headsets, which until recently have been used exclusively in scientific research using wet electrodes to scan brainwaves. As the technology has improved, you can now buy cheaper EEG headsets that use dry electrodes that give you very basic signals from the brain.
After this discovery, Giorgio realised that these cheaper EEG headsets could be paired with a virtual reality headset and from there, his brain waves project started to take shape.
“dBs kindly bought the Muse Brain-Sensing Headband for me to use on my project, which could be worn under the Oculus Rift virtual reality headset. It was pretty clumsy but it worked. Then in Unity I made the software where the music was this tangible thing floating around you in separate stems and depending on your mental state, you could affect the sound and move them closer and further away from you depending on your alpha and your beta waves.
The moment Giorgio figured out how to use real-time EEG signals to control parameters in Unity
“EEG is quite complicated and it's not like it can read your mind or anything like that, it's very general, it's basically a bunch of frequencies that you can detect. If you have a lot of alpha waves it usually means that you have quite a relaxed state, if you have a lot of beta waves it means that you're concentrating on something. The others are even harder to detect. So it's quite broad, but these are just basically a constant number - a signal - that you can just transfer to Unity. I used it to control a low pass filter and the distance of this music, but it’s just numbers; you can literally do so many things.”
In that same year, Giorgio also developed ‘Machine Balance’; a project inspired by AI and machine learning, specifically the then-new Google Magenta.
“It was a very simple game that you play by waving your hand above a motion sensor to balance a ball on a board. The music, which is triggered within Ableton, is made by artificial intelligence and reacts in real-time to the position of your hand. It was a great experiment that pushed me to explore how this technology can be used in my workflow and other projects.
“You learn what you can and cannot take out of technology, but if you really want to make it work you figure something out. That idea is at the heart of why I love interactive audio so much. It involves a lot of problem solving, almost like a programmer, and you get that same satisfaction of achievement when something doesn't want to work and then eventually you make it work.
Machine Balance being tried out at the Game Audio Symposium 2019
“That also exists in sound engineering, but to me sound engineering is a much more established craft. There are just so many techniques and decades of knowledge from those that have come before you; everyday you're doing techniques that have existed for years. It’s more about drilling down into the details and perfecting it, whereas interactive audio is much more, 'Wow, even though it's not really working amazingly well, the result is already really cool.”
Two worlds collide
Though Giorgio’s time as a student at dBs came to a close back in 2019, his involvement with dBs Pro has continued as an alumnus - a common occurrence for many of our graduates.
In March 2020, just days before the UK government announced its plans to put the country on lockdown, Giorgio was recording a very exciting project at dBs Pro; one that would see his expertise in sound engineering and interactive audio coalesce: Koor.
Inside dBs Pro for the recording session for Koor back in March 2020
“Koor is such a great idea that was founded by conductor and filmmaker Simon Capet. I don't know if you’ve ever sung in a choir, but I tried once to improve my skills - I'm a very terrible singer [laughs] - and the problem is they never have real recordings of what you're singing. You have the score, or if you're lucky, you get a terrible MIDI MP3 that plays you the melody that you're supposed to sing to with some horrible noises.”
“Koor tries to avoid all of this by recording professionals singing this repertoire and the app presents all the component parts as separate stems. So you have bass, tenor, alto, soprano - even soprano one, soprano two, depending on the parts of the piece. You can study your own part, remove it and sing with the others, listen to the whole piece and even mix the vocals and musical accompaniment. You get so much more detail that you could not get unless someone actually sings it to you. Things like pronunciation, intonation or where to breathe are so much easier to understand; it’s not just volume and pitch.
“I’d been asking Jay for any work after graduating, and it just so happened that dBs Pro had been recommended to Simon, but Jay was too busy to oversee the project. So he called me up and said, ‘if you want to take this, I’m more than confident in you.’ He came into the first session, but saw how things were going and left me to run it, which was such a great feeling for him to place that trust in me.”
The Koor interface
In what would be an incredible coincidence, Koor was not developed as a response to COVID-19, but instead as a tool to help choral singers who struggled to read music practice at home. With the support of several crowdfunding campaigns, the project was able to grow, with Simon opening a studio in Islington to continue development, and Giorgio was coming along for the journey.
“When he [Simon] started to move from Bristol to London, instead of just getting a local engineer he insisted on bringing me with him because I just happened to have the exact interests and skills to match what he needed.
The project is mostly sound engineering, you need to record high-quality performances from the singers and the MIDI from the piano. Then mixing all of this together is huge, it’s a massive recording and you're trying to give every part a decent sound. But it's not traditionally delivered as in, I give you a stereo recording of Beethoven and you just listen to this nice recording, you need to deliver it as stems.
“You don't really mix it like you mix normally, you can’t just rely on the end of bus, because there is no end of bus (ed. a bus is a signal path that combines multiple audio tracks). The end of bus is the choral singer’s laptop who's practising in their bedroom and trying not to get distracted by the noises of life. That's really quite challenging, but it’s also what I really like about this kind of work because it’s forcing me to problem solve.”
At the time of writing, Koor remains a browser-only app, but developments for a mobile app are in the works. It’s free to use and has several repertoires that can be accessed to practice along with: https://web.koor.app/
The door is open
For Giorgio, his exploration into the world of sound was first informed by his love of music and everything blossomed from there. Yet he’s quick to point out that interactive audio is open to anyone with an interest in its applications.
“Interactive audio is a good field to get into, as long as you’re passionate about it. Now is a great moment as well because it's growing and growing, and it's only going to get bigger.
“The medium is really open to so many things. You need to develop skills and understanding in sound engineering alongside it, but interactive audio doesn't have to be musical. When you realise this, then every other passion you have in life, which could be endangered animals, or protecting the environment etc.; whatever you do in your spare time could become the foundation of a project.
“There are some really interesting installations with plants. I've seen some in Manchester where they have sensors detecting life parameters of these plants and they produced a soundscape based on how the plants were feeling in the moment, which is so cool. There's so many things that you can come up with, so probably the best advice I can give is to explore as many things as possible.”
Alongside his work in interactive audio and sound engineering, Giorgio is also the drummer for Bristol-based band Locomoss and performs with Theo Mizú.
FIND OUT MORE:
Giorgio Cortiana - Game Audio Showreel
Instagram
Are you excited by interactive audio? Check out our BSc (Hons) Sound Design & Audio Programming degree.