 Short introduction because he will show very cool things. I'm just gonna say it's gonna be great talk Ariane Lara Moreno, please give me a very big hand have fun Good evening everybody and thanks for joining my talk. Maybe that's a lot late in the evening I know it might be a little bit tired, but I hope that probably hopefully it will be worth it for all of you I'm going to talk about how to successfully build and program some field control systems My name is Adrian, and I'm the head of research and hollow blood. I've been in the company since already more than ten years So there's plenty of much that we can tell you about I Would probably not be wrong by assuming a bunch of people in the audience area have had some kind of music making experience Or do actually make music so Music music making is an exciting creative process where pretty much there are no boundaries set I would like to distinguish two separate Processes and in the stage of making music Not strictly independent from each other, but not always really Properly interconnected Those would be the music production and the music reproduction In the music production phase is where creativity plays the biggest role and where artists take the leap in adopting and driving innovation synthesizers and samplers did revolutionize the way music is being Produced and expanded the creative capabilities of many artists in ways. They could not imagine before New software driven digital signal processing filters or effects also introduce new unique ways of personalize your production Some artists even made themselves some style or signature that they are well known and recognized for One example of them would be daft camp, daft banked with the use of the effect of a vocal other Traditional music creation using instruments. It's also constantly finding new ways of evolving and combining new instruments and Finally, there are many different ways of interacting with music and triggering sound using very uncommon Objects such as for example the steps on a staircase or use sound Actually an stimulus for other effects such as video lights and many other One great example of interacting with many different of sound interacting with many different materials that would be from the video clip of Nigel Stanford called music versus science, which I really recommend you to take a look at Now that we got our production ready. We surely want to reproduce it and show it to the world, right? There are plenty of different reproduction systems that you can use for that Among the most common ones you can find mono systems stereo surround or even atmos or ambisonics in state-of-the-art projects The biggest disadvantage of these kind of systems is that even when correctly deployed They only deliver the desired experience to a very reduced number of people in the audience That is also what is typically well known as the sweet spot Very few people know actually where the music is going to end up being reproduced and even fewer people take into consideration the sound system for the music production phase Actually the chances of your music production being Properly experienced as you might have conceived it in the production phase are quite low Which actually brings me to my next questions and more or less the topic of my talk, which is What if the loudspeaker system would take part in your creative music making process? I would like to introduce you to something control systems Particularly those with 3d capabilities as it is the case of hollow blood, which we are hopefully Will get to experience today as well The idea of using array of speakers to shape sound using beam forming has been around for many years But until now it has been very difficult to achieve and put into practice both for the horizontal and vertical axis simultaneously Beam forming is a technique that implies the usage of multiple loudspeakers and interference between them and They would put many of these speakers in the form of an array a line array or a matrix array in case that it would be like a two-dimensional speaker surface They also are commonly driven by a digital signal stage, but I will talk more about this a little bit later With beam forming one can control the size shape and direction of the sound field of the loudspeaker But before getting a little bit more into detail of how beam forming speaker systems Function I would like to introduce you to a couple of acoustical theoretical principles that would help you understand how these systems work To understand how a loudspeaker can interfere with another one when they are built in the line array Let's first take a look at what the radiation pattern of the loudspeaker looks like Let's have a look at the radiation of a single cone loudspeaker and we will take the example of a four inch Loudspeaker or driver or a trans user as I would use any of those terms For the same purpose, which is a loudspeaker at low frequencies the wavelength of the Of sound is way larger than the size of the driver So the resulting sound field from the the trans user at the low frequencies will be Spread and spreading out evenly in all directions with the same level At mid frequencies when the wavelength that we are reproducing is more or less the same size Of the driver were that we're using to reproduce sound You will hear louder sound in front of the loudspeaker and if you move sideways the sound level will decrease a little bit At high frequencies when the wavefront is much smaller than the wavelength That we are reproducing Sorry at high frequencies when the wavelength is way smaller than the loudspeaker The radiation pattern would look as it's shown in the picture, which is it would be quite Having a very rapidly level decrease a few move from the oxide from from the home axis different position from the loudspeaker now the relationship between the wavelength and the frequency is given by the following formula where Lambda equal C divided by F where lambda is the wavelength that we are Reproducing C would be the speed of sound typically Value having values of 340 meters per second and F would be the frequency equivalent in from that wavelength as well As a summary from this slide One could say that the loudspeaker radiates Have a directivity back then that looks differently for all different frequencies As a consequence the the beam or the sound field or the coverage for that loudspeaker will look differently Depending on the direction where you are actually listening to that loudspeaker If you could be for example on axis like listening right in front of the loudspeaker You will hear all frequencies with more or less the same level But if you will move sideways you will find out like having Or you will experience different levels between the low frequencies and the high frequencies Now that we know how a loudspeaker radiates sound at different frequencies Let's take a look at the interference of two sound waves Mostly when they are both overlapping with each other. So interference is a phenomenon that describe When two waves overlap With each other and they form another wave which might having a louder higher level or a lower amplitude level Two waves of the same amplitude and frequency Which are in phase interfere in a constructive manner Resulting in a wave of double of the amplitude. This is also known as constructive interference When two waves are 180 degrees out of phase They interfere in a destructive manner and Cancel each other completely producing sound silence This is also called as destructive interference Advanced loudspeaker arrays rely on the principle of controlled interference That is based on the precise manipulation of phase between the loudspeakers built in there in the line array a Couple of slides before Remember we learned that a loudspeaker radiates definitely for load meet and high frequencies If we want to cause interference between several speakers in the array We need to put them close enough so that the dispassion button overlaps physically and they can accordingly interfere with each other To understand how wave interference occurs in space Let's take a look at the couple of examples using a monopole source monopole source is a Theoretical infinitely small sound source that radiates energy with equal level in all directions for all frequencies The picture that I'm showing right now. It's the the diagram the activity diagram or the radiation pattern of a monopole source which from these pictures you can see on the top one that would be the The pressure level where you can see the wave front Which basically tell us how the wave Troubles across space and time and if you would take that same information and Take a look at the the bottom diagram that would be the equivalent level of that sound wave When when it's been taken in a DB scale that would be what is also known as an SPL map Now if you would place two sources with a distance of half of the wavelength That we want to to control or that we want to interfere with That would result in constructed interference in the front and the rear directions Whereas it will also cause destructive interference in the 90 degrees directions that would be on the left and right Axis from the loudspeaker Now using four monopole sources also distance half of the wavelength from each other even more narrow radiation pattern will result in the front and back radiation and In all the different directions we will have like either constructed or destructive interference Depending on the angle where we are positioned It's also important to know that the amplitude of the Resulting narrower beam if you look at the latest SPL map It's louder than the original one from the monopole source as we also land in the constructing the theories interference slide before Now the radiation pattern of an acoustic source is given in a certain given plane in this case the the horizontal plane If you would cut a horizontal section of that group of loudspeakers we will end up with what I'm showing in this In this plot here, which is called the which is also called as the activity diagram In there we can see many different components. The main one would be the main lobe with the main beam which is basically the lobe of energy containing the highest energy in all directions Typically shooting in the direction of interest for example the audience area Grating and side lobes are beams with less energy that travel in undesired directions And they are the result of special aliasing effects That typically happens when you put deposition trans users to distance from each other so they don't really and Properly interfere with each other in a constructive matter The the activity diagram is widely used to represent the dispassion behavior of being forming loudspeakers Now line arrays combined the acoustical output from Multiple speakers to obtain an arrow beam in the front direction of the line array Sunfield control Using line arrays is achieved using many different techniques For example, we could mechanically tilt the array so that the main lobe travels in a different direction The direction where we want to to cover at the main audience area Or alternatively we could build many loudspeakers in that array with different sizes and dispassion buttons and Also tilt them mechanically with different angles so that they effectively cover a wide audience area One example of this type of loudspeaker Probably you're already familiar with them that would be these also commonly known as banana shaped line arrays or also J shape that combined that that effectively combined the Different loudspeakers mechanically tilted differently with different dispassion angles another approach to control the main lobe would be not using mechanical steering or mechanical tilting but actually having still a flat line array and Including a digital signal processing stage Also called as a DSP stage in between the signal input and Line array that we're using to control the main beam This principle is also known as digital beam steering as obviously it involves a digital signal processing stage in this process Now if we zoom in into the required DSP that would be just one example of what we can use We could steer the main lobe to a given direction in this case theta By applying a delay to each of the loudspeakers built in the array The delay for each loudspeaker channel can be calculated using a the formula that I'm showing in the slide Which pretty much is dependent on the distance between the loudspeakers and The angle that we want the beam to be steered into now as a result if we look at the directed diagram on the right hand side The main lobe is now formed to travel in the desired direction theta Which is defined from the on axis angle of the line array Since we are changing the radiation pattern by using controlled interference That effect also the complete radiation pattern or the directed diagram of the Of the line array so as a result not only the main lobe is Steered into the desired direction but also the rest of the artifacts produced by But these kind of beam steering are also moving into different actions. I'm talking precisely about the Grating lobes the real lobe and the side lobes As you see also in the diagram. They are also are coldly steered Depending on the steering angle now I would like to summarize and lay out the differences between different sound field control systems But first It's not really a proper Sound field control system per se, but it's a loudspeaker the point source But nevertheless, I think it's it's worth to mention the behavior and the differences So the first one that I'm introducing our point sources Which they do not actually offer any kind of sound control in space other than the one where That is being decided during the design and manufacturing process of that specific point source speaker Then we have digitally driven line arrays that offer the possibility to control the main lobe in the Direction where the array is built in In the example that I'm showing in the slide the line array is built on the vertical direction. So as a result the beam steering will be Capable of being achieved in that direction. So we will be able to do beam forming in the vertical axis with this line array and Finally many loudspeakers can be stacked not only vertically, but also horizontally Resulting in what it's also called as a matrix array Matrix arrays Benefit from some field control in the horizontal and vertical axis independently Resulting in full 3d control of the beam in all directions Hollow plot would be an unique example of a 3d sound field control system Now I would like to mention a couple of considerations needed to understand the beam forming at different frequencies When using sound field control systems, the highest frequency control is defined by the driver spacing Probably you might remember from the slide of the interference across space As we learned previously The control phase interference is possible when the distance between two loudspeakers Enables the resulting Directivity pattern to overlap physically In that case we need exactly to have at least half of the wavelength of the frequency that we want to control In the activity diagram that I'm showing on the top right hand side The full control of the main loop is possible without any additional artifacts And that would be possible as the example frequency that I'm visualizing would be below the frequency control of that specific matrix array design On the other hand on the bottom right hand side the example frequency would be above the frequency control of that matrix array So as a result we Can control the main loop into a different direction But additionally we would get also some artifacts resulting from under sampling of the wavefront Because of the distance from the loudspeakers would be too far for that frequency that we want to control On the other hand on the other side of the outer spectrum on the low frequencies Because the wavelength is so large we will need to have speakers Spaced enough and sufficiently to be able to to control that low frequencies Specifically again that would be at least half of the wavelength. So that would be at least what we need to have our array built in So the lowest frequency that we can control with matrix arrays would be given by the actual dimensions of the matrix array In this case since matrix arrays have two dimensions the horizontal and the vertical One could Define separately the Low frequency control in each of those axes To give an example and using the plot that I'm visualizing on the right hand side One could control the opening angle of the hundred Hertz frequency beam and Narrow it down to a 60 degree opening angle so to achieve that we one would need a an array of four meters width on the horizontal axis and The other way around if one would want to also control or like come up with the required length on the vertical axis The same plot can be used next I want to Show you some Features of 3d something control systems and how they can be used as a creative toolkit for your content creation But before jumping into the creative side I would like to Mention some of the inherent benefits of 3d something control systems That put them actually in a very advantageous position when compared to other sound systems such as obviously point sources or even line arrays First thanks to the being forming capabilities in the horizontal and vertical axis independently The dispersion pattern of the beam can be defined To precisely and optimally cover whatever audience area that the user can define That would be the beam can adapt in terms of size in terms of shape and As an example, I'm showing the diagram For the SPL plot as well resulting from a simulation of a matrix array on the right hand side Where I wanted to show this one because this is particularly challenging as it is exactly doing the opposite as the nature of some waves propagation would tell us which is a typical sound source will spread with distance and We reduce level in that being forming that we are visualizing on that on these plots The the beam is actually converging on the far distances also increasing the level and on the closer distances the The opening angle is quite open and or quite wide and The level has to be accordingly reduced So this is as I mentioned before doing exactly the opposite as what it should be doing to be considered loudspeaker So Because of that precision in delivering sound only where it's needed the reflections from badly Treated venues or spaces are minimized resulting in an increase of the direct to reverberant ratio In that specific venue from that system As a result the speech and delivery the index will increase Delivering crystal clear messages to most of the audience area even in very challenging acoustic venues Just to quickly mention the speech intelligibility index is a value between 0 and 1 that describes the quality of Specific system delivering a message in a specific venue. So obviously the higher the better Finally and directly resulting as well from the precise sound field control of our systems The noise pollution can be greatly improved Meaning you could run Shows with like either outdoor or in inside the venue with less noise pollution And you could even run multiple shows in parallel If those would be played at the frequency where the array can still control this frequency Now finally, let's take a look at how we can use hollow plot or any 3d sound field control system As a creative tool to enhance your show production with effects and features that you might not never thought about before With hollow plot you can configure multiple beams simultaneously from the same array Each with its own content eq level and shape You can easily deliver Content to various locations using beams with different properties some of the applications of this feature for example could be running a Multilingual conference room by physically splitting the room into different Beams with different languages Another example would be delivering information around exhibits in a museum or Divide of showing delivering different beams into adjacent platforms in a train station Hollow plot sound field control algorithms are capable of tightly focusing audio content Onto small groups of people even down to individual members in the audience area This is unique and powerful tool For creating targeted sound experiences in the venue experiences that actually change as you move across the audience area So basically what you hear what you will experience with the pen and where you are To be some examples that Where we could use this feature What we have done in the past is to recreate and the virtual acoustics of a specific venue For example a church and we have delivered like a single message to a single person in the audience area But only that person with here and the rest of the people with here just the acoustics of the venue Another example would be That we have used quite often in trade shows we had our from our booth that was more or less in the middle of the of the space in the trade show and we were like shooting a Focused beam or focused Targeted area to the main entrance of the hall and we were greeting and cheering up people to come to us as well So this is in great indeed a great tool for sound designers as it really create truly novel Experiences that were not possible before Hollow blood also develops and deploys way free synthesis algorithms I didn't want to get much into detail in this presentation But basically the array Synthesizes also concave wavefront that converge in one specific point in the audience area As a result the sound pressure level at that specific point is way louder that any other position in the audience area That is also what we called and focused visual sources Those focus with those horses can be placed on the audience area creating a proximity effect Something similar as if someone would speak to you at a close distance But you can also place those focus visual sources on walls or surfaces in the venue So that they reflect back to the audience area Something similar as if you would use a mirror to reflect light to a different direction These targeted reflections are localized in the wall and not from the matrix array So as a result You can experience them as if they will be like one additional Sun source or a slot speaker that you can use for effects So basically even if you have a centralized systems, you could still deploy a fully immersive experience in The venue that you are deploying that system alright now that you know enough about some field control systems and You might want to build your own prototype one or your own. So this is what you might need for it You need to get your hands both on the hardware and software development since they both are Dependent from each other and also a requirement on the software end You will need to implement the beam from an algorithms required to make the calculations that I described before I Mentioned before that very simple approach to do beam steering, but there are many more available in the Literature and on the internet that are either more complex or adapted to matrix arrays as well So before you have implemented the or after you have implemented the beam from an algorithms You might be you might want to control them with the user interface Some typical parameters might be of course opening angle or steering angle and Those will be will as a result trigger new calculations of the beam from an algorithms Those resulting coefficients will be applied into the DSP stage Which is the one in charge of processing the audio input to the system a Good selection of DSP blocks that are required to do beam from it would be gain delay and filters such as FIRs or IIRs Depending on the beam from an algorithm that you pick you might need some of them or all of them Now of course our purpose is not to process a single loudspeaker, but to actually Build a something control system So you're gonna need multiple of those and channels being processed in parallel one for each of the loudspeaker built and mounted into your matrix array And finally each of those Resulting Coefficients from the algorithm will have an update into each of the DSP blocks whenever a change happens on the user interface And also triggers a change of the coefficients required Finally, we need to feed all the resulting channels from this broad DSP processing stage Into the amplifier so that we can accordingly and fit the signals into each of the speakers of the matrix array individually and We need to have a multi-channel amplifier as of course each of the signals is Individual for each loudspeaker now if you feel like build building everything from scratch is too much our Friends from hackaday have a nice project description for loudspeaker face the race and of course If you're not afraid of complexity you can go beyond that and build the matrix array your own Here's some old memories from our first prototype back in 2012 As you can see already in the complexity it's quite large and and there's not even any electronics involved in there yet Well now that you understand How great some free control systems are you know You might want to know what is required to build your own system But most importantly I would like to give you as well an overview of all the complexity and the challenges that we had to overcome during the last years in our company One of the main challenges is high-count numbers. Everything at hollow plot is large The number of loudspeakers the computer the computational power required for it the number of supported audio modules The IT infrastructure the data being transferred the number of audit channels being trans been transferred The number of bugs as well Well, that one is actually joke One thing that we've got quite used to in in our system is to to push the boundaries of every single components that it's built in in our system Now we like to give you a quick overview of the infrastructure that we have built So first of course, we need to have our matrix arrays which are formed by stacking audio modules horizontally and vertically The matrix arrays receive the configuration from the controllers Using a redundant network Also, the controllers are redundant Since this is one of the main requirements of our systems. They need to be robust and they need to be having no single point of failure The root the redundancy mechanism was in this case particularly challenging to build because having two controllers only would not provide The majority of of both Required to solve the latest status of the system in case of something went wrong The parameters of the system design and the ones required to achieve the array configuration Come from a planning software what we call whole blood plan Which can generate the projects required for the system both in an online and offline mode and finally the audio source Is coming from either the Mixing desk or digital audio workstation that speaks or can't transmit audio over IP All of the systems are designed to be decentralized and scalable Our biggest installation and project so far we have more than 2,000 audio modules in the same network so delivering redundant data and Audio to so many units in the same network is particularly challenging and And to give an example and many IP solutions that we had to To fight for or that we were considering don't support more than 200 devices simultaneously So this is like way a large difference And also something mentioning Perhaps which is interesting is that what I've what I've been showing here that we have come up in our system Took over more than three years of a team of 20 full-time engineers to build and of course This is not yet fully finished now. I would like to zoom in into one of each of these components to give a little more of information and details how upload plan is in charge of Configuring optimizing and simulating our matrix arrays It builds online and offline functionalities as I mentioned before and it's built using protocol buffers Over gRPC That's a decision we pick because of the easy subscription mechanism the back force compatibility That he offers and the efficient transmission that we can Get from transmitting data into the controllers It also builds a complex 3d environment that can do real-time simulation of our matrix arrays And then again remember any kind of simulation using as many trans users That are built in our systems is again specifically particularly challenging Now let's talk a little bit about the playback section of the system We are also working on a software that where we can control and render audio objects that are going to be reproduced in our system but the biggest challenges come from sending the Metadata of these audio objects in combination with the audio streams to all the modules in the system and that they actually received To each of the modules in a sample accuracy so that's something that we're still working on and This piece of software is also specific particularly interesting for and show designers as of today there is no even state-of-the-art plugins or softwares the only thing they can deliver is to send or to render audio objects in any other system, but none of the Specific features that our system can deliver. So that's also one of the tasks that we are currently trying to solve Now about the controllers the controllers are the core of all the holo plot Infrastructure it is based on Yachto and takes care of sending and collecting information to the audio modules for matrix arrays Providing also updates over the air both for the own system of the controller, but also for the audio modules it is written in Go and It's built with many open source projects Some of them we have even built our own and they are of course released to the community as well So you can you fit to check out our GitHub profile for that and Finally as I already mentioned before when it comes to the matrix arrays or the audio modules The biggest challenge is the in the audio modules come from the high number of Loudspeakers that we have built in there in each of them. We have almost a hundred loudspeakers per audio module Each other module needs to process up to 1.7 billion multiplications per second something which is even challenging for many state-of-the-art DSP platforms and Of course each audio module But that we have done also implements many sensors and many properties that we can track in real-time such as the temperature and Amplifier status loudspeaker status or a bunch of data that we captured from the module in real-time and we send back to the controllers and finally speeding or wanting to speed the development of Integrating by integrating third-party solutions is not always as efficient as everybody can think about So we deal constantly with With the boundaries and the limitations that these third-party solutions Opposed to us such as for example the available APIs that they deliver They are specifically on the edge of what they can afford of what they can allow and As we have like quite an extreme usage because of the again the high number of loudspeakers Finally, I wanted to also quick quick Overview about the testing challenge that we had something that we had to solve by I mean testing our system is quite challenging because we need to test the software We need to test the hardware as we also develop it and we need to test integration from all of them So we had to build or came up with that dedicated testing room Which is especially equipped with IT to automate the testing the compilation and the deployment of the new releases And of course as everything IT related We also faced the challenge that it doesn't matter how many deployments we with testing house Just clients and the users will actually end up always with something different that we haven't ever tested before Or any new device that interferes with live with something else in the system and last to Know as an overall conclusion for for this presentation Greenfielding, which is what we did is both great and challenging So don't be surprised if you end up Of you need to reinvent yourself across the way since you're gonna end up most certainly trapped in a corner quite often however Greenfielding allows you to pick cutting edge technologies that most of the times will actually make your life easier more efficient and Indeed it would be more exciting So now I want to that's the end of my presentation now I want to show you those Examples not not that you know as well the features that the system can do Let's see how You guys react to it. So for that I would like to Thank you. I Would like to ask you guys to stand up and I'm gonna just play something that we have prepared and let's see what happens So So as you saw Because we were able to split the audience area into two halves We were able to also deliver different messages as I mentioned before with May providing different content and yeah, we gave a different message to each of you We are gonna also show just a couple of other examples where we use the same functionality, but the different purpose So we're gonna show it also from the multiple language just a couple of languages for these audience areas Feel free to move or change the audience area if you want as you will also be able to experience the change When you move from one zone to the other one Now Of course, you might think that this is easy because you have like two zones and two matrix arrays So now we're gonna Do the same exercise we're gonna have two separate zones, but it's gonna be coming only for one of those arrays In this case, it's gonna be the left array, which I assume is this one Maybe that's the other way around and we're also gonna show some special effects That are being reproduced in each of these areas, but always remember it's coming from one of the arrays Then later on I will change to the other array so that you can experience the same Effect but coming from different matrix arrays Okay, here's the next one. I made a mistake and it's always coming from this array So that one is not programmed yet. He was the last one There's a couple of more just five more minutes. Is it playing? There you go All right time is out, so If someone wants to make an impression, it's now the time or I have another another demo for three focus points So either we make a question or I show the demo I have two votes. Who wants to ask a question? There's one there's one really wants and who wants to demo over There's more people wanting to demo, but not that much But like I'll give it for you the decision to you. Do you want to do the final demo? I think a lot of people are standing very curious. Let's do the demo. The question can come up. I'm gonna stay around then we can answer Yeah, so do the final demo because this is cool. Yeah So now we have three focus sources. You can find the crosses probably on the floor with neon duct tape Look for them There or somewhere in the middle of the audience area If I if I just can say it's really cool to see so many people looking for sound walking around and looking for sound It's I've never done that before but it's it's triggering and lots of people. Thank you everybody and If you're interested then look for us and as you know of my trunk here We are also hiring so if you're interested also talk to us. All right, there's everybody a very warm welcome for Thank you very much