 So this is a talk about modal synthesis using Synthemodeler. Synthemodeler is software for doing physical modeling sound synthesis, which is a technique where you use the physics. You simulate physics. Oh, I'm running through. So if you use a computer to simulate the physics of a system, you can in real time you can use the physical variables to synthesize sound and And and that's an interesting approach you can use for making music because It's just another you can have a lot of different controls for controlling the sound and you can explore Different ways of doing that and create new timbres that you might not discover otherwise And so it's kind of an interesting thing to learn a little bit about and so that's why We had suggested having this talk so Pascal Kopp and Zach Berkowitz have both worked on this And Some of the text is in German and I apologize. I guess I'll just read it in English though. So An introduction to modal synthesis What is that? It's when you Take something that's vibrating and and and model its resonance frequency characteristics using a computer. So Basically, there are a bunch of frequencies that are decaying and for each frequency you have You know what the frequency is in Hertz. So how many times it oscillates per second? You have a decay time, which is a Measured in seconds which describes how fast something is decaying. So if it has a long decay time Then it takes a long time for the sound to decay if it takes a short If the decay time is very short then the sound decays very fast and it's also important to have an amplitude and so you can take Any sort of uh, sort of impulsive sounding sound and decompose it into Frequencies with decay times and amplitudes And in this case so the if you're doing this use in syntha model or using the Resonators object, it makes a certain assumption which is that basically The the system that you're modeling starts from a zero displacement So it starts with an initial velocity, but the initial displacement is zero. So that means the sound is described as a summation of Exponentially decaying sinusoids if you have a different Boundary conditions, then you would also have some decaying cosine terms here But in this case these are eliminated For a somewhat complex reason, but basically you can you can guarantee Something about the phase of this system Which means that in syntha modeler then when you connect all these different objects together it will be stable Whereas if you don't meet the phase requirement Then you might violate the nyquist stability condition and then it goes unstable So that's something you want to avoid. So that's why these models are limited in this way Here are some more pictures of this actually and so often If you're studying mechanics or Newtonian physics You would think about what is an exponentially decaying sine wave. What is the corresponding mechanical model for that? Well, it's a mass connected through a spring to ground and there's also A damper connecting the mass to ground so you can kind of imagine sort of setting that mass in motion And it moves up and down for a while and it stops And if there's just a little bit of damping then the decay time is very long If there's a lot of damping then the decay time is very short And in syntha modeler we Incorporate all the friction into the links themselves. So in syntha modeler, this is just a mass Connected by a linear link to ground Are there any questions about the basis? of modal synthesis Yes No, I guess so. I guess so. Yes Now, yes I was wondering there was written Sucio acoustic or psycho acoustic maybe on one of the slides or maybe I was mistaken Yeah So are you going to explain the psycho component or do we stick to the to the physical stuff? Well part of part of the psycho acoustic part, I guess is is You know this assumption here that you can have a finite number of Exponentially decaying sine waves to approximate certain sounds other sounds can't be approximated Using what I mean if you imagine some sine waves that are growing in amplitude How could you represent that using exponentially decaying sine waves? It's it's It's not a good idea anyway To try to do something like that but um Yeah, certainly you you You um, you only need A certain number of these and maybe you could get rid of some of the quiet ones and you wouldn't you wouldn't hear it so if you're editing the syntha modeler code and you see that some of the Exponentially decaying sinusoids have a very low amplitude. You could just take those out and it would sound very similar And so that's I guess what I mean by psycho acoustic modeling in the field of audio Uh computing we do a lot of psycho acoustic modeling and using it to Reduce the amount of cpu for example that is required to do something And so that's um, that's that issue there so this the this particular work pertains to um Some of the one of the models really in particular in Inside the syntha modeler library which is if you have For certain systems you can analytically solve for what the frequencies are Which is a really handy thing actually it's really great when you can just solve a differential equation And you don't have to do it numerically And so people have done a lot of this and for for geometries where that's true Um, you can just look up in a text what all the resonance frequencies are But you can't look up the decay time So there's a question Or at least I don't know of where you could look up the decay time It's possible There's a paper or two written out there somewhere where someone has related the the material parameters to the decay times For some certain systems, but I haven't seen it much and this was actually something we experimented a lot with Because in a lot of sort of musical sounding sounds the high frequencies decay faster than the low frequencies That's true for a lot of drums. That's true for a lot of strings sort of percussion instruments in general And I can I can at the end of the talk I can show an example of something where this is not true And you'll hear that it sounds kind of funny. And so we asked ourselves what would be a good way to incorporate this into the model itself and What we did was we looked at a text the text by Rossing and Fletcher On musical acoustics and in that book there's a section on The decay of the partial frequencies in a vibrating string For um, I believe it was a classical guitar or in any case it was some kind of acoustical guitar And they related the different Physical mechanisms that could cause the decay And if you looked at that you basically saw that you had something like this where some of the t-60s at very low frequencies some of the t-60s It would just be constant in frequency So the decay times were all the same for the really low frequencies And then suddenly there would be a cutoff frequency Where some other behavior would become more important and then You know, there would be some other loss in the system that would cause the high frequencies to decay fast so um I think one of those sorts of things is actually the Let's see if I can remember. I think that there is some energy radiated from a string directly into the air actually not through the body but a small amount and because of the The dipole characteristic that happens more at high frequencies. So basically there's some frequency at which that becomes important and then the Decay times become shorter and so um In that text there are actually two cutoff frequencies, but we found Sort of just listening to our models that to first approximation You know, it was this first one that really mattered a lot And the other ones didn't matter so much And so that could also be described as a psychoacoustic approximation that's made In terms of this model, um, you know that we didn't model the next cutoff frequency where it gets even steeper because These frequencies all decay so fast. It sounds kind of similar anyway But i'm sure you could cook up some example where it's important to have the other cutoff frequencies But you know, you've seen the models now. You've seen that if you have 30 sliders to be adjusting Uh, you'll just go crazy at some point. So we try to limit What you could get In in that way and so, um, the rest of the talk is just describing The rectangular membrane model In syntha modeler, which looks like this. So here's the resonator's object And so that when you to create an object like this, you have to specify a bunch of resonance frequencies decay times and uh equivalent masses For each of those decay times And then once you specify those you can connect it to these other objects. So In this case, it's assumed that you're plucking a modal synthesis resonator object in this way And so that's why there's a plectrum link here that's connecting that to a port Yes Would you care to explain more like or at least the Right now, I don't know what the resonators. What is the the physical equivalence of the the resonator actually you? Um Yeah, that's a good question I guess I would call it a a a driving point impedance Is what I've seen it described in the in the electrical engineering acoustics textbooks Which basically is is an impedance whose phase is bounded between Plus and minus 90 degrees If that makes any sense it's it's any you could this is any any transfer function you could measure by Um exerting a force on a point and measuring the velocity of that point in response Is a driving point impedance you can measure that on the bridge of a violin For example by exerting a force on the bridge of a violin and then measuring the velocity with which it moves So that's that's how you would estimate a driving point impedance And you can measure that for all sorts of structural vibrations So does it have like one parameter which is the force uh with which um I don't know with with which the the string is pulled or did I completely got that wrong? Um This would have a lot of parameters. So basically that is described by all these parameters Right here The resonators. Yes. Yes. Okay Did I answer your question? Oh Just go on. This is this is a this is a time domain um equation but you could use the foyer transform to Calculate what the frequency response of this is for example And that would then correspond to an impedance that you would measure on a violin bridge or Um a drum membrane you could measure this on a drum membrane as well Um I guess I I see your point though that this is this is a big approximation You know assuming that you're modeling some vibrational system and you only have one input and one output is obviously a very It's a gross simplification, but you can still do a lot with it, which is why it's it's set up this way um in syntha modeler And so If you if you look in the code for this so those of you who've installed syntha modeler, you'll see um It's called modal membrane A modal model rectangular membrane, I believe and you'll see there's a line in the in the model code Which says this it just instantiates the resonators object with um Rectangular membrane simple, which is a function defined later and you specify How many modes you simulate in the x-axis and how many modes you simulate in the y-axis? So if you set it up this way you get um You get it set up with 40 modes so it'll model 40 resonance frequencies and 40 decay times If you do it this way So this is this is sort of the the line in the model code that I really like to tell everyone about because it it makes it really easy to Increase the order of the model if you want a more accurate sound or to decrease it If you want to save some cpu Because maybe you could use just a smaller number of of modes to capture the sound that you want to have And then And actually maybe what I should do is just show this file just so you can see it with your own eyes As we're stepping through it So this is the the mdl file that you can compile into whatever Target is you want to have and um Right, so basically what that's saying is that right here you can have you can set how many How many modes you want to have and if you want to model two modes in the x-axis and two in the y or one in the x And a hundred in the y then you can You can play around with it there and see see how it changes the sound You do have to recompile the code after changing that though because Um, if you've attended a fast session you will have seen the signal flow diagrams that get generated by fast If you change the number of modes that the diagram suddenly changes a lot in size so so that's that's this part basically and so The remaining slides just really explain a little bit more how to read these these mdl files So if you want to use it with a fire fader You should put two ports in the mdl file one is there's one for each of these But if you had a device with eight ports, then you would need You know more ports there and I see I did this again with um And then you you have to have a pluck link So then you need to define this connection here which connects res in one with dev one And so you get that by saying pluck and then you have to tell it a stiffness in a damping parameter and also The sort of width of the plectrum And an offset which is usually zero and then you give it a name and you just say what it connects you connect the resin one to dev one Which it has right here and then right here. I just wrote above this what the So this is a variable and you can define this variable using fouse code This is really convenient so you can write interaction stiffness equals h slider and then here's where you set the Pluck stiffness. So if you've used fouse, you're very used to writing these, you know When you define a slider you give it an initial value a minimum value a maximum value in a step size And the same thing is done for the the damping in the plectrum So that's like saying how stiff is the pick you're using to play this resonator Is it really stiff or is it is it really compliant and how much damping does it have in it? And then you get to set the material parameters for the model which is kind of fun. So you can set Um what the length is in the x-axis what the length is in the y-axis what the surface tension is in newtons And then also what the area mass density is um And so i'll show those models later, but that's basically that affects the timbre of the sound And then the final bit that you have to figure out in the code is how to actually calculate the resonance frequencies from these variables so this equation Calculates the speed of waves in a rectangular membrane with the tension of t and an area mass density of row And then you look up in your favorite acoustics textbook or if your hardcore you calculate it It's a good exercise actually this is one of the very first ones that will be in the chapter on this um, you can calculate the resonance frequency for the The mode you have to give it a name so you could say you know an index the mth and nth node A mode pardon me would be this then you could look up this equation Um Which is half times the square root of t over row times the square root of m squared over lx squared plus n squared over ly squared and if you if you but you could look up the modal synthesis Equations for or the resonance frequency equations for other Structures that have been solved analytically and and put replace them in this model And and it would be nice so Then yes Is there any Criterium From which or which can be understandable How to tell whether uh, it is possible to analytically solve the equation and get the frequency Not numerically or is that just like there are some models that are known that's a real common Um And I guess just which ones are solvable analytically um Yeah, I mean the same for the simple geometries You know, it'll be it'll be possible to solve them often But not always I I know I I know that when it comes to non-linear Differential equations. I had a professor. I used to always say, you know, there are 17 non-linear differential equations And if it's one of those it's analytically solvable and if it's not Then well, we don't know but no one's figured it out yet. So um So yeah, it's complicated and I You know, it it it would be nice if it was if there was an easy easy answer Sometimes you can approximate them also though Or maybe you could approximate two things and then interpolate between them If you wanted to kind of make a model that works Anyway, it'd be really easy to build a modal model that interpolates between two different shapes That would be kind of a fun thing to do actually So this part is might look a little bit like voodoo, but um for people who Who who knows Faust? Who's planning to learn Faust? Okay Faust, um There's there's a really nice feature in Faust that um Is based on concept of term rewriting that's really convenient in terms of Sometimes if you're writing something you can sort of get it to automatically figure out Well in this case, you know, I made it automatically generate the Faust the equivalent Faust code for this You know rectangular membrane 410 The first time I made this I just pasted a whole bunch of these after each other Just to see that it worked and see what it sounded like but To do that, you know, it's much nicer to just have it automatically do that And you can do that with term rewriting and and I'm showing you how to do this here and this is basically a recursion So who's programmed a recursive program before? So so you you basically want to just write out a recursion so that it can sort of figure out how to do it And this is this is what we did here. So you have to tell it how to terminate the recursion So you have to some so rms triplet Uh, basically tells you Here, let me look that up in the this So that rms triplet generates basically a triplet of frequency decay time and equivalent mass to put into Uh the the resonators instantiation But in order to get that Um It's just more convenient to define it in terms of rms triplet because then you only have to write it once And the point being and I did it again here The point being though when you when you write out this Recursion you need to also have variables in it. So you have to come up with a new name So I called it rectangular membrane simple recurse Which has four variables which are sort of the current the current n and m and then the maximum n and m so it knows how high to go And basically you have to terminate the recursion so that it stops somewhere Because you don't want the fouse compiler to generate an infinite number of lines of code because then you're going to have a problem And then And then you can kind of think of this as a grid basically and then you have to say you know if You want you've already defined, you know, what the one is back here in this corner And you have to define all these other ones in terms of the previous ones Um And if you've done dynamic programming before You'll be familiar with this sort of kind of thinking but basically When you get to the end of one of these rows you have to jump up to the end of the previous row And then if you're not at the end of a row you just go over one by one So it just sort of it enumerates them. Maybe that's a good way to put it. It enumerates them in this order Um so that you don't have to do it manually and and and and that takes care of it basically and so um, the only other parts that are in the model are the uh, definition of an h slider for volume and um H sliders for the excitation position And by the way, I want to add about volume if you've ever implemented a volume in Faust Use db2 linear. It's really handy if you use db2 linear it it lets you um, basically Set the units for your sliders to be in decibels And that gives you a better control over the range of controlling a volume level Um It's it's kind of like the difference between a linear potentiometer and an analog taper potentiometer So you're going to want to use those if you're writing Faust code for doing this Um, and then some comb filters are applied to the output sound. This is just sort of icing on the cake I guess but Some people would probably say it's not modal synthesis if you don't do this. So yeah, we add some comb filtering To so that you can strike the membrane in different positions And have the sound change Um So I guess what I should do is is show the model at this point So let me fire this up here Let's see. I'm gonna show it on this computer instead. I think it seems like the sound isn't on. Oh Why are those there? Okay, so there's the modal rectangular membrane and um If we zoom in it's a little bit easier to see what's going on in here So if if you're playing a membrane And you increase the area mass density Then the pitch goes down If you increase the length And the pitch also goes down if you So I was saying that the decay time was really important. So if you increase the decay time and you Increase the cutoff frequency Then it just lasts a really long time. But if you shorten the Decay time then it's short Like this But perhaps equally important So as I was saying we we're using this t6c to describe the sort of decay time at low frequencies But the decay time at high frequencies is also also important Like that sounds kind of unnatural to me or Or it sounds like some weird bell, you know, you found somewhere Which might actually be kind of cool But you can change the cutoff frequency that way And then it has a much Darker tone Sounds more like a drum maybe But it depends on the dimensions, of course. This is maybe too oblong. This is too elongated to be sound like a drum So if you make the x and y length more similar Then it might sound more like a drum, but you know, it depends on exactly how similar they sound and It would be interesting to do more psychocoustic tests to learn, you know What things actually sound like drums and what things don't You know, what what parameter sets? um But if if we were to go change the the model complexity Well, that's kind of sounds like cutting off more and more modes basically, but because we already have 40 modes We already have a fairly rich sound Which is nice so What else Is there to say an interesting thing about synth the modeler though is once you have A simple model you can start connecting more stuff to it So this shows some masses on top of the resonator So if you want to add some snares to your rectangular membrane that you're you're Simulating You could do that. So there's an example model that does that also Which is modal rectangular membrane with snares Too fast. I think there's a new mode actually to synth the modeler now, which will let you automatically embed If you're simulating a fire fader it will let you automatically have sliders to represent the fire fader But it's kind of annoying if your patch assumes you don't have that On the other hand if you're compiling your modules for super collider It's really handy because then you don't have to if you're someone like me And you don't know how to use super collider you can get it to just already put them in there the way you want but Here's what i'm going to do So i'm going to reopen that and then i'm going to Touch this file so it thinks that i changed it and then it'll recompile it there, okay Yes, and this is with Faust gen which is kind of a neat way to compile Faust code This has the Faust compiler inside it actually Well, maybe I didn't sufficient anyway, so now it's down here So I can do it from here I think and actually you can see the snares sort of bouncing on top of it right there So that's kind of interesting and it really the sound changes a lot depending on How you set the mass parameters So if you set the snare masses to be A lot smaller then the sound will be different Or if you make them bigger Then it'll change Now they don't really sound like snares anymore It's it's interesting. It sounds kind of a little bit like reverb to me kind of like a non-linear reverb Anyway, when the more when you start hooking more stuff together and synth the modeler then you can get more sounds out of it And that's how you do it all you have to do is add those masses and then finally An interesting way to think about making music is to have You know, maybe have have some of those Have something happening in your physical simulation that's happening at a slower rate that automatically generates a score of music That's that's a nice approach to take Um So thank you very much for your attention One last question before lunch. Yes I've been programming a lot of fast programs Based on well, basically variations in a decoder and a lot of them have Oscillators that already have a formant And I was hoping maybe one of these can be used as a oscillator with a formant Maybe That would bring some interesting sound possibilities. I think That would be a very interesting example. I I haven't made an example of that yet, but I hope someone will do that soon All right But yeah, a fun thing you can do with these models is put an audio signal into them And have it get convolved by the impulse response of the model. So that would be a neat a neat approach for that Yes, great Thank you very much then