 Welcome to the second round of talks in Europe, Eitan. I hope everyone is having fun. Oh, we have a very full room. So this is Ram Ratcham. I'm really sorry I pronounced your name wrong. And that was close. Welcome. Welcome. So tell us where you're streaming from. I am coming from Tel Aviv, Israel. Cool. Cool. How's the weather there? It's as hot as always. So Ram, please. The floor is yours. Take it away. Sure. Hi, everybody. Let me share my screen. Welcome to my talk. This is going to be a music oriented talk. It's going to be a talk about live coding and music synthesizer. So this is going to be a live coding talk, meaning I'm going to write code live. I'm going to explain what I'm doing. And I'm going to show some things about music theory. I'll give you some backstory for this talk. When I was 17, I wanted to learn to play bass guitar. I got my basic guitar right here. I'm going to show some stuff on it later. And I got myself a teacher to teach me how to play bass guitar. And he taught me. And he also taught me something interesting called overtones or harmonics. So that was when I was 17. A couple of years back, I remembered what he taught me about overtones. And I thought, can I take that, a new spy phone to make sounds that sound like a musical instrument using math? So I tried that couple of years ago and it worked and it blew my mind. I figured I'm going to do the same thing in front of an audience. And I'm going to explain what I'm doing and I'm going to blow your minds. So that's the story of this talk. All the code for this talk is in the GitHub repo. You can see here. And every commit there is a step that I'm going to do here. A little bit about myself before I start. My name is Ram Rahum. I'm a long time path on developer. Many people know me by my projects, my open source projects, spy snooper and path on turtle. I'm a contributor in a bunch of big projects like C Python, Django, PyPy, and a bunch of other projects. I organized the Israeli Python community and I quit my job a few months ago. I'm looking to move to Europe, maybe Germany. If you have interesting job offers, please talk to me. So let's get started. The final goal for this talk would be to make a Python program that plays a musical piece. And we're going to get there step by step. We're going to expand each step as we go forward. The last goal is to play the musical piece. We're going to start with playing the simplest possible note. We're going to make it sound good. We're going to make the code elegant. We'll play a bunch of notes. We'll play different notes. We'll play lots of notes. We'll play notes real fast. And the very last step would be to play a musical piece. The first step would be to play a note. And we're going to need some kind of sound library for Python. So we're going to start with the text on one called sound device. It's got a GitHub page that the text is on the browser. Yes, I can see everything. It has documentation. You import sound device and just run sound device dot play. And you give it an unpy array with your sound data. First, you know, I'll explain something about sound waves. Here is the simplest possible sound wave, right? This is a sine wave. Air pressure over time. time so that's what we're going to try to do in Python. Feel free to ask me questions during the talk, just send them in chat and our session chair is going to read them to me. I'm going to import math, I'm going to need to do some math stuff, import sound device, I'm going to use numpy and I'm going to use matplotlib to plot my wave. Okay, here is the simplest possible sine function. Now I want to add music related stuff to it so I want to add volume to it so I can control the volume and I want to be able to determine the pitch, the frequency of the sound. Let's write down numbers for these values. I'm going to use 440 hertz which is an A note. I'm going to do a duration for the sound of two seconds and I'm I'm going to do a sample rate of 8000. Sample rate is like a resolution for the sound. Determines how many sound data points per second. I'm going to put all that in a nice sine function and now I want to plot it and I want to play it but before I can do that I said that that the wave is a wave of air pressure over time so I'm going to have to create an numpy array for the time and the numpy array for the air pressure. So let's do that. The time array is just the simplest linear array and the pressure array is a bit more complicated. I'm going to go over each point in the time array and I'm going to calculate the pressure for that time point and I'm going to populate the pressure array that way. Let's plot it to see whether it looks good. Okay this is our sound wave so far so good. Let's try playing it. We have the simplest possible note. Let's listen again. Okay so we did task number one playing note. The next task is to make it sound good right. Right now this kind of sounds more like a dial tone than a musical instrument right. Our goal is to make that sound sound just a little bit more like this right. So now we're comparing these two sounds. We want this to sound like this. Okay first difference you might notice is that the volume for the computer generated sound is constant while the volume for the guitar starts off as loud and then becomes gradually more quiet. So we can implement that in Python that's called exponential decay. So I'm just going to add a figure for the volume and I'm going to use the formula for exponential decay. It requires one constant half life. Half life meaning how many seconds it takes for the volume to become half of what it was. So I'm going to use 0.3 seconds. Okay let's listen to that. Okay this is better. Now it's kind of sounding like a cheap organ and now I want to take it to the next level. Okay still this sound doesn't sound as warm as this sound right. So what is what is the magic in this sound? I'm going to try to explain that. I'm going to show you something about music theory, something called overtones. But first I'll give you a refresher on frequency and pitch of sounds. Let's say I'm playing a string. The string is making a sound with a frequency X or a pitch X and that is determined by a few things. The density of the string, the tension of the string which is determined by the tuning pegs and the length of the string which is the only thing I can actually change. If I play an open note I get X but if I press on the frets I'm making the effective length shorter so I'm getting a higher frequency. If I'm pressing on this fret, this fret is just at the middle so I'm making the effective length half of what it was so the frequency is 2x so I'm getting a note that's one octave above the original note. Okay so far that's what most people know about strings and music. Now I'll show you something called overtones. So I'm going to show a trick I can do. I'm taking my finger and I'm putting it on this string and I'm plucking it and it's making an interesting sound. Now I haven't I haven't pressed on the fret like I'm playing. I just really like to touch it while I was plucking and I can even let it go and it keeps on going and I can do this trick only in a number of positions. I can't do it here. Won't work. I can do it here and here and here and here. I can only do it in a few special locations. So what makes these locations special? What are these sounds and what does that have to do with making a realistic sounding sound? These sounds are overtones and these special locations. This one divides the string into two equal parts. This spot divides the string into three equal parts. One, two, three. This spot divides the string into four equal parts. One, two, three, four, et cetera. And when I'm putting my finger here, for example, I'm forcing the string because my finger is chubby and fluffy. When I touch the string, it can't move at this location. So when I pluck it, it's making one wave here and another wave here and these are opposite waves. So the effective length of the string would be one half. So the frequency is 2x. And if I do the same trick here at the third point, I'm dividing the string to three equal parts, getting three opposite waves, and the effective length is one third. So the pitch is 3x. So x, 2x over tone number two, 3x over tone number three, 4x over tone number four, and so on. Five, six, seven, eight. Okay, so that's a nice trick, but what does that have to do with making a sound that sounds realistic? When I play these overtones now, I isolated them, which means I played just them without anything else. Because when you play any regular note, like this note, you're not playing just x. You're not playing just the base frequency. All the overtones are actually in each 11 note. So when I'm playing any note like this, it has trace amounts of this, and this, and this, and this. They are all mixing together to one note. People who are more musically inclined, when I play this, they hear traces of this. Personally, I don't have that kind of hearing. I can't hear this inside of this, but this cocktail of overtones is what gives the note its character. And it also explains why some notes sound good together and some don't. For example, let's say I take these two notes. Each of these come with their own set of overtones. And so it just so happens that overtone number three of the first note is almost identical to overnote number two of the second note. They're almost identical, so they sound really good together, because their overtones kind of mesh together. And if I were to take two notes that don't have meshing overtones like these notes, they would sound terrible together. So overtones are responsible for a lot of what we feel when we hear music. Okay, so far that's interesting. Now we can just take everything that I explained to you and implement it in Python. These overtones, they are just sine waves, sine waves with different frequency. X to X, three X to four X. We can just have a Python loop that just goes over all of them, adds them together, and let's see what happens. I'm going to go on a loop from one to eight and I'm going to add a wave for each iteration. Okay, and I'm multiplying by I here and the I has a job of being the one X, two X, three X. So the I is the overtone number. Let's say plot this wave. Okay, this is an interesting looking wave, right? I'm not sure what to do with this information, but let's listen to it. Okay, this is an interesting sounding sound. Let's listen to it again. This to me kind of reminds me of a guitar on distortion. I can electric guitar with a distortion pedal. Why does this sound distorted? There was something you might have noticed when I was playing the bass earlier, when I was showing the overtones, something I didn't account for in my Python code. So I'm going to play the overtones again and I'm going to see whether you notice. Okay, here are the overtones again. So yeah, they start off as loud and they become more quiet as I go higher up, right? This one's pretty loud. So is this one. But then they really get quiet, right? So it seems that the volume should be lower. I mean, higher we go with the overtones, the lower the volume should be. So let's add that logic in Python. Instead of having just a range for the overtones, I'm going to have a dict. And I found that this expression is a good one, one over I to the power of one and a half. I found it gives the best sound. Okay, so the change I made here is that from now on, every overtone has its own volume that I use to multiply. Okay, this is what the wave looks like. Kind of a steep ascent. I don't know what to think about that. And let's listen to what it sounds like. Okay, this is much better, right? It kind of sounds like a cheap piano. And so this is our sound and I'm happy with it. So we finish the task of making the sound sound good. Let's listen again. Now you might be asking yourself, why does this sound specifically like a piano? And why doesn't it sound like a guitar or like I know a violin or a saxophone or any other instrument? The answer is I have no idea. I just randomly tried stuff. It ended up sounding like a piano. I was happy with that. One more thing I'm going to do, I'm going to lower the volume because we're going to play a bunch of notes. That's still audible, right? Yes. Cool. Next thing I want to do, we're going to do a lot more complicated stuff, but I want to make the code elegant so I could work with it. I want to take all of this code and put it in classes. I'm not going to write too much new code. I'm just going to take the functions and put them in a class. I want the end result to look like this. I want to be able to create a note with a certain frequency and play it. I'm going to make a note class and that's going to take a frequency, a volume and it's going to have a length. Let's say each note has a length of one and a half seconds just to keep it simple. This sine function is just going to be the pressure method for this note. The frequency wouldn't be a constant anymore and neither will the duration or the sample rate. I'm going to take the code that plays the note and I'm just going to put it in a play method. Okay, I got my play method and I've got my get pressure method. I've got my constructor. Let's see whether this code works. Of course it doesn't. Forgot to change from sine to get pressure. Let's try again. Okay, same sound as before, right? Just now it's with classes. Now it's easier to work with. So our code is now more elegant. Next task. I want to play a bunch of notes, right? So far we've played just one note. Our final goal would be to play a full musical piece. So we're going to have to play a sequence of notes one after the other. Let's say that the interface would look like this. I'm going to create a sequence of notes. For each note I have to include the note itself and the time offset. So here's an example of a sequence I might make. Five notes, 0.2 seconds between each note and it's the same note just over and over again. So I'm going to have to create a class sequence and it's going to take a list of members and I'm going to have to figure out the length of the sequence. That's just going to be the very last note, the length of the very last note and I'm going to create a get pressure method for this sequence. I'm going to make them share a base class. They are both going to be subclasses of audio and I'm going to move the play method to the base class. So you could play both a note and a sequence. I've got my sequence and how do I calculate the air pressure of the sequence? Just add up all the notes taking the offset into account. So it's going to look like this. Let's see whether that works. Great, I got five notes playing in a row. I got a bunch of notes. Let's see how I am on time. Okay, I'm doing great on time if there are questions, feel free to send them. No questions just chat but awesome talk. Thank you. My next task is to play different notes. So far we played the same note five times. I want to play different notes, different pitches. So I'm going to show you the formula for that. I'm going to play it and then I'm going to explain it. Okay, so now we have notes of different pitches. Let's listen again. You must be wondering why, I mean, what is this? Why am I multiplying by two, raising to the power of something over 12? So I'm going to explain. The reason we are raising two to the power of something is because the musical scale is a logarithmic scale. What is it with a 12? Why do we have zero over 12 or four over 12 or seven over 12? That's because this octave is divided to 12 semitones. There are 12 frets between X and 2X. So that explains this map. And now we have different notes. The next task is to play lots of notes. We're going to play a musical piece. It's going to have maybe a thousand notes. We need to be sure that our program can handle a thousand notes. So let's start with just a hundred notes. I'm going to do a generator expression. I'm going to do random pitches. I'm going to use Python's random module. I'm going to choose a random note. And I'm going to play a hundred of these notes in intervals of 0.3 seconds between them. Okay, let's listen to that. Now we have a problem. I press play. I asked the program to start playing, but it's not playing. It's not playing because I gave it a hundred notes to play, and it needs to calculate all of that data before it can play it. Because it's a lot of data. It's now going to wait a long time before it can play that. So we want to fix that. We want to be able to play immediately. And the way we're going to do that is streaming. We're going to play while it is calculating. Let's go to the play method. We initialize the pressure array. We fill it up with data. And then we played it. So I'm going to make a small change. I'm going to initialize it. Then I'm going to start playing it on a separate thread. And then I'm going to calculate it. So the calculation is going to be going on while some device is playing the music in a separate thread. So let's try that. And from now on, I'll be using Python on the shell rather than my debugger. Okay, that works. Now it's going to play a hundred notes. So I'm going to stop that. Okay, this task is over. We're almost done. Now we want to play notes real fast. Why is that the challenge? Let's see. Instead of 0.3 seconds between each two notes, I'm going to do 0.08 seconds. Okay, did you just hear it stop abruptly? Right? I'm going to make it even faster. It stopped abruptly. Why did it stop abruptly? Because I was giving it so much work to do. It can't calculate as fast as it plays. If I wanted to fix this, I would have to make the code more performant, faster. I would have to optimize it to go over the code five places where I could do things faster and then improve them. I don't have time for that at all. I want a solution that's going to just make everything faster without me having to think about it at all. Can anyone guess what I'm going to do? I'm going to use PyPy instead of Python. Right? It didn't stop. PyPy is an alternative implementation of Python, second alternative to Cpython. It is faster. They use alien technology from the future to make Python three or four times faster than Cpython. So if you're not familiar with it, it's an amazing project. Check it out. It can just magically make things faster. So now our code can play notes real fast. The last step would be to play a musical piece. So far, when we've given our code notes to play, we've just given them manually just a list of notes and offsets. And there is already an established format for giving a list of notes and offsets. And that is called MIDI. So I got myself a MIDI file and I got a library called Mido for Python that can open MIDI files. So let's play around with that and see what it looks like. I'm importing Mido and I'm going to open my MIDI file. It's a MIDI file of type one, whatever that is. It has two tracks, whatever these are, and 561 messages. I can put it in the top and I can look at the messages. Let's look at the first 10 messages. Okay, there seem to be all kinds of metadata messages here. The time signature, key signature, tempo, track name. Let's look at the middle of the file. Here we see more interesting messages. There are note on and note off messages. And the messages, let's look at the note on messages. They give us a pitch of the sound, which is the note equal 60 part. They give us the volume, which is the, which they call velocity in MIDI language. And they give the time, which is how long the note is playing. We're going to write code that goes over these messages, reads them one by one and creates notes for them and then plays them using our synthesizer. I want the API to look like this. I want to create a MIDI sequence and then play it. And I'm going to do MIDI sequence as a subclass of sequence. It's getting a MIDI path and now we want to read this MIDI path. And we're going to go over the messages in, in this MIDI file. And we're going to add members. We're going to have a list of members. We're going to add members. They're going to have a time offset. And there's going to be a note with a frequency and the volume. And when we're done creating these members, we're just going to initialize the sequence with the members. Let's look at these messages again. We've got the note on and note off messages and we've got all these control change messages and I don't know what these do. So I'm just going to, I'm just going to ignore all the messages that aren't note on. And now my challenge is to fill out these question marks. I've got to figure out the time offset, got to figure out the frequency, got to figure out the volume. The volume is easy. The message has what they call a velocity and the velocity is a number between 0 and 127. So I'm just going to normalize it to be between 0 and 1. Next up is the pitch. And the pitch, I'm going to do a similar expression to what we did earlier. They have a certain base note that they use, minus 6 to 1 if I remember correctly. And the last thing to figure out is the time offset. So they've got, for each note, they've got the amount of time that it takes. But we need to make an offset out of that. So we have to make a tally of all the time offsets that we've had so far. We're going to initialize it at 0 and we're going to add to it every time we see a time offset. And we're going to use that for the current time. I'm going to use first just the first five numbers to see whether this even works. And let's try playing it. Okay, I got a bug. All right, forgot to make a tuple. Now let's play the entire piece. Okay, this is the code. We have just 79 Python lines, including blank lines, and we have a full synthesizer that creates sounds and read MIDI files. And all the code that I've written is in this repo, github.com, a Python synthesizer. So check it out. If you have any questions, it looks like I have around three minutes to spare. So feel free to ask any questions. I don't even know what to say. Like, I wish I had my, I wish I had my claps set up right now. This is incredible. When you get out of here at the Zoom room and you get into this court, you're going to see, we're talking all the way through it. It was absolutely amazing. Thank you so much. Thank you. Well, I have, I saved one question for you that seems like everyone wants to know, what are on those sheets of paper? Old code. I mean, yeah, practice is like a hundred times. You can do a 45 minute live recording session without practicing it a hundred times. No, so that's the secret for everyone that was asking that. Wonderful. That was really, really, really cool. Like, everyone is that draws are dropped everywhere. Thank you. Thank you. You know, I'll answer one question. I do get asked a lot when I give this talk. Lots of people ask me, what is this editor? Because most people don't know it. So this is called Wing ID. You know, most people are familiar with PyCharm. So that's like, that's like a different brand of PyCharm. It's a nice ID. Got lots of automation on it. So if you were wondering how I'm moving so fast and doing all this automation, that's Wing ID. It's a really nice ID and I recommend it. Cool. Well, thank you for that. And then Matthew is saying, well, awesome talk, and then a minor comment. He's saying that he thinks he could vectorize the calculation of the pressure instead of doing it in the for loop. Using, using map I don't vectorize. I used to do that. The problem is you can't stream when you're doing that because vectorize doesn't let you access the array as it is getting built. I was actually spending lots of time trying to find a good solution for that. Perfect. So you thought about it beforehand. So Vinayak is asking, are there any gotchas through and out this on Linux? On Linux, I think there is like one more thing you have to add, you have to add a delay before it starts playing because the timing is different between Windows and Linux. You have to add like one line for delay. And I actually in the code for this talk, I added it as an optional commit. I'm going to show it to you where the commits. And so the repo has a code commit by commit. So you can see all the steps that I do. And I did one commit that adds a delay that makes it work on Linux. Right? So this is a commit and it just adds a delay just before it starts playing. Perfect. Well, thank you for that. And yeah, I have a question actually that we had like 30 seconds left. How long did it take for you to prepare this talk? Oh, I mean, I've given it in a couple of conferences before I've spent, I spent way more time than I'd like to admit. Okay, I won't force you to tell me that. Okay, perfect. Well, thank you. Thank you so much for it was incredible. It was one of the best talks I've ever seen. Thank you very much for coming. Thank you, everybody. And enjoy the rest of the conference. And thank you, ladies, for sharing this session.