 My name is Uli, I make stuff work. I gave lectures on some yapsies and perconferences in the past, but not in FOSSTEM, so if some of you don't know me yet, that's me. I listen to being a software developer, which is why I'm here, and a translator. I also dabble in music and amateur musician. So this perl computing music thing is one of my personal projects and I've seen some people make a disclaimer about personal projects that the project does not reflect the views of their employer. But I'm self-employed, so I can't disclaim that. However, I don't think there are any controversial views here, which I apologize for, and computer music. Well, computer music is basically not very well defined. One might say any sort of music where computer is used as part of the performance or composition or anything. The thing is computers are in everything today, so I wouldn't really say I would call everything computer is involved with computer music. I would say if you make something interesting with the computer. There are a few things you can do, different things. There is MIDI. MIDI is something that most of my examples, they will be about MIDI today, and MIDI is used both in real-time performance, it's used, it can be recorded in files. MIDI is essentially a protocol for communicating performance events between different devices, and I'll go more into it a bit later. There's sound synthesis. Of course, computers can be used to synthesize sound. Is it before computers? There is software synthesis, there's hardware synthesis, software synthesis is much newer hardware synthesis is before, but with faster computer we could do a lot of something software which used to be done only in hardware. Computing music performance. There are a lot of musicians use various kinds of computers and software, they use it for performance, they use it for interactive music. Usually it's not fully interactive. Interactive means the computer, whatever the software on the computer interacts, responds in some way to the musician in real-time, and it can hear them sometimes, it's acoustic, sometimes the former plays into a microphone and the computer can figure out the pitch tracker, what it's playing and respond to it. Sometimes it's using various electronic instruments, MIDI controllers, I've had a very interesting project when I was studying electroacoustics where we used Arduino and the Arduino, the open source board, the open source hardware. And I connected some physical various levers and stuff to the Arduino and this information, this into sensors and so my physical actions were translated into messages to the software and the software created user to send controlling messages, some of them MIDI, some of them to other software. I also learned that that project I once would read one subject lines before one sends email when I was building my constructing the project, my instructor said, why don't you go if you want to build something physical, some tangible instrument, tangible controller, just to go to a flea market, get some junk, find some nice centerpiece and build around it and after I bought it I took pictures of stuff I bought and I sent it to the instructor and originally I just sent a subject line, here are photos of my junk. Fortunately, I reread it before I sent it and I just put the massive subject line in that and fully interactive is quite difficult so usually musicians use also some triggers which means there are a lot of stuff which is pre-prepared before the performance and they set up some sort of buttons or some sort of controls that trigger a specific short piece of music, short trills, short effect. So triggers are somewhat interactive but they're not full interaction with the computer like another player, it sort of control the head, it's small things that are prepared ahead of time and used in performance at the right time. For instance, when I was doing an electroacoustic piece which is also new too, if you look my name up and you'll find it, there are some of the sounds there are prepared and synchronized, some of them I was triggering during the performance as sort of some improvisation. Controllers, controllers are of course any kind of instrument that is used to control a sound source and most controllers are MIDI controllers and most of these are keyboards, keyboard instruments because these are probably the simplest kind of instruments to connect to simulate, to build as an electronic instrument. There are also of course MIDI controllers which are not keyboards, they are wind instruments, string instruments and I think people can build. So first I'll talk a bit about MIDI and before I go into using MIDI, I'll just explain what MIDI is. MIDI records, doesn't record music, it records performance and I think the simplest way to explain that is by an analogy of something which is simply for us to imagine and that's player pianos. Player pianos, you may have seen them in some old movies, this is the piano that stands at the side and plays by itself, not because it's haunted but because there's some sort of large roll of paper going through it and the way it was done, the company that made those player pianos, the most famous one was Pianola and that name was actually became the generic name for player pianos like Fidgeteer for a Fidgeteer or like Xerox for photocopy machine and those machine had recorder pianos and they would sit a player at the piano and when he stuck a key, it would make an indentation in a paper roll and the depth of the indentation would be dependent on how hard the player struck the key and the roll would go on and the entire performance would be recorded on the roll of paper and then they would set the roll of paper with lots of copies sent it to other people and when you ran the roll of paper for a player piano, it would duplicate the performance of the musician. However, because it was a different piano, it had a different sound, possibly different tuning, it might have been completely untuned. So it's recording only the performance and not the sound. Mid-range works much the same way. Curiously enough, there are lots of piano rolls in various storage spaces around the world and George Garifarin recorded about 300 piano rolls and they still exist somewhere and there are piano rolls enthusiasts who are very enthusiastic about piano rolls the way very enthusiastic about the player who want to convert them to something we can hear today and there are two ways of doing that. One way is to get the player piano, spend lots of money fixing it up, getting it to a studio, running the paper all through it and recording it. Another way is scanning it into a computer using some special scanner and converting it into something like MIDI for instance. And Pearl has several, let's do the next one, Pearl has several MIDI models and just to go through them, I go for a project I've done and when you start composing, I've done some arrangement and composing and composed a bit, arranged a bit, you know like musicians do. So when you start writing a piece, sometimes you start with a motif. A motif is sort of like a short melody which you're probably recurring through the piece. Sometimes it will be precisely duplicated, often it will be modified a bit. And a number of ways of course can be used to record this motif into the computer. I can take a keyboard and MIDI controller, connect the computer and play it. That's one way and use a sequencer program to record what they do. However, Pearl provides us other ways to do it. For instance, one, there's a host of MIDI models used in Pearl and one of them is MIDI Simple. MIDI Simple, I would have to say it stands apart from other MIDI models and that's not a value judgment. It just simply works completely disconnected from all the other MIDI models, it's its own model. It doesn't have anything to do with other MIDI models. And in fact, this has a lot of drawbacks. It can only handle one track. First I'll say something about tracks. Tracks, this is a term that's coming from all the recording technology where they had these big real tapes they recorded on and they could record on and the recording had recorded on parts of the tape and you could record one track and then the next player could hear the original recording while playing on the next track and the sounds of the recording would be synchronized and all mixed together. And even though now we all record everything we record everything on digital, it's a very useful concept. So this terminology holds over from the analog age to the digital age. And with a Simple, while it's very limited in some ways, I like it because it sort of creates a very nice DSL, a music DSL, the main specific language. And if you know a bit of music and a bit of pearl you can possibly figure it out. And n simply means note. And the second parameter, well, the first parameter n is a function problem actually. And the first parameter means what length note and we have here hn for half note. That's two beats and qn, that's a quarter note. That's usually one beat in four quarter music. And we have, oh, en, that's an eighth note. And we don't have 16s notes here, but we do have dhn, which is a dotted h note. That's a dotted half note. That's the dot adds 50% to the length of the noted dots. So this is very nice to figure out. Very simple to figure out. And the second parameter, that's the note itself. And here it uses mideterminology for it. The notes are the American style. You know, it uses the alphabet to denote the notes. And the number after the note, that's the octave number in midi. Midi gives each octave on the keyboard a number. And g4, that would be the g or soul on the middle octave. The octave just above the middle c. And you can see the lower case after, that's a flat. So this creates this motif, which means I decided to start my composition. And this may be time to just, I will go back to that. And this is the midi file it create. This is the midi file it created. And I'm just going to open it with, let's just open here, opens here. Let's not open it with this. Now this is a rose garden. It's one of the sequence of programs. And once I open it here, I can open an editor, I can open it in the full matrix editor. What I actually want to show is the matrix editor. And it's not opening, and we just leave it alone for now. Okay, we just leave it alone for now. I'll go back here. Okay, what I wanted to show was that, I didn't show it here, that even more than sequence of programs, they use two different views to show you the notes. One of them is sheet music, which is very convenient for people like me who learns classical music. But some people are also comfortable with reading notes. So the other view is here in that program it's called matrix view. In some other program it's called piano roll view, which is the analogy I made before. So because of the similarity between midi and piano. Now I can go back to the other midi models. And here is just the midi models. I want to see what kind of events we're written into this midi file which I created. And we have the model, we have, I think we just say use midi and it includes all the other midi models, except midi simple. And the one I want here is midi opus and midi track. Midi track is not seen directly here, but that's what midi tracks is term. Okay, opus is just a nice word for a piece, artwork, a piece of art, a piece of music. It's not midi terminology. It's pearl midi terminology. The midi terminology is song. Because the people who developed midi came from, but they were thinking about pop music. Pop music, one piece of work is a song. But whoever wrote this midi model, I have no idea what it is, but I'll buy my drink if I find out, if I look it up, if I run into him. I just like the word opus better. And what opus does in this here, it can read, it opens an existing midi file. It's very simple to read. Then I can read all the tracks into an array. And each track is an object of type midi track, which we may see in a later example. And then I just use a loop to loop over all the events and output all the events. And it looks something like this. Note on, note off are the most common events in midi. And it's very simple to understand. Note on may simply mean it's like you strike a key. If it's a keyboard, you strike the key. And note off, you pick up your finger, you stop holding the key. The four parameters there are delta, channel, note, and velocity. Delta is delta and duration. How much time since the previous event? Not necessarily the previous note. So it is actually possible to use this to play several notes simultaneously. If the note on is the delta from the previous note on is zero, it means they're playing at the same time. Here I don't have two notes playing at the same time. The delta zero is from the last note off event. The number is duration in some internal counter. Channel, that is a logical channel. You can play midi several channels. For instance, you want to play several different instrument sounds at once. You can say channel one is going to be piano sound, channel two is going to be trumpet sound. And then everything is sent to channel one afterwards, plays with the piano sound, everything of channel two afterwards plays with the trumpet sound. Channel 10 is usually reserved for a percussion. The third one is the note number. Before we saw the note names like the G5, C4 and everything, which is just very convenient for humans to look at, but every note also has a number. So G5, the note number happens to be 55. And the note after that, the number is 60. And the numbers follow, you can just count from on the keyboard. If you follow all the notes, there are consecutive numbers on the keyboard. The last one is velocity. Now in normal English, velocity seems simply another word for speed. In middle velocity is another word for volume. What sort of makes sense if you know how a keyboard works? And when you want to play loud music on the piano, you hit it harder. What actually happens, and there's an entire mechanism there that you hit the key and it goes, it's the hammer, the hammer strikes the string. But what you really get is a lot of kinetic energy. So if you play louder, the key actually goes in faster. So this really comes from the engineers who devised a way to figure out how to play loud and soft on electronic keyboard. Because if you go back 30 years to the 80s, if you bought a keyboard back then, those keyboards could only play in a single volume. They were not touch sensitive at all. So it's a huge drawback. So the engineers tried to figure out how to be touch sensitive. They tried to measure a few things and they discovered that if the key goes in faster, it means you're playing louder. So velocity and maybe actually means volume, volume of a specific note. Okay, what I wanted to do with this motif, I started, I tried, what you usually do when you start composing music, and the motif is supposed to recur throughout the piece. Now, the simplest way, of course, is to repeat it exactly the same way. But that's not very interesting. It's more interesting to develop it, to make slight changes to it. And normally I would sit at the piano keyboard or the MIDI controller keyboard and try to play a bit with it and play the motif for a different version of the motif, try playing it maybe a bit faster, slower, change a few notes. If it's a minor key, try to play it in a major key. But then I thought, is there a way I could use some sort of algorithm to explore the melody space? Maybe it's an algorithm that can suggest to me other versions of this motif that I would like, that would be pleasant for me to hear, that would be interesting for me to include in my piece. And I thought, what sort of algorithm would give me approximate solutions, approximate answers to this question? So I've turned to an algorithm I used before successfully. Well, I said now, okay. Questions of what is similar? What does similar mean? Similar, of course, means sounds the same melody, sounds same rhythmic pattern. And this particular case, I was actually looking for something which is where rhythmic pattern was more important to me than the melody. And the algorithm I figured I could use were genetic algorithms. Genetic algorithms are algorithms that use the concept of evolution to arrive at a solution. They don't guarantee a solution. However, they're very good at quickly arriving at approximate solutions. And a couple of years ago, I presented a way that I used to use genetic algorithms to synthesize sound. And on that case, I wanted a very precise solution, which I was close to arriving at. Here I wanted a very imprecise solution. So what I tried was somewhat unorthodox use of genetic algorithms. And I figured if I use them improperly, I could probably get some nice suggestions for variant motifs. So first, my very, very brief primer about genetic algorithms. For genetic algorithms to work first, you have to find a way to represent a problem. You need to take all your parameters, all the problem parameters, figure out what they are. There could be numbers, there could be strings, there could be various numbers, there could be switches, whatever. And write them out. You just write them out as one long list of parameters. You have genetic operators, you have a fitness function. Genetic operators are, okay, let's start with the fitness function. The way you work with genetic algorithms is you start with a random population. The first population you create is a completely random solutions. That, of course, they're not good solutions. Then you run the fitness function all of them. You rate them, you give them each a grade. And these grades, but how well they solve the problem, they determine which solutions get to breed and create descendants for the next generation. Not only which ones, but how many descendants they get to have. Genetic operators are things like crossover. How do you take two solutions and mix them up? And usually you just take a few parameters from the first one, a few from the second one, and you mix up the parameters, and you get a new solution, which has some parameters of the first parent and some parameters of the second parent. Other genetic operators are mutation, random mutation, which you don't want to have too much of. But it's good to have. And so my first thing, before I could start doing genetic algorithms, I needed to create a representation of the MIDI melody. Now, one property of melodies when you use the Western keys and Western tonal system is if you transpose a melody by several keys or several notes up or down, it sounds the same. So I don't really care about the first note. It's the same note for all of them. And what I do care about is the durations, because if I compare the durations, I have a rhythmic pattern, although the rhythmic pattern is, I care about the intervals between the notes. If a melody starts with a note going from G to A, that's one tone up, two keys up, I want to record that. Another thing was, I figured I can handle strings pretty well and think like string similarity. So I decided to encode the melody as a string. But the first character is the duration. Well, the very first note is fixed, so I don't care about it. So the first character of the string is duration of the first note, because that is a parameter that may be changed. The second is the second note. And the third parameter after that is the duration of the second note and so on until the end of the string. And just to make something simpler, let's use uppercase characters for descending intervals and ascending intervals and lowercase characters for descending intervals. For durations, I said, well, I'm just going to divide everything into 16s notes. So a quarter note would be four and there's a half note would be eight. And use the characters from eight to eight, from eight to the first 16 letters to the note how many 16 notes they have and the longest note I have is a whole note. So I also limited myself in how long the notes can be. So this code, it just takes the same things so before to create the list of notes. And it creates the chromosome this way. The chromosome, that's the terminology used in genetic algorithms. We use a lot of terminology from genetics. And this would be there. My melody encoded as a chromosome for the purpose of the fitness function. For the genetic algorithm itself, I used another model I've used before successfully called AI Genetic Pro. It's quite fast, it's got very good performance. It can handle lots of very large populations. Oh, I'm doing a time list, not of, okay. And when you start doing a genetic algorithm with AI Genetic Pro, there are several things you need to give it. First, reference to the fitness function because you need to write your own. You need to write your own because the fitness function is specific to the problem we're trying to solve. The rest of the parameters are pretty straightforward. Type, list, vector, this is what the kind of, the kind of what type of chromosome. If I give a list of parameters, it's a list vector. If I just want to give a list of ones and zeros, I can get, it's a binary chromosome, to other options. Population, 1,000 is a very nice, the very big, nice number for population. If populations are too small, you don't have enough genetic diversity in the population and you might reach, you might get an early divergence on a very bad solution. So population needs to be large. Cross over, this is the chance of a crossover. How, what is the probability that the crossover will happen when two solutions mate? And mutation, this is also a probability. It should be a very low one. If it's zero, then you might miss out on some very good, on some very good genetic possibilities which are not in the population right now. If it's too high, it might actually spoil some good advances that you've made so far. So it should be low. Usually 1%, 2% is 0.010.2 is all you have. Parents, unlike biology, you can select a number of parents. Two is fine. Sometimes you can also have three parents for one solution. Selection, selection is how the selection is being made. How are the parents for the next generation are selected? And there are all sorts of different strategies. I'm not getting into them now because that's not the main thrust of the talk. This is one of them, roulette distribution. There are also others that's roulette. Basically it says that instead of determining ahead of time how many descendants each solution will have according to its rating, your term is the probability of whether or not it will have more descendants. It comes about the same point. Strategy, points two. This is strategy of how the crossover are made. This is a two-point crossover. The difference between one-point crossover, two-point crossover, one-point crossover. Only one point is selected on the chromosome and then everything before that point is switched between the two chromosomes and it's created to the generation. A two-point crossover, two points are created and then everything between them is switched between the two chromosomes and that's create the new descendant. So everything from the first position to the first crossing point comes from parent one. Everything from the second crossing point to the first crossing point, second one comes from parent two. Everything from second point to the end comes from parent one again. Again, what is better? This is a lot of trial and error. Except in the particular situation, I was not concerned, thank you, with precision. In fact, I wanted imprecision. So this was how I started my initialization. This is how I tell the genetic algorithm model what to put in the various parameters. As you can see, I have my duration parameter. It's the first one and the second one is my interval parameter. My duration parameter is, of course, one to the first 16 letters. My interval parameter is lowercase letters, uppercase letters, descending and ascending intervals and zero is presenting the same note. Okay. And then when I want to start telling it to evolve, just there's the methods evolve. The number in parenthesis says how many generations? The more generations you have, the longer it takes. And the better results you get. Usually everything under 34 years is better useless, but gets fittest, gets me there. And, in this case, 40 top candidates, one with the highest ratings, and the last one just to print them out to a file, because I want just to keep the chromosomes themselves, to keep a record of them. And the fitness function, this is really the heart of the matter here. It represents all the things I care about. Well, not all of them because I cut it in half way, but it represents the things I care about when I want to mutate what I consider to be similar. So, I have the descriptions, which I created some, yes, I created another function. Descriptions of the two melodies of the original melody and the candidate melody. And one description, I break it up before I do this into intervals and durations. Durations, I get the written patterns. From the intervals I get the melody, whether it's going up, down, sideways, whatever. And I compare them and I have the intervals of the melody. I have the inverted melody, which is exactly inverted melody, exactly the opposite of my melody. If my melody goes da-da-dum, the inverted melody goes da-da-dum. It's exactly the opposite. It's useful. Inverted melodies are used a lot in foods, more than in other types of music. And I'm saying if it's similar to inverted melody, it could also be a very nice variant motif for me to use. But it gets a lower score than similarity to the melody itself. Duration, similar to duration, also get it. And I'm not sure it made it in here. I guess actually it did, maybe next time. To check similarity between strings, I just use a combination of models that do this. L-C-S, L-C-C-S, L-C-S-S, that's the largest common substring, which is nice but not good enough for the particular, not quite good enough here, because a string might be modified in the middle, but the biggest subring is still quite similar. So I also use trigrams and something called string similarity. Trigrams are very useful to find similarity between text, they're used in translation memory when you need to find a similar sentence than one you have now. The entire text is broken up into sequences of three characters. It's not for one, two, three, four, five, six, it's one, two, the first, second, third, and the second, third, fourth, et cetera. And then they're counted. And if the count is similar between two texts, then the texts are probably similar. In fact, if you take a text in certain language and in great trigrams, then you get a random text from those trigrams, someone who does not know the language might think it's that original language. It looks that similar. And string similarity is another model which I think is another algorithm for similarity. And, okay, we have five minutes and any questions? Just quick questions about this. Yes, please. Would you please demo some? Yeah, I really want to hear it. Yes, I want to hear it too, but unfortunately I don't have sound here. So it will be on my site. It will be on my website, all of this. There already is one posting about where the website was out sound. And I was playing it, yes. And my site, it's brooksioil, brusk.co.il. So there's one where the post is out sound and I add the demos there. And I add the stuff about synthesis which we didn't even get to also. And probably there within a couple of weeks, two to 52 weeks is the most, you know, that's your stuff. Any other questions, except for demos, which I... Yes, please. Yeah, I talk about music or make music. Yes. I want people from the internet to be able to make music on my machine for whatever reason. But that would mean I would have to make music and do other stuff at the same time. Would it have any impact on the responsiveness or the delay in all the MIDI handling? It probably would, but if you do that, I probably wouldn't use prayer for that. It would be fast enough for the music people. Sorry? As in, can I do MIDI handling? Yeah. And do other stuff at the same time? Well, first of all, you probably do it all the time because your phone can play MIDI and your computer can play MIDI files and they do it all the time. Okay. So, yeah. Okay. Any other questions if I have time? Thank you. Thank you.