 I'm very excited to be here I was telling the organizers like this is the conference I always wanted to talk at and I'm here the last year so I'm very excited thank you for accepting my talk I hope it's weird enough for you and so I'm gonna warn you in the next like 25-ish minutes you're gonna encounter some like very poor singing too many cat gifts and just an excessive amount of silliness so get ready first I want to my bio is a little bit out of date so you can you can see up there says I live in Omaha, Nebraska I know I no longer live in Nebraska I live in San Francisco as of like two weeks ago but fun fact about Nebraska the tourism department just released a new tagline and get it so this is what it actually is honestly it's not for everybody like that's that's it like that's Nebraska's like tagline I'm like really you're right it's not for me either as of actually later this month I am going to be working at thought starting at Thoughtbot in San Francisco very excited about that I know there's there's a few thought butters and former thought butters in the audience hi if I haven't gotten to meet you yet I'm excited to after this so but let's get started okay so I've been told by a few people that one of my strengths is thinking outside the box or as I like to call it being really good at coming up with stupid ideas so some of my stupid ideas have included feces book a social media website for your poo there is kombuzcha which for those of you who don't know kombucha is a fermented tea drink but this is a super boozy kombucha so birth announced birth announcements for features that you release at work but I don't have a name for that one yet I got to think of that one there's a Meowaka and so it's a US based anti-social network for your cat and I tested it out in a smaller market in Omaha it's called oh meow ha so slight slight change of subject still very relevant who here is familiar with Game of Thrones raise your hand okay I'm assuming most of you because it's the most popular show in the world currently so I'm gonna give you a little bit of background and why I'm talking about Game of Thrones all of a sudden this is me my wonderful husband we got married in a bar to the disappointment of both of our parents that's not the relevant part we have some cats so this is a Xiao Gui and Clementine and their sweethearts and this is Geeta she's an asshole and yes I just for a professional picture of my cat so I'm not I'm not person this is also me and my husband relevant because we like Game of Thrones this was a few years ago after like season one aired so you can't even tell the difference right I I can't I don't know which one's me but onto onto what really matters is you might be confused about what what is it with Game of Thrones that's okay because I'm going to play you the Game of Thrones theme song and let's see if it should start playing not yet not yet hold on you guys just heard the sound right okay let's try this again I'll stop it there so it's an it's an instrumental score right there are no lyrics or are there well there are if you add them yourself so what we're gonna do I said there would be singing we are going to meow the Game of Thrones theme song and I'll get us started and one second like so why now I have no idea like it just started one time like we're watching Game of Thrones it comes on and we're like oh there's nothing to sing along to so we start meowing and are you guys ready because it's kind of addictive once you get started so it goes meow meow meow meow meow meow meow meow meow meow meow meow meow meow meow meow see it's fun it's fun so like we just we just meowified the Game of Thrones theme song probably set like a world record I'll submit that one to the Guinness book so meow fire is the name of my application and it's just one idea and a long line of bad ideas I've come up with over the years but it also kind of seemed like a really interesting coding problem to solve and I finally said what the hell and I went ahead and built it so how does it work well the basics you upload a song's audio file and meow fire outputs a new audio file with that song's melody sung by cats okay so I wish I could say there was such a thing as a cat choir I googled it and nothing came up so all up to me to figure out but my internet research I came across one of my favorite musical cat cats of all time oh no does he not play now sorry so imagine rest in peace keyboard cat I know he's gone cats don't live that long you know videos like the early 90s cats almost as old as I am so I actually discovered I'm not the first person to come up with this idea this is this is a cat organ I think it should be called a cat torture device but so in the 16th century a historian described scene like a cat organ being played wait for it being played by a bear when King Philip the second wrote into Brussels so so there's like a psychiatrist who was like hey there could be some medical potential to this cat organ thing because like it's the only thing crazy enough to get like crazy people to pay attention I was well that's one way to put it but it was likely just a hypothetical instrument until now so I was ready let's let's do this right I know so I had some big challenges building meow a fire I knew that I was gonna be facing some problems so I but the thing I had going for me is I had momentum I had this idea I was excited about it so I decided to go ahead and start building before I had like really done any research like how am I gonna do this what's the solution the thing I did have going for me was that I had read pooter practical object-oriented design patterns in Ruby one of the best Ruby books of all time I think highly recommend it if you have not for some reason in this room if you've not read it please read it so thanks to Sandy I knew that I need to make my classes pretty stupid dumb down I also decided because of some previous work experience with legacy untested legacy code that this application was going to be 100% test driven and I was like yeah let's do this have some standard set in place I'm gonna be able to build this thing so I knew building meow a fire I was gonna have three big challenges so one finding a way to obtain the notes of only the melody from a song's audio file to correcting the meow length to match the length of the note in the melody and three creating a multi octave library of meows so let's get started the melody so it's pretty easy for a human especially one with any musical training to pick up the song the melody of the song so I'm gonna take it a sip here because I'm gonna be singing poorly in a bit yes that's alcohol so the melody is the principle part of the song so every pretty much every song you hear on the radio or TV is polyphonic which just means that there's more than one note going on at a time but if you were listening to Bohemian Rhapsody by Queen the melody would be the part that Freddie Mercury is singing so you know mama just killed a man or the I see a little silhouette to other men's covers the whole thing so those are the melodies so if you were to include the harmonies and the bass parts and all the extra extra musical instruments in that you wouldn't necessarily like recognize the resulting song because it would kind of all sound jumbled together so I didn't want the baseline or the harmony muddy and things up so what did I do well computers are pretty unintelligent right so somebody has to write an algorithm so that a computer is able to extract the melody of a song and you're thinking oh did you write an algorithm no I did not write an algorithm I did not even try to yet at least I did what every good programmer does and I searched for something easier and already existing and I found a tool so I didn't have to write it from scratch but the first try it was it was interesting yeah a little weird so the first tool I tried was called Sonic API and it offers professional grade audio technology with high-class world-class algorithms they're like high-quality world-class algorithms and so I was like well I'll give this a shot it's a free for the first you know so so many like gigs of whatever so this is kind of what it looks like so I have a simple song parser class and inside this class I have a parse method so all I need to do is pass the proper params through an HTTP call and the song file and my key with the song file in my key and it's supposed to extract the melody for me so the response that I get back from this API looks so it's gonna look messy it's a little something like this it's very long I don't expect you to see all that imagine this but hundreds and hundreds of lines long and this happens to be the first few notes of the Game of Thrones theme song so we're gonna take a closer look and see what these look like okay so there's four pieces of data in each collection so the MIDI pitch maps to the pitch of a note so MIDI is anybody here not familiar with MIDI it's like musical musical instrumental digital interface something like that that sounds about right I think I have it later on we'll get back to that but so it's a standardized thing that electronic instruments use so that everybody every instrument electronics like electronic instruments playing the same note standard notes so what we have to do is so this MIDI pitch is 35.99348 so what I need to do before I can map it is I need to round it up or down to the closest whole number so this MIDI pitch would be rounded up to 36 let's see what that gets us so here's a oh I was right MIDI musical instrument digital interface so the 36 maps to the C that is two C's down from middle C if you're a musician you'll understand I'm gonna show you in a second here this is I I'm a pianist so this is what it looks like on the piano and helps me visualize a note when I look at it on keyboard so maybe this will help you so if we want to play this note it's way down here and what I ended up doing is like I said I'm lucky MIDI is standardized so I just went ahead and I made these constants in my note converter class so this comes into play after I've parsed my song again here is MIDI note 36 and then I have it mapped to its corresponding pitch C2 and now a little time for a butt scratch break a little bit almost halfway done so you know you're not scratching your butts why did I even put this one in here so so what happens next well so there's that append array with note method so that method adds the standard pitch to each line in the collection that my HGT HGT the request got back so I'll show you here so it takes this collection and this hash and it adds another piece of data on the end the note C2 so we had this big collection with four pieces per line and now we have five pieces and it just imagine hundreds and hundreds and hundreds of lines of this so so what's next well if you're looking at if you looked at the hash you could see it might have something to do with like onset time or duration or volume and you're right it does have something to do with duration note length so I need to correct the meow length to match the length of the note in the melody and a melody is gonna have notes of varying lengths so if you think back to Bohemian Rhapsody Freddie doesn't just hold each note for like half a second each some of them are like a quarter of a second or like even a whole second so it's not mama just killed a man it's mama just killed a man so the problem is I can't have a meow folder containing meow files of every conceivable length so that would be a bloated monster of a library so instead I had to find a tool that could either cut or extend a file to fill the proper amount of time and if you were here for Will's talk just before mine you know that there's some good tools out there and like like I said I do as little work as possible I found the laziest way to do this which was ffmpeg so like will was using it it's like been around for almost 20 years I think so I'm sure a lot of people in this room have used it many times so the only problem I came across is that it almost had too many settings so like it took a quite a bit of stack overflow searching searching to find exactly what I needed but I got it working and it was pretty it was it's actually a pretty really messy method and I haven't refactored it in about two years so it's hard to determine what's going on here but like I said it took a long time so I was like I'm done it works let's do this so let me walk you through it so we pass in the past song so it was that collection that the API returns to us and then we append so we passed this in and then at this point I need to tell you that there is a library of meow files living in my application and we'll talk more in depth about those in a bit but all you need to know right now is there approximately 88 short audio files each with a meow in a different pitch like the ones you'd find on a piano keyboard so this is the part of my code that creates a meow with a correct duration so this first piece of logic the if statement is the part that shortens a meow file and the else statement lengthens it so short notes I'm going to tell you how this works short notes were pretty simple so thank you so when the length of the extracted note was shorter than the library file you make a copy of the library file and then you trim it using ffmpeg to the correct length so it looks a little something like this so the note duration is only 0.48 seconds and the note to adjust is a whole second long so all I do is cut it down really really easy to do so the other part wasn't as easy it's a bit trickier so there's there's a few ways I could I could have gone about it I chose the easiest way to begin with so if the note the extracted note was longer than the meow library file you keep duplicating that meow file until it's longer than the extracted note and then you combine them to create one audio file and then you shorten it so it looks oh here's my code so here's a logic for the long notes this is actually one of my favorite parts of the program to write and I'm actually gonna completely rewrite it because I thought of a better way to do this but for right now this is how it works so you find so here's the note length 2.46 seconds and the file length is only one second so how am I gonna get that well simple math take the note length divided by the file length which is 2.46 and then I use the seal a method which returns the smallest integer greater than that greater than or equal to the float so that's three so what I do is I grab three of the meow files and loop it and then combine it and then all I got to do is crop it to the correct length so that's how the that's how it currently works and pretty happy with it oh man that's really really big on this screen well here we go okay so my head might be in the way so all those files get stored in this array and the last part is in my song builder class I combine all the files together and output a new file so so hey that solves the note duration problem so what's next this part is next the meow library so it's a lot of meows if you haven't noticed very few cats meow in like the bass baritone tenor range and where do you hire a cat choir to begin with so I had to make my own custom meow library and it started out very interestingly I was sitting at my piano literally like playing a note and then trying to sing it into a microphone the problem is my piano is a little flat and then I'm always a little bit flat so it didn't didn't quite work out how I expected so I was like well I'm gonna try this a different way so my next step was singing into my laptops microphone well like I had one of those like tuner apps on my phone so I'm like yeah yeah like just you know I was like okay this is gonna get really old and it's not actually a cat meowing okay I think I got like half an octave's worth of notes like doing that before I realized it wasn't cutting it so turns out I only recorded five notes before I got bored but at this point I went to my tests to see if just to see if it was working and I knew there was I knew of one song that only had five notes in it and yeah yeah yeah yeah yeah my test passed but obviously like the five notes wasn't cutting it so I went down that easy road again back to the internet and I finally found a few octaves I finally found a few octaves of a man meowing like an autotuned man meowing yeah it was on this free sound website called like free sound I was like okay cool I'll give this a shot it's more than what I have so far so here's what my library looks like I've got like some notes so a keyboard has 88 notes if you're thinking that this does not look like 88 notes you'd be correct it is actually only about this many notes 49 so just about four octaves worth so what was I gonna do if an analyzed note fell outside the range of my meow library well instead of actually like making a whole meow library I was like I'm just gonna write this method that adjusts the octave so to match what I actually do have in my library so if a note fell in one of the octaves I didn't have a meow for I adjusted the octave number so if the note was f7 I would adjust it to f6 and so on and so forth so and then another problem with the library downloaded so each note has a pitch octave designation and so any note with a sharp is it designated with with a pitch and then hyphen and then the octave so you can kind of see like g-6 dot wave so that's a G sharp in the sixth octave so I was like oh but I already wrote all my constants out and they have the actual sharp format in there so what was I gonna do well I created a new note mapping or a new a new method to format the notes for me because I was lazy instead of changing the files so I'm like there's a thread in here like I'm kind of lazy so that's what that guy sounds like singing jingle bells that's great I was like that all right it's not quite what I was hoping for so like I took all that time like adjusting my app to work with that specific library which was a mistake because I still wasn't happy with it so I decided to just bite the bullet and make my own cat meow library it spans this many notes I no longer need this formatting no note method because I renamed all of my meows as I created them and I could rid of it and let's and I can get rid of this set octave modifier method oh this this guy here because I was like well I have all the notes I need which was great so now look at all these notes that I have those are all meows so it's the moment you've all been waiting for almost so just to remind you this is what we're going for theme song everybody I don't know if you heard that but doesn't actually sound like the Game of Thrones theme song so there was a problem and it started way back in that first step with the melody analyzer it wasn't actually that great of a melody analyzer apparently you have to pay lots of money for a really good melody analyzer and I was like I don't have that kind of money so I'm gonna try something else my second try so is this this library called melodia and I found this guy he'd written his PhD thesis about melody extraction I'm like great somebody's super smart he developed a software and also command line tool and it was free and I was like what the hell I'll give this a shot so I signed the release form before the university where he did his research would send me the software this is what the release form looks like I'm not sure they actually read it because they actually sent me the software but hey I got it and this was just like a simple Python command line tool it ran really fast like almost too fast I was like you know what I'm okay with that so I started testing out the command line tool I haven't saved the output files directly to my desktop the difference between this software and the previous tools I was I was using is that this one returns a MIDI file so now I needed a new class to translate the MIDI files into so I was like but I don't want to write a new class and to like I know that this works so I upload this file it returns to me to a free MIDI analyzer app and so it turns out melody extraction is really hard this is actually melodia in action so you can see like three different types of music there's jazz pop and opera and it's really good for jazz music which nobody listens to and it sucks at pop and opera which I know everybody here's a huge opera fan so when it turns out like so the blue here is the frequency that the computer estimates the melody to be in these graphs and the red is the actual melody so you know it needs a little bit of tweaking so it turns out this actually wasn't gonna be worth my time to add to my app so at this point I I'm decided to stick with Sonic API the first the first one because at least you know I got something out of it and I didn't have to write a new class but um I'm just I'm stuck with this now hold on where'd it go so is it perfect no it's far from it I mean the good or the bad thing is that I'm not getting paid to do this thing so like I'm gonna keep working on it slowly I'm I'm hoping that somebody eventually releases a better cheap or free melody extraction tool so what's next well there's a lot of changes I would love to make like I said a better melody analyzer I would actually there's no client side yet like this is all a command line tool and like I'll finish the client side once I actually have it working you know so I was like what else can I do well I was like I could pivot slightly slightly so I had this other idea there's this you know Shazam the song identifier application you guys have heard of that what if I use a reverse melody analyzer tool like Shazam to figure out which song a user uploads and then I have my app scrape the internet for a MIDI version of that song and meowify it using my piano library so I wish I had this already built out but I just spent the last two months moving I'm so sorry so that's one of my next steps and as a way to say like there's other things I'm gonna do I'm gonna save some money and some time so what if I store a song like in AWS or something every time somebody like inputs a song if it if somebody inputs the same song I don't have to reanalyze it I can just send them back the one that already exists so just a few little like good ideas like that I think they're good they could be really bad I'm not sure but um so that's that's it like you can find my slides on speaker deck you keep an eye out on Twitter I'll maybe be releasing meowifier and to the public soonish if you have any questions please feel free to ask me in person or on Twitter my handle is hobert dashery and you can find me these places and with that thank you so much