 Hello, my name is Charlotte and I'm a software engineer. So in the last year I finished my degree in computer science and what I did for my funnier project was to try and create an app to solve what I think is a reasonable problem. So the problem is that we have sheet music, which this is a kind of basic definition for anyone who isn't into music, which doesn't really tell you a lot about what it looks like. So it's kind of a symbolic language which combines human languages with symbols. And all you really need to know about it in order to understand kind of why I think this is a problem is that it's pretty complicated. So up and down on the five lines that's how we recognise what pitch or frequency we're trying to produce with each sound. And then horizontally we've got time. Each unit of music is called bar, those are delineated by the vertical lines. So yeah, it's pretty complicated. There's a lot of information that people are trying to convey, but this particular line of music is only for one person. So it gets even more complicated when you add more players, which that's usually what a composer is trying to do. They're trying to write something for an orchestra or for a group of musicians. So that's why we need it to be so specific because unlike a speech that I'm giving right now, every single musician could have their own interpretation of how exactly to perform the music. So you need to be able to control it so that the composer can make it sound exactly like how it's sounding in his or her head. So I've titled this slide which you can't really see, a multidisciplinary and multi-faceted problem. And that's because computers can't read sheet music. So at the moment we're trying to solve the problem of organising it, which I'll come on to in a minute, by having stacks and stacks of physical sheet music. And there's several problems with that. One of them is if you have an orchestra, then you have a big collection of music and finding a place to store that without it being destroyed in some kind of way because spaces like that tend to be a bit damp and stuff is difficult. On top of that, there's a lot of it. And there's a lot of information that we can't get without actually physically looking at the music and saying, this is in 4.4. This is in whatever. So essentially I'm saying we need a sort of search engine to be able to do this. The next problem is like I say, computers can't read it because it's not English. It's not a computer programming language. And the formats that we have for the moment are all images. And there's already a lot of problems with us trying to get computers to see properly on images. So in existence for teaching a computer how to read, we have optical character recognition. I don't know how many people have actually looked into doing research on this, but we haven't perfected that yet. So trying to perfect something that's a little bit more complicated than that is difficult. Personally, I have this problem because I am a musician. So as well as a software engineer, I've been playing a clarinet for a long time. And on top of that, I have two elder sisters who play different instruments so they're playing an oboe and a flute. But we all kind of play a lot of different instruments because that's all what musicians do. So houses and stuff, we have this physical collection of sheet music that I was talking about. But it's all kind of stored in one cupboard with two shelves. And sheet music tends to be in leaves so you can't really see the book binding or anything to be able to tell you what it is. So there's several problems here. One of them is like how do we pick an organization method that works for all of us? So do you try and organize it by whose music is it? In which case that's difficult because some of us play similar instruments. So we'll all play a saxophone, we'll all play a piano. So that means that if you're trying to do this physically without any kind of computer method, then you're going to duplicate part of the music. And then you try and look at organizing by instrument. But there's a varying amount of music that we have per instrument. And then you can look at composer, so on and so forth. There's problems with every single one. And the fundamental thing is that there's no one filter method that I'm looking for at one particular time. Usually I just want to play something. And that's difficult to do when you've just got it physical. So as I've mentioned a little bit, orchestras tend to use the one on the left. And regular musicians who aren't playing in an orchestra tend to use the one on the right. And this just doesn't really a great system. So that's why there's the job title of music librarian who tends to know a lot more about their collection than anyone else. So they can try and dig through and find exactly what they're looking for. And they'll probably have some kind of system like you find in a library. But yeah, I'm just saying that physical isn't really a great storage method. If you look at the one on the left in particular, that one doesn't really help you to see the music itself because it's all flat and the book binding will only actually tell you like the composer and the title. Sometimes you want more information than that. So what software is already available for this kind of thing? It sounds like a problem we should have solved already. And to some extent, there's already been research into being able to create music. So this is Sibelius, which I don't know about you, but that looks pretty complicated. And there's reasons for that. Mainly, it's music is complicated. So it's hard trying to make an interface which is usable by everyone. I don't really know any other reasons other than that. It's just a pretty bad piece of software that a lot of people find difficult to use. But the issue that I'm talking about here isn't to do with composition. And basically, I feel like we're missing a step. So if Sibelius can create it or any other open source software such as MuseScore, then after that, we haven't got any way of organising it other than putting it into a series of folders and PDF files that computers can't understand. So a lot of musicians end up just printing it out and giving it to their fellow players. And that sort of ends our foray into using software. So I decided I'd try and solve this. So my idea was a way of organising sheet music, which I've already discussed. The problem that I originally wanted to solve was trying to get a computer to understand music, but I only had a year to complete this project. And I've already mentioned how complicated that is. So I decided I'd do the step after that and try and make it so that we'd got a platform to work with beyond being able to get computers to understand it. In addition to this, the problem I've not mentioned is that if you ever try and buy more sheet music, in general, again, it'll be printed. You'll try and order it off Music Room or something and they'll send you a book. So that's just kind of continuing the problem. And I haven't really tried this, but I can't imagine Google really understanding that 4.4 means I want a piece that is in this particular time signature. So there's not many search engines out there that already cater for you being able to expand it if you wanted something that's not physical. So that's kind of a rough proposed idea that I decided to work on. Then there's my actual solution. So the first thing I needed to look at was what format I'm going to use, because it needed to be one that is already used by composition software so that we could make it so that this step is almost seamless, but one that computers can already understand, and preferably one that's open source so that I could use it. So I came across MusicXML, which is already an output format from Sibelius, MuseScore, Finale, and probably several other music composing platforms. So my program needed us to be able to render it, because there's not really much putting together a search engine if you're going to just let people open up a PDF anyway. And it needs to be able to provide some kind of interface for people searching it, and preferably one that's not just... I would like, preferably one that makes it as easy for musicians to use as possible, because that's who I'm aiming at. Not aiming at software engineers who can learn a long list of commands. I'm aiming at people who won't necessarily have used a computer as much as I might have done. So I'm aware I'm a little bit of a special case, so I might not be the best person to use this software. I also wanted to be able to expand by using the same kind of model of data. So for musicians in the audience, I tended to pull out the clef, the time signature, the key, and a bunch of other stuff which I thought would be useful. So I wanted to be able to find an online source which would let me search for all that. I'll explain a little bit more about how I did that in a minute. So for developers in the audience, I had several different choices to make about what tech stack I was going to use. So I went with Python, partially because I like Python, partially because I'd had a reasonable amount of experience with it, but also because that's a well-used open-source language. I open-sourced everything that I did on this project, by the way. And also, I feel like if I brought more musicians onto this who may not have as much experience in programming, I know there's a lot of programmers who are also musicians like myself, then Python's a little bit friendlier for people to use. PyCharm's just a personal IDE choice. Then I needed something to store my data, so I went with SQLite, which is quite a nice format because you don't have to install a full database stack. You don't have to have it online. It's used by multiple different programs already, and there are famous ones, including iTunes. And it just basically lets you create a file on the person's system and be able to use SQL on it. On top of that, there were some decisions over graphics libraries. So in Python, there's not many people who are actually writing stuff with graphics libraries. It tends to just be little scripts that people have written. But, again, I've already mentioned my target audience isn't tech people, it's musicians. So I wanted to have a GUI so that it would be nicer to use. So PyQT is quite nice because it's cross-platform, which was another one of my aims. I wanted to make sure that everything that I was writing was available to Windows users, OSX, and Skinny-slash-Ninux. So it's nice because it's cross-platform, it's got designer, you can style it using CSS. I also had to use Poplar for PDFs, which caused me a few problems. Beyond that, XMLSax parsing, which is just a way of parsing, and I decided to do test-driven development because I've already mentioned how complicated music is. So I wanted to make sure that my system would have as much test coverage to make sure that I knew that anything that I was changing wouldn't affect anything that I'd already written. So I got to around about 3,000 unit tests, which were mostly for the music parser. Beyond that, I did continuous integration after I'd handed it in, which I think is quite useful for the same kind of reasons as doing unit tests. So now I'm going to try and do a demo. So one second. There, okay. So I don't know how well this is going to come out, but this is the platform that I came up with. So basically what it will do is at the beginning, you can pick a folder, and the intention is you'll store all of your music XML files in that folder, which are probably going to be generated from Sibelius or from MuScore. My preference is toward MuScore, just because it's free as well. You should donate to it, but anyway. So down here on the side, I've got a list of all of the pieces, and I can organize those by composer or lyricist, or title obviously. I think there's also a file name option maybe. On top of that, I've got my own playlist. So this was kind of with the idea of a lot of, there is some software out there that will let you organize sheet music, but in general, it's very manual processed. It might pull out the title for you. It might pull out the composer, but beyond that, you have to add your own kind of tags to everything. So my own playlist was a way to kind of make a set list if you wanted to. Then comes the kind of fun part. So I got my processor to pull out a bunch of different stuff. So this is a big list because I've got a lot of music in here. So here we've got time signatures. So if I click that, then that'll give you a list of all of the pieces that are in twofold. And then you've got like a lot of other information. So you can simplify that and pull out like keys if you're interested in that or clefs. What I've tried to cover as much information that I thought would be useful as if you wanted a search engine or something. So that gives you a list. And then if you click on one of these, this is actually a piece that's downloaded online. So that's not my own parsing. That's a PDF that someone else has rendered. That'll give you in this screen like a list of all the information that we've pulled out. So we've got the title, we've got the composer, instruments that are in it, et cetera, et cetera. And then beyond that, that should give you a list of all the playlists it's featured in. And if you wanna go back to the playlist that we just clicked on, then that will load up all of the other pieces that are in that queue. There's a bunch of other stuff that I put in. I put in this to try and make sure that if you've got a piece which has multiple pages, which this does, and you've got multiple screens, then you don't have to do the really annoying page turn problem that we usually have with physical music. So beyond that, I've also got the search engine up here. So this'll do a little bit of processing. So it can, so if you type in a number divided by another number, it'll figure out that's a time signature. If you do a beat type and what's just happened, okay? Quota equals 80. Then it'll figure out that that's a tempo and it should also show the pain with an online option. So that will be music that's not necessarily in your collection. You can't really see that very well on the screen. But there'll be an option that's not in your collection but you can download it. So it will have got all of the information about each piece that's in the sources that we're using. And it will, like once you click on that, then it'll download a PDF. So if I go back to my slides, and that's the initial page that'll let you pick a particular collection. So the screenshot that I've got there is the Ubuntu version. So I tested it on all platforms. It should work on all platforms. I haven't run it recently. So don't quote me on that odd. But yeah, I was pretty happy with how well I'd done on that. And it turns out so was my university because I came out with the award for best project. So I was pretty happy with that. Thanks. Yeah, so I was pretty happy, not just because of that because I'd managed to make an app that looks okay. I think a lot of problems with people who do this kind of research is that they're very focused on what features you can throw in. But it's like if your features are all broken and it's not actually usable by anyone, what's the point in writing the feature? So I got to a point where I was like, I'm happy with what I've got in. I'm gonna try and make it so that as many people can use it as possible. And that looks very good to universities because you'd rather have a finished product, which maybe has half of the features than a broken product, which has all of them. There's always something that goes wrong. So part of what went wrong was my conversion from music XML to other formats. So something that I don't think I mentioned was that I didn't do the PDF rendering myself because that's not really what I'm aiming at. People have already tried to solve this problem multiple times. There's already, as I said, there's already composition programs that'll let you do it. And there's already open source options for you to be able to do it without having to write the rendering yourself. So one of these is called Lilypond. So if you're particularly academic and you've used Latek, Lilypond is sort of a add-on, not an add-on to Latek, but it's built in the same kind of way. The syntax is very, very similar. But the thing is that music XML is more of a visual language and Lilypond is more logical. So I'll take a very basic example. This is what's called a repeat bar. So essentially, in a kind of logical way, I'll explain what you'll play. So you play the first bar, which is the first two notes, and then you play the bar marked with a number one. Then you go back and you play the first bar again, and then you play the bar marked with a number two, and then you get to the next bar and you finish. So what's interesting about, what I found interesting about this and what you might not really think about as a musician, because I don't know, to musicians by this point, or at least by this point in my life, like this is a very, this is sort of like reading English. So separating myself from the problem was kind of difficult in places. And this is one of them, because on the left we have music XML. That's the blue and red syntax highlighted stuff. And if you properly read that, then you notice that it marks out the ending, the ending marker of the first bar, which is got a number one over it, then the end of that ending. And then a repeat bar line at the end of that bar, which is exactly what we're seeing. And that's fine if you wanna think about it like that. I would rather think about it logically and explain how you're gonna play it, because that's what's important. So on the opposite side, we have Lilypond, which is almost what I explained in the sense that you've got the repeat, which explains to you that you're gonna play the first bar twice. And then you've got your alternative endings. So these will usually run in order. But what's difficult is getting between the two, because it means that the objects in the middle, which I wanna specify that I didn't make a converter for music XML to Lilypond, because I'm aware those already exist. What I did make was a program which would take in music XML, pull it into kind of a tree of objects, and then output Lilypond. And the difference is kind of subtle. Basically, it means that I don't have to write several of the parsers from music XML to another file format if I wanted to do another output format. And similarly, I don't have to write a parser for a different output format or a different input format. On top of that, it means I was more able to pull out the metadata without it affecting any of my rendering process. Okay, last two slides. So I've got to hurry up with this, but the other problem that I had was working with different platforms. So the main thing is that PyQT is a very nice thing for you to work with if you're doing cross-platform. But as I mentioned, I had to put impoplo in there. And that's a PDF library because PyQT doesn't have one. So I basically had to, I don't know, I had to dig out the Windows binaries and that doesn't work very well because not a lot of open source people tend to like Windows. So you won't tend to find people building many binaries for them. And that was a little bit difficult, but I eventually tried to solve that and I did get something written for all three platforms. So in future, I'd like to do a little bit more of this. I'd like to expand it a bit more. I'm sort of imagining this as kind of the bit in the middle between whatever else we're gonna do with it. So I'd love to do optical music recognition if I had the time. Or similarly, I'd like to try and find an open source alternative to me doing it myself because I've already mentioned how complicated it is. We also have stuff like nGram viewers which allow you to search by writing some music, which to me seems a lot more, I don't know, intuitive for musicians. And obviously you can see there's a keyboard. So if you play piano, then you can just, you can use a search engine using a piano. There's other things to do with page turners which I'd like to integrate, which would mean that it would be more of a view for orchestras and stuff. So thanks very much for listening. Top one is my blog. The other two are the links to my GitHub repos, everything's open source. And finally my Twitter handle. So if anyone has any questions, I'll probably hang around outside somewhere. Thanks.