 All right, I think we'll go ahead and start. So this session is about learning to read music with the Web MIDI API. If you want to get to the slides, you can get to them here, bit.ly, web-midi-api, if you want to look at any of this later. I'm really excited about this talk. This is the first time I've given it, and it's also, I think, the first time in many years that I've done a talk that was just totally based off of a random fun side project. So I'm really excited to be able to do this today. So I'm Jan Kleiner. With me is Ryan Jarvan. And we are both developer advocates at Red Hat. I'm going to be talking through most of this. Ryan's going to be helping with the demo, and probably jumping in also with more color as he goes. He's a lot more knowledgeable of music in general. I would say electronic music than I am. So he may have some interesting thoughts that I don't touch on as we go along. No pressure. All right. So the way this came about is related to this picture. So I have two kids, two daughters, and they both take piano lessons. And what you kind of can't see here at the top is this is a web app that their piano studio uses as part of their practice and their music theory stuff. And so it's really cool for what it is. So it shows at the top that you can't see right now the treble clef with a note displayed. And what they're supposed to do is click on the keyboard what note that is. So if it's middle C, they click the middle C on the screen. And it's green if it's right. It's red if it's wrong. And that's really cool. But what's always kind of bugged me about it is I don't feel like it's as transferable as if they were actually pressing it on the piano. So we got a new digital piano maybe about six months ago. And it had MIDI out on it that was like USB, which is kind of exciting. And that's what this basically came from. I wanted to create something so my kids could practice like their note homework basically. But by actually pressing the notes on the keyboard so it would really trigger in their mind, OK, middle C is there, not just like a picture on a screen. So that is what I'm going to demo for you later is that little web app that I put together. But in order to do that, I had to learn about MIDI and the Web MIDI API. And so that's what we're going to talk about now. So if you're like me, when you think about the word MIDI, this is typically what comes to mind. See if we can get it. You can only hear it through here. But this kind of sound is typically what I would think of from like mid 90s websites. What's that? Oh, that's OK. You know what it is. But MIDI is not actually that kind of sound. MIDI is not sound at all. You can create sound from MIDI messages. But MIDI is not sound and it's not that tiny kind of stuff that we're used to from old PC games and things like that. What it is instead is stands for a musical instrument digital interface. And it's a technical standard that's been around since the 80s. And there are several components to it. But essentially what it is in the context of what we're going to talk about today, it's a way for digital musical instruments, audio devices, computers, anything that can transmit these MIDI messages to communicate with each other. And there's a ton of things that you can do with that. But it's basically a protocol, a communication standard for these devices to be able to communicate this musical information. So to go into a little bit more detail on that, there are quite a few different types of messages that MIDI composes or comprises. We are only going to look at a couple things here. For what I'm doing, I'm only using something called a channel voice message. And even more specifically, the only message that I care about for this demo is note on. So what note on means is somebody has pressed a note. So in the sense of a keyboard, somebody has pressed a key, that would generate this note on message. And they've got this 144 in parentheses here because that's the numeric value for that note on event. So the MIDI message that would come through, it has got three parts. One is this event, this note on, so that would be 144. The next piece of information that comes through is the note number. I found this interesting. The note numbers can go from 0 to 127, which is more than the number of keys on a piano. So the notes that you can represent with MIDI are beyond what you can do on a standard 88 key piano. So I don't know if that's useful information, but I thought it was kind of cool. So the note number comes through middle C, my voice. Middle C is 60, which you'll be useful later to know. But so 60 is middle C. As you go higher up in the notes, the number goes up, and same for as you go lower. And then the third piece of information is velocity. And as Ryan is demonstrating over here, velocity is basically, in the sense of a keyboard, how hard are you pressing the key? So if you gently press the key, the velocity is going to be a low number. If you just slam it, it's going to be a much higher number. And so that can be useful if you are then taking this MIDI information and playing it back out in some way that can influence how you decide to produce that sound. In our case, what we're going to be doing, we're not producing any sound with this demo. Instead, we're using the MIDI information that's coming from the keyboard to match against what's on the screen and see if they're playing the right note. We'll look at that once we get to the demo. But so those are the three important pieces of information that I ended up needing to use in order to do this. Was in this channel voice message, I want to look at, I'm listening for the note on event. I don't care about note off or some of these other types of events that can happen. I need to know what note number was played, and then I don't really care about the velocity, but it's coming through for us. All right. So that's MIDI, but then how can we get that to actually work with the browser? And that's where the WebMIDI API comes in. So WebMIDI API lets you have MIDI devices communicating with your browser and with web applications. And the API itself is actually pretty simple. The most important thing to know first, though, is does the browser you're using actually support the WebMIDI API? And this is the slightly sad part. So this is the request you can use to find the answer to that navigator.requestMIDI access. If your browser supports it, you'll be great, and you can move on. If not, there are some alternatives. And here is the current state of the world. From can I use for WebMIDI, so Chrome supports it, Opera supports it, and like the Android, Chrome for Android and Opera Mobile browsers, but nothing else. Maybe that'll change in the future, but that's unfortunate. That makes it not as easy to use these things in a widespread way. But there are some alternatives out there. Now, I haven't actually tried this, but this is what I've always seen referenced as an alternative that makes this work in other browsers. There is, I think this is called, it's from Jazsoft. This Jaz plug-in that you can install, and then there's this package that you can use that makes it work more than just in Chrome and Opera. So again, I haven't tried it myself, but this seems to be the supported way of getting this to work in a broader set of browsers. All right, so going back here for just a second. So the first thing you need to do is see if MIDI is supported in the browser, you're in. And then what we're going to do is if it is, then we're going to actually request that MIDI access, and then we've got a success handler. If everything's looking good, we'll end up over here in our success handler. And so this is giving you kind of a sense of what this demo app is going to do. It's a scrappy app. There's probably much better ways of writing this, but this is what we're doing today. So to start, we have an array of notes. So I mentioned before, middle C is 60. So what I'm covering here is basically from middle C, then C sharp would be 61, D would be 62, D sharp is 63, and so on. So every note goes up by one. That makes sense, right? So we're going up to 72, which is the next C higher than middle C. The reason I stopped there is that's where this demo keyboard stops. So there's no sense in going any higher. If we were doing this for a full piano, there's probably a much more efficient way of doing this, but for demo purposes, that's where we're at. So what we're doing in here is I'm shuffling this array because I don't want to present my kids the notes in the same order every time or they're going to just memorize which note is which. So we shuffle that, then we're going to draw whatever the new first note in the array is. Draw it on a staff. And then we're going to get our list of MIDI inputs. I'm showing outputs here just so you know that you can output to a MIDI. You can output as well. I only care about inputs, what's plugged into my laptop. At our house, it would be our big digital piano here. It's this little keyboard. Could be multiple things. So where we are cycling through, I've only got one plugged in. So we're going to get our input and then when a message comes through, so like a key press, we're going to get that message and do something with it. Is this all making sense so far? Cool, okay. All right. So we talked about the note on command here. That's what we're going to do when we get a MIDI message. We're going to look at this message. Now here's the three parts that we talked about. The command, the note and the velocity. So those are those three pieces of data that we get. We have a switch statement with one case. That's exciting. But you could build it out. So that's why we've got that there. And note on is 144. So when we get that one, what we're going to do is I've got just like a little like text string that comes up to say what note was played. And then we're going to go into this note on listener. And that's where the actual logic happens. And rather than printing all the code out for that here, what it's doing, it's checking if the note that was played matched the note that we had displayed on the screen. So checking those note numbers against each other to make sure they match. If they do, it changes it to green. If they don't, it changes it to red. My kids have already given me a feature request that also needs a thumbs up and a thumbs down picture next to it. We don't have that today. Then after one and a half seconds, which is just to give us time to register the correct or incorrect feedback, it's going to display the next note in the array. So it's going to cycle through all 13 of those notes, giving you a chance to play them and get it right or wrong. And at the end, it'll tell you how many out of that 13 you got correct. So it's really simple, but it kind of can show you what you can start to do with this sort of API. And I love that the API itself is so easy to work with. That's really for a basic thing that's really all there is to it. So Ryan, do you want to? Now this isn't fair because this is a treble clef demo and Ryan's familiar with bass clef, so this was super unfair of me to do this to Ryan. But I don't want this to get unplugged. So I can give you cheat sheet hints if you want. Yeah, I may need some help because my music education was not on a piano and was definitely bass clef instead of treble clef. So we'll be making some guesses at these. No worries, we can work with the team. So this is our middle C on this keyboard. That's our middle C. So this is E. E, oh jeez. Yeah, that's going to be up a good, let's try. That's it. Let's try this one. Oh, very good. So you can see, it displays quickly, but you can see it's going to show you that the command, which is always going to be 144 because that's all we listen for, the note and the velocity. So now this is the... Couple notes higher, this is a H, right? Yes, hit it really hard. So the velocity is 50. Now on this next one, that's like half step down, so A sharp. Hit, oh, and that's what happens when you get wrong when you slip. What's the high C here? So the velocity is 107. So you can see if you wanted to do something useful with that velocity information, like maybe make the note get bigger if you press it harder. Yeah, yeah, we were talking about what you could, we don't really have this playing out audio, but we could wire it up to all kinds of fun things, like change the brightness of a light bulb if you had some IOT lights. The harder I smash the keys, the brighter the lights are, I don't know. Gales up pods in your Kubernetes class. Yeah, you could do all kinds of... Fun, exciting ideas. All kinds of home automation, where I'm playing creepy Count Dracula music in order to make the bats fly out of the chimney or something fun like that, I don't know. Instead what it does is it turns the green or red, but you get the idea. So you can cycle through these and actually I added something in to kind of give a very simple sense of other things you can do. If you go up, I'll do it because I know you're trying to hold it. You can also, so these lower notes down here on the keyboard are just changing the background color of the page just to drive home the point that like, it's musical information, but you don't have to just use MIDI for musical applications. Like you could use this to do whatever you want with. It's also something that is often when you're doing more interesting and complex things using the WebMiddy API alongside the Web Audio API would allow you to do a lot of really interesting applications. So I've seen some demo apps out there that are kind of like a digital synthesizer where you could do all sorts of cool stuff with it, but for this example, we've got this use case here. And we'll have these two keyboards in the red hat booth if you feel like stopping by and plugging in to test it out with your own code. Yeah. So, I think, did we ever get through to the end of it? Let's see if we can get through. Oh, now I've got it in the back. Oh, I'm on the wrong octave. So that's another piece of information. Combined score, five out of 13. So we need to practice some more. We need to go back to school. All right. So, and the thing is flashing like crazy now. I don't know what we did to it, but yeah. So that gives you a sense of how you can work, you know, a web application with this MIDI data. Something else in case this is new to you all like it was to me. So I'm just using SVG to draw the staff and the notes and then basically just I've got the measurements of like where the note needs to be placed relative to this staff. But I hadn't worked with SVG much in the past and I didn't realize you couldn't just like set the color of it. You can't, but you can use filter, a filter on it to get the color to be what you want. And it's really kind of interesting how that works. So you can like rotate the hue if you think about on the color wheel. And this might be obvious to everybody but me, but I thought that was pretty awesome to go from one color to another for the SVG thing. So, bonus. All right, so that was our demo. And then there's a lot of information out there that I also want to point to you. I used a lot of resources when I was figuring this out myself. One of the best ones I found was this article on getting started with the WebMedi API. There is an awesome example where they are doing, they programmed kind of like a lock code for almost like an escape room thing where you had to figure out the pattern of keys to play to solve a puzzle. Like in Goonies? Ooh, maybe. Like where they have to play the notes in Goonies to... Yeah, totally like Goonies then. Totally. I'm so horrified that I can't remember that part of the movie. But it's a really cool example. They've got some of the code there and that's doing a slightly more complex thing that's not just a pure music learning application to it. There's also information on the WebMedi API on midi.org which is a great resource if you just want to learn more about midi in general. This page here also has a bunch of links out to cool examples and demos and things like that. This is the slides. And then if you want to play around with the code for this demo, you can find it here. Again, like I said, it's scrappy. There's probably ways it could be way more efficient. But it works and you are welcome to try it out. Add in bass clef if you want. And let me know if you have any questions. So we finished a little bit early. Does anybody have questions about this or want to come play the keyboard? Yeah, no. I'm curious now though. Yeah, the question was about Google Magenta. I've seen some demos and it's definitely on my list of things to look into. I have not personally had experience with it but it looks really, really interesting. Did you have a comment on that? That's cool. Yeah, really cool stuff. He said if you weren't able to hear, start off a melody and it will detect based on the first couple of notes, it will follow this progression, chord progression and try to make something Bach style sounding based on TensorFlow neural nets and outputting MIDI, I guess. Well, I'm not sure, yeah, probably MIDI. Yeah, but Magenta sounds like a really interesting creative process to research. Yeah. Question back here. In here, we were just using one note. It looked like what about if there's a chord progression and there's multiple notes, anything special you think to handle those? Like, for example, I know human hands won't precisely hit the chord at the exact same millisecond, for example, but it should be considered. I don't know, what do you think about that? So in this, I don't remember the actual answer to your question but I know that there was some chord related stuff. In this example here, I remember reading about it. I don't remember precisely how that's handled. The note on callback or hook that you have can catch multiple note on events. And so just because one note has been pressed on doesn't mean that they've let the button go yet. You're just signaling a press, not a release of the button. So that's a separate event and you have to catch them both or the note will stay stuck on. So sometimes you'll have, when you're doing MIDI, if it misses the note off event, it'll just be, like have this long tone and you'll have to run a clear all notes command to basically signal that it should be clear. So hopefully you don't have to run clear all notes but these are all asynchronous. So that would be a poly, poly notes, I don't know, yeah, yeah. That makes perfect sense, I think, sorry. We're only keeping track of one thing at a given time but if you were keeping track of how many had been down and then were they lifted back up, that would work. There's also something, I can't remember the name for it right now but there's another message that can come through if you've pressed down the key and then you press more, like, yeah, see, I don't know these things. I just know how to make my kids do their homework. But that's another message that can come through if that's useful for things outside of my kids doing their piano homework. Was there another question? Yeah, this isn't really a question, just a comment to follow up on the note off. Some MIDI keyboards don't actually send note offs, they send note ons with zero velocity. So just like FYI, if you're playing around with a really cheap keyboard and all of your notes are getting stuck, like that's probably why. Good to know. Nice, nice feedback. Yeah, these controllers that Jan and I are using have after touch velocity, so you can play a note and then kind of hit it again and it'll send another, the after touch event. But that is something that's been kind of added on to MIDI in later years and it's funny, there's still some clear kind of limitations with the language of MIDI I think that one of the absolute limits since the 80s has been the range of values you can express has to be zero to 127. That's your, you know, the largest integer you can basically have within this. So you can do a lot, but if you wanna get really fine details, you almost have to fake the tempo to add more data or something. Whoa, revolutionary. It's the first I've heard. MIDI 2.0 is a thing and it's been out a couple weeks and it has floating points? Incredible. Oh yeah. Yeah, no hardware yet. Yeah, good luck finding a keyboard or an API for that one. That's cool, I didn't know that. Yeah, yeah, that'll be really interesting, more expressive quality I guess in the MIDI. Yeah, anyway, good luck experimenting with this stuff. 127 different values is still a lot to work with and you have a full range of notes and a full range of, there's 12 channels to work on, so you could send a lot of data over a MIDI bus and yeah, it's a fun thing to work with for all kinds of automation. So I hope you enjoy it and have learned something fun. You got any closing thoughts, Jan? No, except that I hope you learned a little something about MIDI and can do more creative things with this than I've done here. Yeah, stop by the booth to tell us your ideas of if you plan on doing any fun automation or MIDI projects, we'd love to hear about them. Thanks for attending all. Thank you. Yeah.