 Good morning everyone as that part of the slide says my name is Steve and as that part of the slide says I'm a little bit of a geek if we've all turned up to the right room I'm going to be talking about web MIDI for the next 20 minutes or so We're going to go through who I am what I've done to be on the stage What is MIDI in itself so we can talk about what is web MIDI and we'll talk about the protocol of MIDI and some examples Which means I may be playing some music, which is why I have a keyboard and a microphone It's just a excuse to show off really isn't it so who am I? Well, I've had unto deserve to be on this stage or as it should probably be called the ego slide where the speaker brags about its achievements for 20 minutes Not gonna do that because I haven't done anything Industry veteran by the way just means I'm old Developer and entrepreneur means absolutely nothing it means no one would give me a job so I had to create my own But the important thing is I have a term in This is the only musical instrument you can play without actually touching it Even though if you go to the museum of music in Brussels, there is a sign on the telephone that says do not touch Go figure The important thing I guess for the purpose of this is that I do like composing music. I'm just not very good at it so I Thought okay, I will do what I can which is code so for many many years MIDI has been the standard for musicians to Store their music into sequences and a playback now Mini means multiple things to multiple people When you say MIDI That's not enough MIDI sometimes will mean the serial protocol that goes down the cable Between synthesizers between computers and since the sizes or drum machines or other equipment like that MIDI can also mean the file format and there were three forms of file format They've been interestingly labeled format zero format one and format two Zero is pretty useless everything's on the one track, but it is very easy to pass We file my format one has got every piece of music on a separate track Which makes it very easy and many file format to used by almost nobody, but you're gonna know it exists Some people mean that when they say MIDI And MIDI is also the hardware specification There is a chunk of the spec which says exactly how much current has to go through the cable What sort of tolerances it has to be how fast it's running and all of that stuff? We don't need to worry about we're in the safety of the sandbox of a web browser So you don't need to worry about any of them, which is lucky because web MIDI is an API for that protocol and Nothing else all the bits that would be useful That are necessary in a sequencer are not available in web MIDI. Thank you for coming up to my TED talk. Goodbye It's just not almost nothing in there, so if there's nothing in there it will at least be easy to explain So you want to decide for the message yet? That is a MIDI message If I present it in hex does that help? No, at least one person's nodding well The first part of this message is the command one for 499 and the second one is the channel now a couple of things you can work out from this Hex numbers four bits long from 0 to 15 that means there are only 16 channels That's part of spec. It's been there since the year dot is not going to change Unfortunately, however, MIDI web the MIDI spec version 2 has just been announced So this talk will be completely redundant probably by the time I finished giving it The first bit is the command and again, you'll notice It's a single nibble in hex, which means there are only 16 possible commands. It's another caveat. So two possible things This command dictates these bytes So serial protocol they're coming one after the other and it will your program will see that first byte It says what is this? It's a note-on message at which point it understands that the following two bytes will be a note value and a volume So the obligatory hello world Apologies for the use of the word navigator in there. It's probably not really necessary But this is the thing that will take in a message Passes it out and says well, we know note-on is this 90 and we can track it out to the console So to prove that works Probably shouldn't say that and then do a live demo. I should do the live demo and then so let me clear that If you look onto the bottom window One message for on one message for off still only three bytes each time 64 volume one Where's the time? Where's the timing information in that message? See that can anyone tell me where the time is gone? The time is in real life All the time in Web MIDI is controlled by how much time you wait until you send the next message There is no other way around it So you are forced to rely on JavaScript Giving you very precise timing in order to create very precise music This is why I titled it garlic bread Is anyone aware of the Peter K sketch the stand-up he does where he goes garlic bread Because it's just incongruous you have a MIDI Protocol that is for music that is real time that needs to have very precise timings And then you go and stick it in a web browser Which is sandbox and is not very good at precise timing Which makes it a little bit painful at times if you try to do anything live So let's continue with that so If it's a serial protocol, there is one problem with handling it How do you know if you're in the middle of a message or not? We've established this is the note on command This is the note on message. Well, what happens if the first byte we read is 60? What command is that? What's it suddenly gonna start doing at that point? What if we come in here? What about a hundred? Well, the clue is by writing it down in binary So if this most significant bit is one it means this is a command Something is happening. Now you should be paying attention and if the most significant bit is zero it means this is the data for a previous command Very simple to pass. It's one operation and present zero x eight zero But I just said there are 16 possibilities for this number if we're using the most significant bit Then actually we only have eight possible commands And that's it. The entire specification is just eight commands And that and that that's these ones so note off is the most important one because it's the first one No idea why note on is the second one? Although you will find and this is where it gets fun that hundred if First byte is the command the second two bytes of the data bytes now It like you have any number of data bytes you like they're controlled by the command This is the volume Some keyboard manufacturers have decided that if you put zero as the volume on a note on message that means note off Because obviously it's logical right So you've got no on and off the key pressure just for hitting it harder essentially parameters and programs for changing the sound because although there are no sounds in the MIDI spec and there are no mid sounds in the MIDI files There is still the indication that there would be a sound So you say I'm just telling you the note. I'm saying this note goes on this note goes off I'm not telling me any more about it, but I will tell you I would like the synthesizer to play a piano or a string sound Doesn't tell you what the string sound is or how the string sound is formed just something in the general area of strings Changing the pressure more after touch thing the pitch wheel obviously this expensive keyboard does not have pitch wheel But if you do then there was a special message for it and then there's the meta events for everything else It's like they started designing the keyboard and they thought yeah, we need notes on we need the ability to change the program We need the pitch wheel. Oh dear. We've run out So everything else is lumbered in this bottom stage and there are just hundreds of these meta events system s Exclusive messages that send patches. It's just full of them This one I know it's it I know it's got each file because this is that This list comes from my original C version of this library And it just lists them all in there So what is message or what we're missing in web MIDI will have established the timing message is missing That is just something you have to time yourself When you deal with MIDI files and I'll show you some bits later you have to specify that time So you have to count the time yourself So you say this note comes on then this note comes on and there is half a second between it you then write that in The duration again if you don't have timing messages, you can't have duration So you have to handle that yourself think about how long each of these notes are gonna be chords Cords are kind of fundamental music. This is harmony every single piece of reasonably good music will have harmony and Quite a lot of bad music will have harmony as well But there is no way of describing a chord You have to say this note on then this note on and then this note on at the same time or As close to the same time as you can possibly manage in your code Ideally you'd want that time to be zero or at least shorter than 10 milliseconds and There are no conventions. So if you use one of the system exclusive messages It's up to you how you decide to handle that Every keyboard manufacturer has their own set of things that they put into those messages And if you want to process them or understand what that keyboard is sending you're on your own so We've got the protocol is a stream of messages and I was going to give some examples, but I Will have to go out of here for that Don't be shut that down This is the one that I've been fiddling with for the time It's they put this up as loud as I can and then put this on switch to base Now you can tell that there is a little bit of a latency on me pressing the keys and hearing the notes which is unfortunate But that's unfortunately nothing of my doing. I'm not that bad a coder It is just the time that it takes for the message to come through and the browser to react and the sound engine inside The web audio engine to play those sounds Because there isn't anything much is really doing there That's a web audio backend being triggered from from the from the JavaScript in the browser Now if I that's not good. This was just an example. This has got nothing to do with this is an example I was playing with earlier. I was curious about waveforms and how waveforms have created. So I built this web audio thing to Create waves using different harmonics. I just needed time to hook it up to the setup today But if I show you another one like this soundboard for example, if I play notes on here now I didn't get the looping quite right on it, but All I've done here is I've listened for the note down message. When I get that note on message I trigger the sample loop. I don't do anything else when the note up comes I don't stop it when the second note down comes. I Stop it. That's logic. That's one variable. That's one check Which you can have hours of fun with Now while I'm doing that you might hear something else going on There's a bass sound going on under the underneath each of those samples that bass sound the bass sound From that window When you open web midi and you say I would like to do some web midi Please mr. Web browser and the web browser goes sure which would you like? And it will allow you to enumerate every midi device plugged into your machine Which is great. It means if you've got multiple keyboard You can have one keyboard doing this thing and one kid or doing that thing works brilliantly Unfortunately, it works brilliantly for everyone else as well So if you happen to have two apps that were running in different browsers, it means you can see the other person's traffic And I say this because I'm using web midi for music stuff mostly well as much as you can call that music just a loop But because I'm using it for music. I think very much in the music sense, but these things are cheap It's a tenner of anyone's money. There is no reason why you can't take that apart Use the switches and hook them up to something else You might like to do a game controller, which is more accessible than traditional game pads That would make a very cheap and easy Accessible game pad controller and it's controllable through the web browser with web midi Unfortunately, if you were doing anything sensitive like you were requiring someone to put in a passcode using those controllers it would also be visible in another window and Currently as it stands There is nothing to stop you It will be visible in every window and you can't stop you can't say oh, I want this for myself That controller is available to everyone and you can start that up on your web page Without a click without the user having to perform things if you've done any kind of web audio things in the browser You'll have probably noticed that it doesn't work until the user has actually clicked on something It requires the user to interact with the page before it gives you control of the web audio device Not the same for web midi. You can just look at that traffic at any time another example we had which is probably the interesting one I suppose is That one I think and of course it's still running in my other window, so let's shut that down I Was about the closest I'm gonna get so I'll leave that they are a quick one on my head Put that back on Now this is obviously a proper example of a sense which I didn't write But it is quite fun and is it is again using web audio and Web audio as you say you've got to click here before you can actually make sounds But it's a nice example and you can see that it is possible to build quite big and fun things and at this point I realize the person who's written this might be in the room Nope good That would be awful. It's happened before actually sort of mid lecture activity thing I was at a computer thing Funnily enough me computer thing who'da thunk it I was at a computer thing And I was we're talking about old computers as my friends often do and I happen to mention this trick on the ZX 81 About how you could put components across the chip to fix the Square root bug that existed on the ZX 81 at the time and I was explaining how it worked and why it only worked in that case and then I found out the guy sat next to me built it and You know the mansplaining level was a 10 out of 10 at that point. I mean fanboy was off the charts But it's what it's one of those things. I'm sure I'm going to give this talk and the person who wrote it It's gonna be here Which is a shame because it's really nice. I mean that's And it works real time. So I remember back in the 90s. You can make a whole album with just that I Don't think I should have done that. Let's do that instead. Oh, so I got some library links. So so as I saying I Use this for music, but you could create a game controller by ripping out the innards and putting that into you know Some sort of something else as we say, it's not keyboards drum pads They're very Very fun. They're very easy. I like those sort of things the innovation lump pad Which I can't quite afford, but they are very pretty and I have lots of lights on them and You could do you can control lighting rigs When I did some theater stuff a while back the lighting rig that was a big desk but controlled by midi Same protocol note on meant light on like a note off meant light off But it's it's doable. It's a it's a standard protocol that seems to have gone beyond what it should have ever been doing The rate of midi is very slow. It's about 33 100 Bowel rate essentially Well, she's slow and you think of how many notes could be playing in a piece of music Even if a fairly straightforward symphony with only maybe 50 or 60 players There's 50 60 notes in there minimum You've only got 16 channels and you've got multiple notes So you're probably not going to get all of them playing at the same time because you can't play all of them with zero time Gap there will always be a time gap between each note. So it'll never be quite right This is the the slide you might want to take pictures off if you want to go and play with that synthy thing later Also, there is No file handling. Oh, sorry. Did I not did I go it? Okay for those that want to take the picture. I'm ready for my close up So file handling is missing if you want to save the midi file You can't if you've generated something nice. You've got to save it yourself. Don't worry. I've written a library for that There'll be a link later for as well The sysx abstraction. So any as I say any Manufacturer such as Korg and Yamaha They will have their own system exclusive data Which you just have to go and find out for them and any actual sound the actual noises themselves not in the spec It's just the indication that there is a piano sound here If you do want to have a look my midi library is the second one these other things are also perfectly valid midi libraries My one was based on the C++ library that I wrote many many many years ago I think my original midi library is actually older than this conference and there's not many things that are So those were available if you've got the pictures now. Yeah So my sort of basic conclusions is the web midi is really interesting for input Lots of things support it not just the keyboards the drum machines and all that sort of stuff Very good for handling you can build your own, but the output is more interesting Because now the browser can control synthesizer rigs and it can control drum machines But the number of people wanting to do that is probably even fewer than want to do input So with that I shall leave it open to last five minutes for questions I should update my foster m scorecard That's now the 12th different talk I've given and I still haven't got good at it So we've got a microphone if there's questions And I'll have my beer So it kind of looks like Someone took a naive translation of a wire protocol and dumped it straight into JavaScript without any kind of design process What was there actually a design process or did someone manage to persuade the Chrome team just to put in this API that no one had really thought about I don't know While how web midi came to be I don't know the people I suspect That looking at it they said it's not so much dumping the wire protocol But dumping what every other midi library does every other midi library just gives you that raw data here is the stream Partly because you can't process the sysx messages There's no simple way of abstracting that other than say it's a sysx message and here is the binary blob They could have been abstractions. They could have created an API which said on You know with with an event and you say I want this function called when I get a note on event But they didn't do that. I think because they wanted to maintain a direct relationship with all the preceding libraries and the way that people wrote midi applications But that's my conjecture Yes, my library has attached off off the github there does abstract away a lot of this ugliness And it handles that odd stupid case as well where if the note is on but the volume is zero it actually means note off Which is one of those things that fox me for ages There's also the odd thing as well in the midi spec that if you do a note on message and then another note on message You don't have to say note on again You just keep going So you don't get that most significant bit being one But for the next message So you just see note on is a three-byte message. Why am I getting five bytes? That's just because there's that quirk that was in the original midi spec because it will cut down on the bandwidth used You don't have to send a bite and back in 1982 when this midi was created that one bite was important Now it's not but we're stuck with it So we've got these little caveats in the original spec that have carried through to Web MIDI have a microphone for you So how stable is the web MIDI stuff could it be used for editor librarians for older? Synthesizers like I have some old stuff that I need to use an Atari emulator to actually edit stuff on could web MIDI be used to reliably do that Do you think is it stable enough? So is web MIDI stable enough to use as a librarian tool and so forth. It is stable enough Where the media as a spec isn't gonna be changing the API is not gonna be changing There will be upgrades for web 2.0, which is coming through the pipeline now But it's more the case of do you want to spend that amount of time writing a librarian tool? When there might be better ways of doing it It would probably be good for you It would be good for me some of the time But because it's coming quite late into the game I'm concerned that it might only be used for fun projects like this Rather than serious projects that have already got librarian tools and other system tools But I'd like to see it more done more I know as soon as someone puts a VR headset down, I know that interest is going there and not here Okay, so can we must the two minutes of interest No, no Thank you a lot about gracefully. Thank you very much. Thank you