 Everybody hear me? Okay, this talk is about Open Source Music – Tracking 2.0. I'll start with the Open Source Music part. I mean, we've had music notation for centuries, Open Source music has historically been Open Source… …right? Not quite… be fact impressive We've always had humans playing instruments in order to actually get the music from score to performance And humans doing this stuff in real-time is kind of hard Looks a bit like this And this takes a lot of practice if you're really really really good If you can do it in front of an audience Yn ymwysig arwain fod ymwysig yn cael ei griwch darkwch o gydych chiad yn ein cyfor ddo, gyd-do chi'n trefyd wedi cael ei hunanio'r gwybarhau, i ken lawerwch eich gweithio'r greu hefyd. Felly, yn y cyd-dweithio'r gweld, rydych chi'n gweithio'r pian o gyda, a ydyn ni'n dynnu trefydau o arfer o'r gwybul. Ond nid gwneud i'r gwybloguion arfer, a felly, sydd yma i rôl, a gydy'r brosiectional yn meddwl ysgrifennu, ysgrifennu'r ar y rôl. Felly, os cymdeithasol gweithio ysgrifennu, felly, felly'n gweithio ar gwaith ag ysgrifennu. A gael o'r 1930s, mae'n gwybod. Rym ni'n gweithio, better, and better. On pretty much by the 1980s we had recording that was indistinguishable from a live performance, even to a trained listener. And there has been this assumption for a long time now that recordings are the best way, the only way to distribute music simply because they have been around for such a long time ac a llwg ddylch yn deilio'r technolol ychydig, ac bod mwyaf o'r dyfyniadau ym mwn o'r llwyddiadau, ac mae hyn yn iawn i ddim yn ystod. Roedd gennym ni'n gwybod bod y'r ysgrifennu o'r rhai, bod rwy'n gwybod am hynny. Oedd ym mwyn oedd yn gweithio'r cyfrifedig mewn gweithio'r cyfrifedig a'r cyfrifedig yw'r cyfrifedig yw mwyaf? Rwy'n gweithio'r cyfrifedig, rwy'n gweithio'r cyfrifedig, that we want to be able to tinker with it, we want to be able to put it back together again. We want to re-mix it exactly how we want to do it. You just can't do this with recordings, So, before computers actually caught up with enough memory and bandwidth and whatnot to be able to do CD quality audio, computers were used for music. This, I'm sure you've run into this stuff before, Well I'm suspecting you have, we had all of this stuff by the early 1990s and this stuff basically started on the Amiga but it spread very rapidly into the PC world and to every platform under the sun basically. And we've got these big four formats here and they were sample based by which I mean basically there were samples and pattern data which consists of what pitches and what order and what times to play the samples back at. This is surprisingly useful, I mean you wouldn't have thought that that was a particularly useful way of making music but not just because it was all people had but it is actually surprisingly flexible and a good way of doing things. And it has fallen out of complete, it's actually vanished at this point when MP3 came along, people no longer write stuff using this kind of stuff and they certainly don't distribute it. Using this kind of system. So just some kind of show of hands here, who has run into these formats before? Anyone written stuff in them? Has anyone written any software to play them? Any kind of, anyone written a tracker or play routine or a demo that has used this stuff? Okay that's good to know. Problem is they have an image problem these days to trackers. Here's a little quote from a few years back but things are essentially unchanged since then. In fact they've headed even more towards the idea that this stuff is retro and there is no point in using it if you are not actually interested in old technology for old technology's sake. I have some points of agreement with this guy. He is right that trackers suck. What I have a disagreement with is that they are unsalvageable as technology and the nice thing about them is that they are open source, they are hackable. You get essentially complete control, the same amount of control as the person who wrote the tune had and that is a really nifty thing to have and I think it's worth preserving. So why trackers suck? There's technical reasons for why actually people don't write stuff using them anymore. A lot of people who were well known for writing and distributing track music in the 1990s have switched to using MP3 instead. Who knows what they make their stuff with we don't know but they certainly don't distribute it using trackers. Trackers have their own problems with distribution which is people like having a single file format for all of this stuff and MP3 is the standard these days. What you don't want is this stuff originating in the demo scene and all that you can say about demo scene is they are not big on things like documentation and unit tests and what not. Actually getting a specification for these formats. The original implementation didn't come with source code. There were dozens and hundreds of people who went and re-implemented this stuff over the years. They all had their own quirks actually trying to get stuff to play the same on the gardens of the software. Trying to get this stuff to play the same on every playback software out there is a pain frankly. It's a hard problem and we have got this accumulated baggage of the fact that this was the overwhelmingly popular format for music. It was everywhere. People used it in games, in demos and we got to the stage where people were just cramming more and more into it. So people have stopped actually distributing this stuff as tracker modules. Instead the concept of tracking programmes have kind of morphed into what are called digital audio workstations. Which are basically sequences with digital signal processing capabilities. And even contemporary trackers that actually call themselves trackers and have got this demo scene heritage are starting to go in the direction of DSP plugins. And there is this standard called VST by Steinberg who made Cubase which has been around since 1996 or so. And even the standard is proprietary. But there is this whole ecosystem of people who are selling proprietary DSP plugins and instruments and sample libraries and so on. And it is all based on the assumption that the musician is the person who goes and buys all this stuff. They accumulate the library of the stuff they want to have and be able to use. Then they make their music with it, they mix it down, they have a recording, they distribute the recording. And there are massive amounts of assumptions in here. I mean, even things like Modplug and Renoise which call themselves trackers have got support for this now. But Renoise is particularly interesting because it's a proprietary payware app but they have a free demo version. And there is one, precisely one major feature missing from the demo version. It can do pretty much anything. It can load and save Renoise files. It has got the full editing capabilities. The single feature they left out is Render to Wave. They are betting that entire business model on the assumption that people who want to use this software seriously will want to mix down to MP3 or some other format that is just basically PCM. Which is astounding really. I think this is heading very much in the wrong direction. I think they are missing the entire point of trackers which is that if you are going to make this stuff on a computer then you should be able to share it on the computer. You do not have to depend on having the person who you are trying to send it to. Having copies of all of these proprietary DSP plugins and the proprietary software that they plug into and all of the other ecosystem. This stuff should be open source. But this is not the only problem with tracking. It wasn't well designed from the beginning. It was designed by demo-scenars and they were trying to solve the problem at hand. It has grown by a creation since then. Here is a photographer command set. It is kind of bizarre and ungainly and not very well thought out. There is all sorts of stuff in here and not a whole lot of orthogonality. Everyone who has come along since, who has been trying to make a file format has thought we want to be backward compatible. We take a copy of the Projacker command set and bolt bits on to it. It only gets worse from here. Here is the XM command set. The stuff on the left is the stuff that was in the previous slide. Then on the right is all of the extra stuff they added this time. It just gets incomprehensible quickly. Actually trying to learn all of this stuff. I have not got this stuff memorised and I have written a Projacker player. It is just crackers. I think that checking the assumption of... If we want to be backward compatible we need to have all of the commands from the previous... There is this assumption that pattern data is somehow sacred that you should keep the pattern data and you should modify the player routine to add all the complexity in the player routine to keep the pattern data untouched as it were. I think that is the wrong way to go. If we have a look at this again, if the stuff that we actually need there is a whole lot of stuff for pitch slides and there is a whole lot of stuff about volumes if we just for a minute imagine that we have abstracted out all the pitch and volume stuff what do we actually need out of this? It is not a lot. This is suddenly starting to look a whole lot more minimal. I think that minimalism is very much the way to go here. What we need to do is pick a good set of primitives a good powerful set of primitives and see what we can build with them but by keeping backward compatibility with the existing formats by having a sufficient set of primitives that you can express all of the stuff that they could do but without having to have all the baggage that they carried along and I am proposing doing this in JavaScript which sounds a bit cracker, but it is not that bad and there are reasons why I am doing this in JavaScript not least of which is that it solves the software distribution problem which is that nobody has these players anymore nobody can read these formats Apple did not even consider having a mod player in the iPod but web browsers these days are actually pretty good In the past 12 months both Google and Mozilla have brought out browsers that have sufficiently fast JavaScript engines and have a JavaScript audio API essentially a web writing to dev.dsp from JavaScript which suddenly makes a whole lot of things possible and nobody has really explored the possibilities what happens if you stick this stuff on the web and I think it is an interesting direction to go There have been attempts before that there are a couple of people who have tried to write a mod player in JavaScript but they are basically weak-hand hacks and as far as I know they are not actually attempting the editing problem which is the big problem actually writing a Pro Shack of Play routine is peanuts by comparison so there are people this has not been tried yet I am wondering if people might and we have things like CC++ to LLVM we have LLVM JavaScript back end so people can compile C++ code to JavaScript and there is a whole lot of existing code for mod playback out there and it is only a matter of time before someone goes and ports all of this stuff I give it 12 months tops so this time next year but it is not really the problem I am trying to solve because what I want to do is solve the editing problem I want to be able to hack this stuff and just having a player is no good at all there are people who have looked at the editing problem there are a couple of trackers but they are not really... one of them is a port of an existing tracker in Flash which does not count in my book a soft synth birth-based one soft synths are awesome I'll come back to soft synths I do ultimately want a soft synth but sample-based synthesis has an incredibly useful property which is it can playback anything if you can't do what you want with a soft synth and you are trying to make a particular effect or you have vocals or whatever you are completely screwed there is nothing you can do samples for all their limitations you can actually render anything you like to a sample it's not the direction I want to go in but it means that we can slowly expand the base of what is possible with a soft synth while having a backup of just rendering it as a sample which is a very useful thing to have and it makes life a whole lot more useful in terms of what you can actually achieve but all these web-based tracker people they are not seeing the big picture they are essentially looking at trackers as they have always been that is not what I want to do what I want is a web-based tracker plus a repository of music which is free and free I don't know what the licence would be best for all of this stuff I mean I'm claiming that this is source code I don't know if that means the GPL is best or not but I suspect not I suspect it's its own little niche that we'll need figuring out but sticking what is essentially a music editor together with a repository of music and music components and essentially making this into some kind of wiki slash github mash-up I think would be awesome so that is what I would like to do I have this idea for a project it is called Fouble it used to be called ModPlayJS what can I say I suck at naming it is still vaporware it does not exist yet but in the time on this tradition I have some kind of demo of what I have so far which I am hoping I will get to show you briefly this is running in Firefox as you can see there and this is playing one of the classic 4-channel mods called SpaceDebri and we can see some very sparse pattern data here let's just see if we can get audio out of this are we getting audio? there we go one of the nice things about this is you can see here all of the slides at the beginning have been pre-processed into a pitch envelope here this was the you heard at the beginning it originally started off as a whole bunch of incomprehensible commands out of the command set I showed you previously and this was pre-processed when this file was loaded it all in JavaScript to something that is a whole lot more user friendly now you can actually see from the pattern data there are pretty much no effects here given that this is a protracker file originally nothing in here is actually using protracker effects other than the very, very, very small subset that I was using previously because it has all the spin split out which I think actually getting this stuff to be usefully editable is a much harder problem but as a proof of concept of a minimal play routine I think this is pretty good we have got good support for loading protracker at S3M and XM is a kind of wonky but it sort of works and more formats would be nice but what I want to do is to settle on a good set of primitives to build an editor with that people can actually go make music with and it's just a bonus being able to load all of these existing files how are we doing for time? okay that's good we've got a few other bits and pieces here we have an actual sample editor using canvas and you can play et cetera and we've got all of the other stuff here but all of this is just basically a proof of concept I very much want to get this into some kind of collaborative music making tool and I've got a little quote to end with which I think sums up my position on this kind of nicely which is this we we yeah it is long enough it's worth it we tend to think music is a consumable end product versus something you build or assemble music is something you buy, something you download, something you play that's what text used to be too but putting text on the web and specifically assembling text using code changed that further just as we mixed text from different sources dynamically overlay, transform it, translate it, et cetera what if you could listen to music being mixed and altered live in your browser when music becomes algorithmic, scriptable and composable any number of new things will happen and this was from a guy at Mozilla who was trying, this was his rationale for why they should actually build this Firefox audio API the one that we've just been demoing and this kind of vision this is the world I want to live in I want control over this stuff but you do too if you do come talk to me thank you not work five minutes for questions I think my phone, Mike I wanted to point out that mod and files like that have kind of died off so ask a question and you can repeat it not so much about question this type of protocol and these sort of things has continued, it's just they're taking strength approaches to it around 1991 the playstation 1 developed a piece of software called MTV Music Generator which is basically exactly what they're doing on a console it's sort of like a tracker where you can input sample data into the file and create a file that has all this stuff already on there it comes with its own database of samples and sounds that you can keep it there the promise of time is the limitation on the amount of information you can insert this sort of thing has actually been continuing like people aren't developing this sort of software they're just taking strange avenues for it look up like it's the same idea ok so the non-question was about people having tried this kind of stuff on consoles at the playstation can you repeat the question can you repeat what he said I'm very sorry I'm trying people have tried to do a whole lot of this stuff has been around for a long time people have tried to do it on consoles people have written this kind of stuff for Game Boy people have made it for all sorts of strange platforms what they have not done is actually tried to a community of people actually sharing this stuff they have basically been in it because it's interesting from a technical perspective not because they think it's the best way to actually create music and I'm not entirely sure what to say about actually just at the minute have we got any more questions yes we've got two more we've got time for two more I've got one from our angel over here yeah what are the chances of this thing running another process API in Firefox on open standard or their own pardon what are the chances that your software can run in other process than Firefox you're asking about the Firefox audio API yes is the API of Firefox standard API or is this something is Mosella specific but Google are trying to standardise it Google have got a different one which is not the Firefox one this works in both of them but I suspect the Google one is a whole lot bigger and is this any canvas for audio but where is Firefox basically you write PCN samples was that your question last question have you heard of something called the free music archive I have not heard of the free music archive is it a is it a repository of recordings exactly what it sounds it's a publicly available repository for all kinds of self produced music based on the creative common license it's run by a station called WFMU in New York City it's pretty much this is exactly the kind of thing you should be looking at okay that's good I am aware of a whole lot of free repositories of music recordings I'm aware of at least a couple of places that have got free samples for a contribution yes okay well thank you very much everybody I'm very sorry there are more questions where can people find you if they have more questions outside thank you