 Hello everyone So we're gonna present this work No, sorry for this Platforms called Bella. We're gonna do a short not the short workshop after that and So Bella is an embedded platform for Audio and sensor processing the spin developed at Center for Digital Music at Queen Mary University of London by Mostly by people that are not here as in started with work from Andrew by Andrew McPherson and Victor Zappi a couple of years ago and they Had to make a hackable musical instrument, which should probably be somewhere well I'll show I find it later. They had to make a hackable musical instrument where you could Sort of break out the internal connection of digital musical instrument so that people can hack them could hack it on a Breadboard like as if they were doing circuit bending on an analogs instrument and to do that Yet you needed a few things like you needed plenty of Ios possibly at all your rate or similar so that you could use Normal values of capacitors and resistors to actually make some something happen You need low latency because there were multiple in and outs and you're you needed the signal the signal need to go In and out a few times And that's why this platform was designed in the first place And now we just released it to the rest of the world as a platform for all your generic platform for all your sensor processing We did that through a quick Kickstarter campaign that was closed that ended 10 days ago or something and So yes, let's see what it is Yeah, good. You see I rehearsed the slides very well so The idea was to get away from cumbersome Solutions like having an Arduino plugged into your Raspberry Pi So that you could read them through pure data. So you could read gp the digital industry pure data to make some sounds Or having it plugged into your laptop or your or your General desktop machine Or using Raspberry Pi on its own each of these have different levels of Different reasons why this platform get that and this platform sort of tries to tie all them together Of course on the Arduino You can't really do any serious or your processing you just don't have enough power But you have the advantage of being able to connect to at very low level with any sort of hardware on the Pi I guess you could pretty much Similar IO stuff you have digital GP Ios you probably have Analog ins as well But it's embedded platform So it's great because it can be embedded into musical instruments and sort of things but you need a Custom environment as we'll see later to make the thing actually work Properly I try to run Raspberry Pi with a USB dongle as a audio in and out and I was getting some Some 50 milliseconds latency with with PD so that wasn't really something useful for playing music Of course, you could you you could use a little your laptop But again, you would you would lose at that point a low level connectivity and this platform tries to package everything in a convenient package Which is built on top of a bigger one black. We're actually using a stock bigger one black and a custom Cape Custom Bella Cape comes with those eight analog inputs eight analog outputs are sampled at 22 kilohertz You can actually reduce the number of channels to get higher sampling rate. These are DC coupled string DAC and What's called again a successor approximation? ADC's so they're a bit noisy But they're very fast. They have no built-in There's no latency associated with them as in as it as in studies case for the audio codec Because we also have audio input and output stereo audio input and output But these converters are sigma delta So you have a round-trip latency of the converter itself. That's around 38 samples We'll talk about this bit more later so There's a custom software environment that that does match that's much with the Cape that actually is based on the design of the of the of the audio Cape for the bigger one black But doesn't even she doesn't have an ulcer driver. You can't use it as you would use a regular Audio interface simply but because of the fact that we're bypassing the audio kernel using the Xenoma extensions to get to give the audio thread much higher priority than the Linux kernel at that point you couldn't go through also and This allows you to achieve buffer size of two samples Which may be which brings you to less than one millisecond relative latency for the audio and that one millisecond is mainly due to latency to the Filters in the converter You could get to rest less than a hundred microseconds round-trip for the for the analogs if you really want to Yeah, so the sensors the audios Analog inputs now with a sample that 22 kilohertz, which is pretty remarkable Which means you can do sort of audio rate. I always there for instance you could use them for You probably wouldn't want to use them in the audio path even though you can But you may want to use another for some side chain on a compressor for trigger inputs those sort of things from from a Piezo for instance or just or just to get to that 22 kilohertz sampling rate from next around just so you can properly filter the data These are synced up by the audio clock, so they're aligned and we're also using the onboard GPIOs 16 of them To get also 16 digital inputs and outputs inputs or outputs The Zenomai Software the semi on the software side Zenome extensions allow to give the audio thread higher priority Then the Linux kernel as we mentioned earlier and so in practice you get a real-time audio environment So you get the same Real-time performance of a microcontroller in a sense because the current what the kernel does can't interrupt your audio But at the same time you get the full stack of Linux if you need to do anything like storing files doing network SSH in compiling on the board those sort of things Right now, so it started using C and C++ now there are ways to run pure data patches on it Using a couple of things we'll see later and recently we had the support for Faust and We're working on getting super collided done and but there's more in this later We targeted mainly to musical instruments and live audio performances and You know music installations would be the whole thing in the case I think the kids sold on Kickstarter for 110 pounds or something like that including shipping So if you need to do Musical installations much better to leave one of those rather than your laptop, I guess But guys over there. Oh, sorry. Sorry. Yeah, I'm Julio and Liam is over there and Astrid is over there We all from the center for digital music Yeah, though they use it for a kinetic sculpture where they were controlling stepper motors with it You can use it to do vector graphics on a oscilloscope all sorts of things go to the website to see a few examples of that We already mentioned this we bypassing the Linux kernel using Xenomai But still you can use USB for instance the media of a USB is available Just like with the standard With through the standard Linux API Good So not everyone may be familiar with Xenomai I wasn't before starting working in this project and I'm not an expert just now But what I found is that there are a few things that need to be You need to be concerned of which Partly apply to general real-time programming, but some of them don't So what the the Xenomai Scheduler can do is can interrupt the kernel even in non-preemptible Operations, which is something you wouldn't be able to do otherwise This allows you to get back to your audio callback every time you need it even a very small intervals And the thing here that's called mode switches. So a Xenomai thread is just a thread that's That's scheduled by the Xenomai scheduler and it should run always in it doesn't have to be should it Sorry should run, but when it starts it runs in Xenomai mode And that allows you this all this whole high priority thing But if you do something that goes through the kernel, then it will have to switch into kernel mode now each of those switches carries an overhead which is bad and also makes it more difficult and Also takes more time to reschedule To to wake up a thread that is that's in kernel mode So you want to avoid mode switches if possible, but you also want to make sure you surely want to make sure your audio thread has No mode switches So of course you need to avoid disk IO and socket IO and print f's in the audio thread But we know this already from well actually you would occasionally put print f's in your audio thread And if you just put a couple it's not gonna be a big mess in a in a regular real-time Programming environment, but in this case a print f would trigger would trigger a switch mode switch And that would be much worse than regularly. So there's alternatives. There's a function as of my functions called RT print f That does exactly the same thing P threads would call the same issue and We found out in super we're working on super collider that available that there was a conditional variable C plus plus thing When you try to wake up a thread using notify one that would also trigger mode switch So that's not something that would necessarily be bad as a general rule unless you're using Xenomite in which case you need to work around these things This is the board layout Changed bit we don't actually have the old the GPI OS here would be created just all over the board But this was better for commercial purposes, I guess Connected embedded fast and hopefully easy to get started we'll find out The The API so again, I said I mentioned there are various ways of running it of writing code for it But it's it all gets wrapped into this C Interface so you have set up function where you do guess what you have render function That's the audio callback and then you have a cleanup function at the end It's pretty standard under this regard and if you have something that's running on a different API You probably have similar functions to call So our website is down here. It's Bella.io and if you go to slash code You'll find all the documentation for the code source code and the lecture slides for this presentation Examples and sort of things Sorry, I moved around the slides and I get surprised by these things this is sort of what I mentioned earlier you get The few reasons how the latency adds up Well, we don't need to care about this just now. Oh, sure. What's happening here? Oh, yeah, nice like this one. I didn't make the animation So well, this just shows you that That's simple double buffering So we're using double buffering, of course, so you have time one buffer of time to do your computations while the Something else is reading the inputs and writing the outputs and I didn't mention this so far The big but the reason why went for the big one black actually was that there's a onboard microcontroller sort of things called the PRU programmable real-time unit that's inside the the same chip that holds the arm a8 core and This program for real-time unit is a program real-time unit that runs at 200 megahertz as a limited set of instructions and Can share memory with the main arm core and that's a great part So we using that PRU to communicate with the audio codec and with the and via SPI to them That and the ADC to get the samples in these are written into a location in memory And then we set a flag or send an interrupt to the arm core that will know that there's a new block of samples to be processed Amazing well, we generally using to leave date the interleaved format for the At the moment we use interleaved format. We may switch to non interleaved or give an option later So, yeah, we have as I mentioned we have eight ADCs running at 22 kilohertz To audio running 44.1. So you see we have every Two audio frames if you consider a frame and LR pair If it's two of your frames, you have the whole eight In inputs and outputs and I think that at this point we can give out the kids So we had 25 of them you need a machine Hopefully a Linux will just work. You just plug it the USB in and you you're online On Mac you'll need to install some drivers for USB that tethering and They've had some big issues lately. So good luck We give it we're giving out the kids and then see how the thing works I Do that in a second So in the meantime if you want to go to Sorry, if you want to go to our Good if you want to go to our websites Go to bella.io slash code Should I put the link here? Good. So if you want to go to bella.io slash code You we probably put together all the instructions you need and you'll tell us where the bugs are Sorry, are there any questions so far about the structure architecture thing I'm sorry about that we We didn't translate it. Oh well, of course the I can't tell you where you can change language. I Mean I can Look look somewhere up there. Maybe Well anyhow, it is not gonna be texting in Russian. So if you want to go to the wiki Sorry about that it's not sorry So if you go to the wiki, which will be the one two three four six seven Then you'll find what you need Okay Good. So make sure before you plug the big one in of course you ready did but make sure the sd card is pushed in otherwise It will boot from the embedded memory Good. You'll find you'll want to download the files from the From the files section I Should I think I put I put it up. Maybe I didn't update it. Maybe I didn't set it available for you Yeah Yeah, there's the better mini lag dot tar archive. I thought you would like the tar archive It's on the fat files download sections section Despite the road files here Maybe that's what it's called Yes, cool files. Okay in that tar archive you'll find a copy of the repo which is available Otherwise through the repository here through with a mercurial clone And there Few more things not much really But yeah, I thought it was handy to provide it Good so if you're done with it You don't Mac users if you go to the wiki and you go to Getting started with Bella, that's where you'll find the link to the drivers This is your link and if you're on a captain Let's hope it works Carry on Good so Good so hopefully most people will be Online at the moment. I mean I didn't say what online means. It means you can access H into the board through at the address one on two one nine two one six eight seven two and Or more simply you could just open up your web browser and type the same address here And go to pay to port three thousand One into one six eight that seven to two Column three thousand and if you hit enter there and my board is not faulty and oh I need to If config it up, I think Yeah root no password one I don't know if on my board the idea is for whatever reason not running I don't think so. I don't even know if the FT if the FT DI drivers actually needed I put it there because it's in instruction on the on the on the big old board website But and I never tried without Yeah, the horn this is There you go, and you should show be shown something similar and I think I'll let limb Oh, yeah Say or can we switch to the hand to the handle or use a handle microphone Okay, so to use the web browser based IDE you don't need to install any of this software apart from the drivers You simply need to open up a web browser and just go to the IP address on port three thousand and so the idea of this IDE is that it gives you everything that you need to program in Bella Straight right there so you can see you've got your setup function here And you've got your render function here, and you've got your cleanup function here The render function is called once every single audio block and it gives you this thing here called context And inside the context are your audio and analog buffers, which you can then access To you to get at the audio data Looks like it's not part Is anybody having is does anybody manage to get this idea up and running is anybody having problems? We got this? Okay, okay, so if you've got the idea with running if you come over to the right-hand side here There's a series of tabs and this one with a folder icon is the most useful one So in here, there's this example menu here where you can open some examples and see some code That's already running so one of the most simple one is the kind of basic analog input And this will demonstrate to you how to basically access the Context to get hold of the analog data and the audio data Another cool thing about this IDE It has inbuilt oscilloscope So if you open up for example the scope basic project from within this example thing And then click the run button over here and Then click the scope button, which is down here This will open up and it will show you so This patch is basically generating three sine waves in real time separated in phase and then the Scope API is actually plotting them in real time so that you can see exactly what's happening on the board as it's happening Which is very useful. Yes So Basically, you can program it from within the API here And so you can send any variables that you have in your program in your program including inputs from the analog or audio Or any other variables that you like there's a very simple API here where you basically just say Scope log and then you pick the symbols of the the variables that you want to plot and you have to call that once every single audio sample And then it will plot that to This scope over here You can control the how the how the scope is plotting things in here exactly like a normal oscilloscope So you can trigger it based on the different channels You can change the trigger mode and that sort of stuff You can zoom right in and have a look at or zoom right out and have a look at what's going on with the thing So you can see here these these signals are sampled at 44 kilohertz So you can actually zoom down and see the sampling rate You can see the kind of the step-based sampling that's going on there And So basically that's a really quick introduction to how to use this IDE there are of course several several other ways Methods of programming with this if you'd like to try We can have I don't know maybe maybe if Julia wants to describe some of these So we've got like a we've got a PD based workflow where you can design your program in pure data And then we use a cloud-based compiler called heavy which compiles it to optimize code Straight for Bella and we have instructions for uploading and compiling with that We also have another PD based workflow using live PD, which is totally open source this one And and we can also describe how to do that in detail a little bit later There's also a fast workflow as we said before and you don't have to use this idea If you don't want to you can also program on your own and use some scripts to upload and compile to the board And yeah, so if you come over here on the right hand side of the idea as well It gives you a few options in here so you can control the buffer size of your audio samples You can actually go right the way down to two Audio samples that means that you're you only get two audio samples of about in a buffer I know you have very low latency of one millisecond But you can also increase that in order to increase your kind of amount of programming capacity Well, I so I always use to because I like having the super low latency and I do a lot of kind of Feedback control with it and obviously like low latency is really useful in that So to give you an example of the different kind of performance That you can get at With a buffer size of two samples you can get about 500 wavetable oscillators And a buffer size of eight and above you can get more like 800 wavetable oscillators So you get quite a lot more processing power If you sacrifice a little bit of latency Yes, absolutely Yeah for the for the audio Inputs and outputs most of the buffer most of the latency is in that The analog inputs don't have those converters and so they don't have that latency and you can actually get down to Around about 100 microseconds with those guys You want to do some pd stuff? good Yeah, also need to mention that we we haven't yet optimized We are basically the samples are read from the From the converter as integers and then we converted them to float in the On the arm chip and we haven't properly optimized that yet. So this Uh So this gap in the Performance could be probably taken down a bit. So right now we probably have around 18 percent of cpu used Just for running an empty loop and that can probably be taken down to Two a bit less I guess we because we're using the vectorial floating point unit Which is not called vectorial floating point unit It's called neon on the arm chip and that can run same instructions with four instructions at the time Sorry, I didn't want to show the apple Good so Uh, yeah, these are things just mentioned and I don't think we're gonna do a real a real Sorry Yeah, sorry. Here we go. So We brought a few sensors FSRs and Potentiometers, do you have anything else? Peters your pure Peters LDRs, okay good. Yeah, do you actually have them? Oh amazing Yeah, exactly we better give them out So the audio the audio outputs, sorry, let me go back to the whole presentation mode Uh, the audio outputs are the two three pin molex The audio connectors are three pin molex connectors on the side of the board And they'll be carrying I mean each of the connectors carries a stereo signal We're giving these 3.5 mils Cables connectors female connectors, but you can actually do whatever you want with them And whereas the analogs input and outputs will be on this on these headers The marketing orange and green respectively You also have an i2c four-way Connector just there a more detailed pin out is available through the wiki So you know exactly where what goes where It's going to be here And it's going to be here at the top of the wiki. There's a link to the diagram if you need to actually connect something Good So we should have bread for breadboards as well So that'd be that make it more fun more easy to connect the sensors Do you have breppers out yet? Good before going into the proper Thing i'll just show you a few work a few workflows that you can use with Bella so you can see if you want to do some c++ things or if you want to use pure data or fast or help me with super collider So i'll just keep over all of this again the pinout is on the On the website on the on the wiki page Why did I leave this right here? Am I running the wrong i'm running the wrong presentation that explains a lot I don't know actually that that one was the wrong one good, so Okay, so let's have a look at the c++ api as you've seen in the in the ide looks like this You just call these three functions set up and render and and clean up More stuff most fun stuff happens in render where you get past this context variable the context variable holds pointers to the Analog inputs and and analog outputs and audio inputs and outputs and digital inputs and outputs And also contains sort of interesting variables like the number of channels you're running the number of Analog channels you're running and something right which is by the way fixed at 44.1 We have a couple of macros that allow you to Access basically to read and write the buffers without caring about the fact that if they're interleaved They're actually they're not macros, but they are But the functions are probably aligned by the compiler So yeah, I'll just have a look at what these context variable Gives you so they're going to be in the code docs up here and I think module not classes The thing at the top here. So again, you can find If you've got the context audio in and context audio out will be the pointers to the audio in and out buffer context analog in a context analog I will be the analog input and output buffers and digital is sort of peculiar in the sense that To do digital IO we actually Using a 32-bit word where half of that sets the direction and half of that sets the value Or reads the value so you you'd be better using the The macros to read and write digital because you don't want to go down in bitwise operations Number of audio frames all your channels sampling rates All sorts of things you could do and also counter of the samples that I've elapsed so far now before we give out any Wires and sensors and stuff. I need to tell you that about voltage tolerances So you can plug Up to five volts in an analog input. Although the converter is scaled between the the 0db full scale is 4.96 volts So you don't get much by plugging five volts. Don't plug something more than five volts in there The analog outputs are five volts And the digital inputs and outputs being they the actual bigel bones gpio's They shouldn't get anything about 3.3 volts. Otherwise, you're gonna destroy the bigger one So yeah, be very careful when you plug in there Okay, so that that was it for the c++ api We've recently worked with ollie larkin and jan and stefan let's to get uh, uh foust A bella target for foust So it's now already available through the online compiler Just yesterday, I was booking with jan speaking with jan and we found out that he upgraded the Compiler on the web server on the server and basically now the binary produced right now It's not compatible with the with the sd's we have here So if you want to compile files code, it's still possible You just need to generate your cpp file and then put it in the in the appropriate folder on On the bella Well, and basically use the build project script You'll find in the scripts folder of the file that you already downloaded And you use the script the build projects folder with this syntax here You just run build project And then you just give it a path to the file or to the folder that contains your c++ files and includes And to control to map bella analog inputs to To foust You need to use something like this. I'm sorry. I never used foust before and only I'm only run the examples of the of the playground But I tested these things and they work So I'm not going to tell you what this thing is called, but that's how it works again This is on the wiki uh, if or on the slides if not, so I'll now skip over this um We got it to work with super glider again it's not just about creating an additional target for the An additional audio server an additional Whatever it's called a driver in case of super collider But it's also about making sure that there's no mode switches in the audio thread This issue with super collider was that uh notify one function that was causing mode switch anyhow, we got it to work and The point is I don't know super collider, right? So Um, so I know I can run 120 sine waves and get 50 cpu time But I don't know how that scales to practical use So if there's anyone here that's familiar with super collider have one image when it where it runs And if you want to play around with it, I'll be happy to see what you can achieve Uh, and then there's the whole pure data thing which happens to be in another side of slides Uh, as mentioned you can use Uh, you can use pure data patches in one of the two ways One is using the heavy audio compiler, which is I'll go down here and the other is using libpd. I assume you know what libpd is Uh, so here's just quick comparison between heavy and libpd Uh, let me see if this thing goes full screen View present Yeah, great. Um, so Heavy is an online service that Takes your pd patches analyzes the graph connections and writes And gives you back optimized c code It actually does many more things in the sense indeed for us gives us Give it gives us back c code that's that all has all the includes for arm and actually compiles Very well on our platform But you can do many more things you can target Uh, javascript, uh, you can target unity so c sharp and you can also download binaries Of vsts so you can turn your pd patch not not just into vst as it would be using libpd But uh into something that's more optimized and runs much better much faster. Um, and there's some performance things down there um On the other hand this thing you could still do it with pd, right and libpd because there have been ports for Other things like to open frameworks or for javascript Or again, I've seen something recently. It's called camomile. I think for uh running vsts running pd into vsts Uh Downsides of heavy is that it requires an internet connection because the service is online. They don't share the binaries They don't share the source code And then you need to compile it and if you compile it on the board It will take maybe one minute Which may be less than what your average vst takes But you may not be willing to wait for that libpd on the other on the other hand You just you just run it you just restart the pad the the the binary once you upload update the patch and and it just starts immediately But yeah, the really advanced the great advantage of heavy is that it gives you optimized code So if you compile it with clung you get maybe 25 cpu time cpu usage on something that using libpd would take 55 and so in an embedded platform like this, that's really useful So I'll just try this See if it works straight away. It's not prepared So we have a set of scripts as mentioned So I'll just go to the folder that I gave you And scripts and then I do build pd Dash i and that's probably where I have it. It's here. Maybe Actually have one on the big bone itself Oh, it's here. It's here and looks like this Is it? Yeah, so I have these nice patch here and Good I do that and it should be enough Almost except that I didn't do something I guess All right, I need to give yeah, I always forget about it. I need to give the An address the address of the path to a folder. So I need to create a folder for it Okay, so that already came back the code already came back from the online compiler, which is very fast And now it's probably hoping to run it on the board assuming that my board is up Yeah, there it is files copy to the board and now it's running copy compiling This is the first compile. So it's compiling also the core cpp files, but Future compiles will take a bit less And this thing is plugged in here. So by the time it starts you may be able to hear something The core files take relatively little to compile By the way in the meantime while that's compiling I'll so you see and it's very easy You just run a script and that does all the heavy lifting for you before that the first time you run it You need to have created an account on the on the website and this again is Down here in the wiki Compiling pew data pages for now here this running pew data pages on bella And that's where you find all the instruction to set up this thing Basically apart from what you've just seen of running the script You need to first create a login and the first time you run the script you need to enter your credentials But apart from that you're all set. Okay, so you're compiling Alternatively if you were thinking of running pure data patches using libpd There's already libpd compiled for this version For this on this sd image is already libpd in slash user slash lives slash libpd.so And on the in the home folder there should be a binary called So here's the batch running Again, it took a while because it was the first compile will take will take less In future compiles if you want to run a libpd patches then there's libpd bella binary file in the home of the bigger one that we gave you And they're just run loads the whatever underscore main dot pd file is in the Is in that folder if I try to run it now, we will not run Because there's the other ones still running of course you can only run One of these programs at the time But if I do this then it will just run in libpd The same thing running on libpd Okay I think we can skip over most of the rest and just And just see if you want to make something useful with it Well, I'll just bring up a couple diagrams on how to connect things So We have potentiometers For sensitive resistors. I don't think we have leds and piezel And piezel mics Piezel discs actually and so yeah, I guess you know how to do these things but leave these on in case you need it You have these lights online anyhow. Good. So I guess that Oh, well, there are there any other questions? amazing so So I think we can just play around with this will be around to help you setting it up and get into work And hopefully we'll come up with something useful by the end