 Someone tell me when that comes up and I'll put my password in, it'll be nice. So this talks mostly about what makes a viable video audio platform and how Linux and OSS software achieves it. I use Linux to compose and record music at home on a studio and one of the difficulties I had when I started doing this work was working out how the sound system on Linux worked and the learning curve of the applications that came with it. I suppose perversely that's also what kept me interested in the subject and made it more rewarding for my sense. So the suitability of Linux as a platform for audio production came together with the amalgamation of the real-time kernel and development of several key applications for recording and mastering sound. So I'm going to focus on the developments in the real-time kernel today and later I'll go over some examples in the tutorial about what we can do with Linux. So the real-time kernel is a generic name for a series of patches and settings that enable applications that must replicate the response time of the real world. Excuse me for reading, this is why I had the screen shot so you didn't have to look at my read. An example with the sound is when you plug in an electric guitar into your computer and you need to hear the sound coming through the speakers at the same time as your ears hear the strings throwing. It's not the same as when you plug into an amplifier where the analog signals converted into electrical impulses which blow through the rise and circuitry to the speaker. In the case of the computer you're doing an analog to digital conversion and the sound card and then the CPU and the operating system redirecting that digital screen and re-converting the digitized concept of the sound back into the electrical impulses and sending it to the speakers. The question for me when I started was why do I need a real-time kernel to do this when I play sound on my computer and my laptop and my applications, they just work fine. So this is different from using the PC for live performances and for recording. So the traditional normal generic release of the 2.6 Linux kernel is designed for overall throughput of multi-user applications. Has an emphasis on fairness amongst the application processes sharing the resources. It's a wonderful and robust flexible arrangement, much loved but it's misaligned with the requirements of the real-time audio which I'll go over soon. I'm just going to reboot that again while I play. Real-time operating systems have been around since the late 80s. They're a requirement for embedded applications and control systems such as those used by the military, live financial trading systems, medical applications for imaging, live support and audio telecommunications and multimedia production. It wasn't until 2004-2005 that it was fully realized on Linux that there'd been attempts beforehand that they weren't completely successful. There's an emerging of the code from an embedded Linux vendor with a series of patches from Red Hat developers, to provide these capabilities to the standard Linux kernel. The real-time patches and settings are about making things more dependable and more predictable for application designers to support the main functions of audio production, mixing, recording, editing and mastering. The prerequisite is that the operating system delivers a consistent and stable platform for sound. It's best illustrated with an example. The example is to capture a sound like a note from a tuning fork and play it back with a visual representation of the wave. The sound emanating from the vibrating steel in the air takes the form of an alternating pressure deviation from the equilibrium of medium. It causes regions of compression and re-affection and these vibrate the cone of the microphone in the same frequencies. The vibrations can be mapped into lines on a graph. This is something we can work with digitally and the code that converts the analog, longitudinal or compressional waves into a digital wave tool. The sound card buffer holds this information in easily digestible packets and the kernel, the sound card driver modules and OS libraries all play in the CPU. A PCI interface connects the card to the motherboard and input and output connects for the microphone and speakers are controlled with hardware interrupts and signals. The sound card doesn't have its own digital signal processor like an old laptop then it will claim ticks from the CPU. The sound can then pass from the application that's capturing all of this information and rendering it on the screen as a nice series of amplitude values plotted over time. Lastly, the sound is reflected back to the listener through the speakers. So the sound needs to be captured, converted, rendered and reproduced in the same time, mostly, as it takes for the real sound to travel into the air for the brain to hear it. If there's a lag in the sound system then the error detected the user will experience an echo or drop out of the audio signal which would generally be bad. So the first requirement for working with audio is low latency between the system components or the ability to keep up with the real world. Generically, these dropouts and lags are called X runs and they typically occur when a system CPU is too busy to either refresh the input or output audio buffers in time to keep up with the sound card. But they could also come from latency from the hardware and interrupt handlers and the driver and the application design. It seems like quite a lot but basic PC hardware is mostly able to handle it without too many problems. As I said, there may still be a need to tune some hardware to minimise latency from certain interrupts. Historically, the traditional architecture of the Unix kernel is not really designed for this type of workload and in this case, its design can inhibit the integrity of the audio system or any real-time application for that matter. If someone wants to come up and pull the battery out of that laptop and restart it for me, that'll probably come up because I've had this problem a few times before. Mind you, this is the laptop's fault, not Linux. So as I mentioned, Linux multi-user time-sharing kernel is excellent at handling and prioritising many concurrent processes and does this by allocating time in the CPU according to a scheduler on our priority system which is tuned for fairness among the processors and overall throughput. It aims to reduce context switching. Can you hear anything? I swear I didn't touch it in case it picks up. So we talked about why Linux doesn't support real-time audio and look at the ways in which the traditional kernel does handle audio in real-time and why it needs to be patched to get over the problems that it incurs. There's five things to look at. There's the way processors preempt one another for the CPU resources. There's the scheduling of processors. There's memory allocation and paging, interrupt handlers and timing. First of all, we'll look at the scheduler. When an application's run creates a new process we'll usually fork off synchronous or concurrent casts in separate processes as required. Conceptually threads run within processors and have a similar ability to smaller scale to split off within processors and run separate tasks synchronously. The allocation of time in the CPU for all processors but not all events on the host is prioritised by the scheduler. The scheduler can only run at certain intervals and events like interrupts and critical kernel code have priority over the scheduler. So to get CPU time your process has to wait for the interrupts in the kernel code to return before the scheduler can allocate at the CPU. Sometimes the kernel needs to do a comparative lengthy task which is not broken up by scheduler runs and if your scheduler doesn't run then your task doesn't get allocated to the CPU and the sound card's in you. The buffers inside your sound card might empty out and you'll get an X run. The other problem we can have with the CPU is with locks and the preemption priority. The design of the kernel is so that code runs safely on symmetric multi-processing systems where the CPU is constantly switching between tasks. Individual competing threads and processors of the application code are allocated time in the CPU and interrupted when their time is up or a process with a high priority taps it on the shoulder. The kernel itself arbitrates this and each time its scheduler is run it checks for tasks that are ready and finds the one that has the highest priority with the processor. In order for this to work and it works extremely well in most cases critical application data or code is protected with locks. A spin lock waits or spins until the process that's holding the CPU to run a critical section is free. When the lock is released the waiting process can then grab the lock and continue to work so this is one mechanism through which the processors may hold lock on the CPU long enough to starve the audio application of resources. Similarly threads use mutexes, mutual exclusion locks and semaphores to control resourcing and program flow at a kernel level. A different but related problem is when the priority of a running process is the cause of an X-Run. A process or any number of processes with a higher priority than an audio application will be allocated to system resources so without a predictable reliable access to CPU resources and devices like the sound card the audio application will be susceptible to X-Runs. One way around this is to run your user space audio application with a higher priority than other processes but in any case priority inversion can occur when a low priority thread holds on to a resource that your high priority task requires. For example, a low priority thread requires a mutual exclusion lock on a resource your application uses. Next, a high priority process preempts the first one and starts executing CPU bound code but the first one still holds on to that mutex that we need. Now the audio application running an even higher priority thread attempts to acquire a mutex that the first was holding and the first thread has been preempted by the second thread effectively taking its CPU sources to complete processing and blocking your audio application with the locked mutex. So a high priority is not a guarantee of reliable timely application of resources. This problem isn't limited to mutexes. Any resource that can cause a task to block can be involved in a priority inversion situation for example communication packets, signals, and or events, file data and memory. To improve performance and reduce memory consumption traditional Linux loads the corresponding pages into RAM only when they're needed. When a part of the address space is accessed like a particular part of the code it's not in RAM yet and a page fault is raised. This triggers the loading of the page in RAM and usually involves a reading from a disk and possible unexpected latencies. In the default behaviour of standard kernels interrupts, exceptions and system calls are never preempted. This means that they have a high priority in the CPU and are run before any task under the control of the schedule. They can lock and block user space audio applications accessed to system resources. With timing the traditional kernel timer is not accurate down to the millisecond which makes response time unpredictable at low levels of latency required for audio production tasks. Over time many of these developments that come from the real-time patches have gone into the mainline kernel so these settings can be accessed mostly through your desktop. That's all you require but if you need to do live recording or performance using the desktop then you need to have the real-time patches. Now I can look at the changes to the Linux system that characterize the config preempt real-time patches. Number one is in the scheduler. The first characteristic is a deterministic predictable scheduler. The schedule runs more often and events such as kernel codes and IQs that used to be prioritized over the scheduler runs were brought under the scheduler's control. Under the config preempt real-time kernel there's also finer control over the scheduler classes that can be chosen for your applications and processes. Critical kernel code that used to be protected by locks are made preemptible by replacing them with real-time mutexes with priority inheritance. This means that priority inheritance is implemented for spin locks and semaphores. The process that locks a particular resource is temporarily raised to a highest priority of all tasks waiting for that lock and a process which attempts to acquire a contended spin lock will no longer spin but instead goes to sleep and waits for the lock to become free. We might finish up a bit early. We can use that laptop in the next presentation for the example which would be much more interesting. Alright, so I'll move up a layer now and talk about the sound systems. Basically about OSS, ELSA, Pulse Audio, Jack G stream and most of which you probably know about. But during the late 80s and 90s the sound on personal computers was revolutionized by the Creative Sound Blaster PCI card. The units extended from multimedia and gaming applications were set from that level from then on. The original driver for the hardware was to form the open sound system. OSS was the default on many other unixes and supported many common sound devices at the time. Its driver was designed around typical best practice unix calls so open, closed, read, write, eye, control, select, and map. But by the late 90s limitations of this design for new audio features and hardware was becoming a problem. ELSA was created to provide state-of-the-art book support for audio and the Libay Sound library and drivers represent a set of functions to configure hardware and software parameters and supported a variety of application design models. ELSA offered us support for USB and Bluetooth audio devices, support for AC97 cards and HD audio dial-up soft modems such as the SR-3055 which is fairly common at the time and I don't really see them anymore. Interestingly enough, this has the AC97 driver still on it because it's a dirty old laptop. Okay, so OSS version 3 was the original sound system for Linux and it got dropped in 2002 when version 4 became proprietary software. Version 4 was then released to get in 2007 when forefront technologies released its source code and provided it under the GPO license. OSS is still widely used and does have some advantages. The library, it's cross-platform and it's well documented. It's fast to start and it's got high quality sound but because it uses those low level higher controls for direct kernel interfacing it makes it difficult to use with sound servers like Pulse Audio and user space drivers such as Bluetooth and FireWire. In the early 2000s, Jack has implemented as a spin-off from ARDA's audio engine. ARDA is one of the applications I'll look at later on as a digitally or digital audio workstation. And Jack is what really kind of brings most of the audio work on Linux together so we'll go over that a fair bit. Alright, so in mid-2000s work begins on Pulse Audio. KDE stops using ARTS as a sound server, Gstreamer emerges and desktop providers start providing their own libraries to go with their desktops such as Phonon and Lib Sydney and it just makes a complicated landscape for audio production because you've got a whole bunch of different libraries a whole bunch of different servers there is main ones that you can choose but there's also a lot of options out there that make things a little bit more difficult for you. Well, I've got five minutes to go I'm going to get this up because it took me some time to do it Thank you OpenOffice. So this is some example of some audio systems that are available under Linux that's a audio meter a VU meter have a look at them all you've got So we got up to talking about Pulse Audio being a sound server for POSIX and Win32 systems it's got better network support in terms of streaming and it consists of a library and a server demand which most of the sound servers are can be used by eSound, ALSA, OSS and Gstreamer applications and latency measurement tools and it's the fault sound server on Fedora since version 8 Nibbun 2 since 804 and it's often chosen as a sound server for embedded systems Gstreamer is a cross-platform framework for building multimedia applications we saw a lot about this morning it's got a very small core library and as they said this morning it's used in things like Rhythmbox, Totem, Caffeine and Amarok So that's pretty well it for me There's a lot I haven't covered I didn't talk about hardware options for audio input like Firewire and USB and MIDI I didn't talk about output with speakers and 3D and surround sound We skipped entirely over mixing and MIDI's influence on the development of hardware and sampling rates and codecs we also didn't get into and we didn't look at sound creation on any other PC type device like Windows or Mac or stuff like that but what I really wanted to try and focus on was the benefits of the real-time platform for the low latency work and the reasons why it was required To apply the real-time patches to the Linux system it's not a particularly difficult job and it's well documented but it's possibly daunting if you haven't patched your kernel before or you're using your system for another main use and you don't want to put real-time patches into it so lots of the vendors give out live CDs where you can test the platform just using the DVD drive which works pretty well and there's a whole bunch of ready-built platforms out there like Studio64, Ubuntu Studio, Planet, CRMA, ProAudio, JackLab Music which is what I've got here and allow you to test the real-time features of the software without doing a rebuild of your existing system so I would recommend having a look at some of them. I've found some of them to be more stable than others some of them work a lot better on a low-spec laptop like mine but I suppose what really makes these distributions outstanding is the wealth of software which runs on top of it which you would have seen all the way through my talk if I'd got this running but we'll look at some of them later on and that's about it for me Thanks. Sorry for the technical difficulties.