 Thanks a lot for joining us today. Thanks a lot for joining us today for this webinar about the sound subsystem of the kernel. My name is Ivan. I work as a software engineer for a company called Code Think. And like many of you, I am a kernel enthusiast. I started 10 months ago as a mentee in Linux Kernel mentorship program. Now working with the kernel is a part of my job responsibilities. But in my free time, I'm still trying to contribute as much as possible through the various subsystems of the kernel. And during the past half a year, I've been actively contributing to the sound subsystem of the kernel. I maintain one of the drivers there and one of the self-tests for the ALSA subsystem as well. Why does this talk actually exist? First of all, honestly, the documentation for the sound subsystem is far from being perfect. Some parts of it are not documented at all. And this talk is aimed at revealing some of its dark corners, explaining the obligatory and necessary terms which are required to understand the sound code in the kernel. The another reason for this talk is that for some reason this area is not really popular among contributors. So I really would like to bring more talents into it to inspire you to start contributing to the sound subsystem, developing your own drivers, improving things in the sound subsystem of the kernel. Our approximate plan for today's talk is to take a look at the basic structure of the ALSA subsystem, talk a little bit about the core obstructions inside ALSA for core structures, talk about different components we can register on our sound card, how we can control them, how we can play with them, how to initialize them, and so on. We'll discuss a little bit the errors which you can face while developing the sound applications and sound drivers, and the debugging approaches which you could use to address such issues. The ALSA, the Advanced Link Sound Architecture, consists of the user space part, the ALSA user space library, and the kernel space part, which includes the ALSA middle layer and device drivers for the actual sound hardware. The user space library communicates with the ALSA middle layer, which resides in the kernel space with IOCT-L system call. And the ALSA middle layer communicates with device drivers using callbacks. This talk will cover a little bit of the middle layer of the ALSA and the part of device drivers as well. Now we will take a look at the core structure of the sound subsystem, the structure which is called S&D card. This structure represents the abstraction for the sound card hardware. Usually, the drivers for sound cards are based on one of the underlying interfaces. For instance, if you have a PCI sound card, it is based on the PCI bus. The device resides on the PCI bus. And we will register our structure. We will create all the components in the probe method of the underlying PCI device. Also, we can register our own components of the sound card because obviously, the sound card hardware contain various components. For instance, controls, because the user obviously would like to control some hardware parameters of the sound card. PCM devices, we will talk about them later. Timers, MIDI bus, and so on. Additionally, we can define our own components because obviously, not all of the hardware components of the sound cards could actually be standardized. How the sound cards are being initialized in the source. As I already said, in the probe method of the underlying interface, for instance, in the probe method of the PCI device, you can find the following code, which calls the S&D card new function, or S&D DevM card new function, if you prefer managed resource allocation, which is pretty convenient because it will free the card for you at the end of, in the existing function of your model. We run the S&D card new pass the parent device, for instance, the PCI device to it, pass various components after that, pass various arguments, after that, we set the name for our driver, the short name for our sound card, the long name for our sound card, which usually includes even the IRQ number, which it resides on. After we set all of the names for the card, we initialize all the components of the card. And only after that, we call S&D card register, which creates the procFS entry in prog slash a sound for our sound card. How to use sound cards in your system? There are a few ways how to view available sound cards in your system. And the most convenient one is to just get the procFS entry, procSoundCards. Let's take a look at it. So if we get this procFS entry, we will see the sound cards attached to our system in the format. Here we have the ID of the sound card, the numerical ID. After that, we have the string ID of the sound card, the name of the driver, which we set, the short name of the card, and the long name of the card. As you can see, here it is set in the similar format to what I described. It has the IRQ number as well. Let's talk a little bit about various sound card components which we can register on this core sound card structure. And we will start with the PALSCODE modulation device. PALSCODE modulation device is one of the most important components of the sound card. And it is represented on the actual hardware via a few hardware components, which are responsible for translating the analog signal into the digital signal and vice versa. So to store sound on our computer, to store sound, for instance, on our drive or in memory, we should translate the analog signal, the sound waveform into the digital form. And this is the responsibility of the PCM device. But how we can actually translate the analog waveform into the digital format? For this purpose, we have a process which is called sampling. The idea of sampling is pretty simple. We just measure the sound with some frequency and simply save the measurements every time. So we actually save the points on the waveform with some frequency, with some intervals. One single measurement of the sound at the moment of time is called sample. Here you can see them on the picture. On the left, we have the analog sound waveform. And on the right, we have the corresponding digital representation, which we received after sampling the analog waveform. The frequency we are making measurements of sound with is called rate. And it is measured in hertz. For instance, if we have rate equal to 8,000 hertz or 8 kilohertz, that means that we measure the sound 8,000 times per second, which is pretty common for your phones. So when you're talking with your friend using your phone, the phone records your voice with rate equal to 8,000 hertz. Here you can notice that the resolution of sound waveform representation, which we get after the sampling, depends on the rate. And the higher rate we have, the higher resolution of the representation we get. Obviously, if you have, for instance, stereo device, for instance, these headphones, you would like to keep track of two sound samples at one moment of time for the left ear and for the right ear. And for such purpose, ALS introduces the concept which is called frame. Frame is a simply group of samples for each channel at one moment of time. So for instance, if we have stereo, the frame includes samples for two channels. Is it clear enough? Please send plus to the chat if you understood what I was talking about, so I can see the reaction in a second. Come on, where is my chat? Chat. Plus, plus, plus, more pluses. Great, we can go ahead. But where does the sound hardware store frames? And usually, the sound hardware stores frames in the hardware buffer. Usually, the hardware buffer is allocated using the DMA. So the hardware could access it directly and write samples directly into the memory. During this talk, we will talk about two different kinds of buffers, the hardware buffer and the software buffer. The hardware buffer, as I already described, contains the frames processed by hardware and the software buffer is allocated in the user space memory by the application which would like to read some frames or write some frames to the hardware. So the main goal and main responsibility of the ALSA MIDI layer here is to translate recorded samples from our hardware buffer into the software one. Obviously, if the ALSA MIDI layer had to copy the data from one buffer to another, the old buffer at one time, it would be insane this low because copying a large, huge buffer is a pretty complicated task. It takes a lot of time. To reduce the time required to copy data from hardware buffer to the software buffer, the ALSA, it introduces the concept of periods. So the buffer is split into a group of periods so the period is a small part of the buffer. Each period contains multiple frames. Each time the hardware prepares a new period of data, it calls the ALSA MIDI layer to process this new chunk of data, the new period. Another important thing about the samples and another important parameter of the samples is the samples format because obviously the quality of the sound which we store in the digital format depends not only on the rate but on the form how we store the samples as well. For instance, if we have four-bit format as on the picture on the slide, we would have only 16 different variations of how we can set the value of sound at one particular moment of time, which would not be pretty precise. Usually we use larger formats, for instance, 8-bit wide or 16-bit wide or 32-bit wide. To refer to one particular format in the kernel, we have a type which is called SNDPCM format type. And usually we refer to one particular type using one of the predefined constants. They are predefined in the ALSA layer in the kernel. For instance, SNDPCM format S16LE means that each our sample consists of 16 bits. It's a signed integer and the engine as always signed integer is little engine. So the format also matters. The order of hardware stores the samples in the hardware buffer in a particular order. And we can distinguish two main orders of samples in the buffer. And this is the interlift access mode and non-interlift access mode. If the hardware puts the samples into the hardware buffer in interlift mode, it puts the sample for channel one and after that puts the sample for channel two. After this, again, sample for channel one and sample for channel two if we have stereo. So the samples are interliving. If the hardware puts samples in non-interlift access mode, the hardware dedicates the part of the buffer for storing the samples for channel one and another part of the buffer for store sample for channel two. So the whole buffer is split into the groups which are lying continuously in the memory. So one part for channel one, one for channel two, one for channel three, so on. Also we can define our own complex access mode, but in such, because hardware could work in some completely different way from the ways I described. It could put the samples into the buffer in some different order or it couldn't have the hardware buffer at all, for instance, so the hardware defines some another method of communicating with it. And in such case, we can declare our access mode as complex. And usually in the kernel, you will see different variations of the interleaving and non-interleaved access modes and M-Mapped and not M-Mapped access mode, because we can divide the access mode into the access modes into the M-Mapped and not M-Mapped ones. The M-Mapped access mode means that we can actually take the part of the hardware buffer and M-Mapped into the user space. So the user space application through the outside user space library could write to the buffer directly. The non-M-Mapped access mode means that we cannot do this for some particular reason. In such case, we would also need to define our own functions to copy data to the hardware because it could work in some other non-predefined, non-generic way. And as I already mentioned, you will face different variations of interleaved, non-interleaved and M-Mapped, not M-Mapped in the kernel. Another important concept of which we need to discuss is PCM substreams. Some sound hardware has a functionality of mixing multiple sound streams together. For instance, in the older hardware, when we didn't have such sound servers as pulse audio, which could mix multiple streams together and send it into the one stream of audio to the hardware, the hardware had multiple sub-streams which are mixed together on hardware. And to provide an abstraction for such hardware, the ALS introduces the concept of sub-streams. So every sub-stream has a direction because it could be used for playback or for capture, but not for both of them. And the single PCM device can have multiple sub-streams. So we can stream sound to them simultaneously. Every time you open the PCM device for playback or for capture, the PCM middle layer checks if we still have the free sub-streams on this PCM device with the desired direction, for instance, with the playback direction. And after that, if we don't have such sub-stream, the ALS middle layer simply returns an error. And if we have the one, the ALS middle layer allows us to do some operations with this PCM device. We will talk about it in one second after we take a look at how you can view the PCM devices in your system. Yvonne, there are a couple of questions in the chat. I think it might help them. If I'm to help them follow the personal presentation better. So would you like me to read them out or can you? Yeah, you can see. Which buffer period decides what will be the latency? What do you mean by latency? I mean, yeah, what do you mean by latency? Oh, you mean like the overall latency of the sound system. We can think logically about this and if we have a larger periods of frames, the copy operation will take longer. But if we have the small buffer and small amount of periods, the copying operation will be faster but we can face some errors which I will describe a little bit later because the hardware could accidentally overwrite the buffer before we read the data which we haven't read yet. The period size or the buffer size, what describes the latency of the sound system, the buffer size of the period size, the both of them. Did I answer the question? You were saying even that both contribute to the latency. Yes, for sure. Maybe there is no single field or format that affects it, both of them do. Yes, yes, yes. Actually a lot of parameters affect the latency but usually the thing which affects latency most is the communication with the hardware and usually it's not the issue of the buffer period sizes. The other question is, could you explain the sense of a sub-stream when recording? For instance, some time ago I was developing the driver for the sound card which has multiple sub-streams because it literally has multiple inputs which it could mix together. So when we record the sound, let me think. Yeah, so when we record the sound from it, it provides the sound to multiple sub-streams. So it actually has a few inputs. When we capture some audio data from it, we capture the sound for, oh, sorry, I'm a little confused. Again, the practical purpose of multiple sub-streams when we are recording the sound is not so simple as then we are talking about the playback because obviously when we do playback, the purpose of sub-streams is more clear because we can mix the sound from multiple sources, for instance, from multiple applications together, but still some hardware has multiple sub-streams for capture. For instance, it could record the sound from multiple sub-streams simultaneously and for such purpose we could have multiple sub-streams. That's all that we have. Hope that answers your question. Just follow up questions in the chat and then we'll give Yvonne to continue the presentation and there is a Q&A time that's coming up. Thank you. I hope I answered the question. So to sum up, again, we could have multiple sub-streams when we're recording sound, when the sound card records the sounds simultaneously from a few different sources, for instance, the sound hardware which I had had multiple MIDI ports, so it produced the sound from multiple sources. Usually nowadays you will face the PCM devices with only one sub-stream for capture and only one sub-stream for playback because sound mixing is usually done by the sound service as pulse audio. The PCM device could be initialized via S&D PCM new function. We pass the pointer to the sound card which we have already created. We pass the count of sub-streams for playback, the count of sub-streams for capture and the pointer for the structure to be allocated, the S&D PCM structure. To view PCMs in your system, you can use a few different options for that. For instance, you can use the same ProCFS interface, but instead of the cards, we have the PCM untrain it. Each role describes the PCM device. We have the sound card number here. We have the number of the PCM device on the sound card. We have the ID of this PCM device, the name of this PCM device and the amount of channels for playback and the amount of channels for capture. Another option which you can use is to use one of the really powerful Alasa tools, the A-Play or A-Record. The A-Play will show you the PCM devices which are available to play some sound and A-Record will show you the PCM devices which are available to capture. As you can see, some of my PCM devices are not available for capture. And every row here contains the number of the card, the information about the PCM device itself, the amount of sub-streams, sometimes they are called sub-devices. And yeah, that's the most common two ways to view PCM devices in your system. Another important concept is a structure called PCM runtime. Every time when we open the PCM device for playback or for capture, we need to keep track of the hardware pointer of the software pointer and another variables of our particular capturing or playback process. And to keep track of them, Alasa introduces the structure called S&D PCM runtime, which is allocated to a particular sub-stream each time we open it for playback or capture. Runtime structure stores a lot of different information, including the buffer pointers, the configuration, the spin locks used by Alasa middle layer internally and so on. Now let's take a look at the whole picture. At the path, the sound goes from the hardware to the user space buffer or in the opposite direction when we are playing sound. Let's imagine that user space application would like to play some samples, play some sound on the hardware. And through IOCTL, the user space process falls into the kernel space function called S&D PCM LibExphere, which does pretty simple thing. It takes a look at the, if we have any frames prepared by hardware, recorded by hardware, and which we can find in the hardware buffer, if we don't have such frames, it's simply the user space process simply sleeps until we have such frames. And when we have such frames, the S&D PCM LibExphere function copies the frames from the hardware buffer into the user space buffer. The hardware on the other way, every time then the hardware prepares, on the other hand, every time the hardware prepares and another chunk of data in another period, it fires the interrupt and the interrupt handler in the hour driver calls the Alasa middle layer function, which tells the Alasa layer, hey, I have a new chunk of data prepared, you can use it. Here I also should mention that the DMA buffer, the hardware buffer is assumed to be circular. That means that then the hardware achieves the end of the buffer, it starts overwriting it from the beginning. Here I have a more detailed description of the reading process. I will not stop here because we don't have enough time for it, but if you're curious, you can take a look at the presentation after the talk and take a look at the actual functions, look into the code and figure out how it works in details. How the Alasa keeps track on the hardware position in the buffer, so the position the hardware writes to or reads from and the position in the software buffer where we should take our frames for playback or put our frames during the capture. For this purpose, Alasa middle layer introduces two variables called hardware pointer and application pointer, but don't be confused, there are not pointers. They mean the count of frames processed by the hardware and the count of frames processed by the application. So every time then the hardware processes a new chunk of data, it updates the hardware pointer, tells the Alasa middle layer that the hardware pointer is updated and the Alasa middle layer increases the hardware pointer correspondingly and the same with the user space application every time when we, for instance, read more frames, the application pointer is being updated. The difference between these two values is called available frames. During the capture, it means the count of frames which we can copy from the hardware device right now which are prepared, but which have not been read yet. And during the playback process, the available frames means the amount of frames which we can write to the hardware device which are prepared by the application. Here we should discuss the most common error or the group of errors which you will face during developing the sound drivers and sound card and Alasa applications as well. This group of errors is called X-RUN. The X-RUN group of errors contains two different kinds of errors. The first is overrun and the second is underrun. The overrun could happen only during the capture process and it happens then the application doesn't read from the hardware buffer frequently enough and the hardware buffer gets overwritten. The hardware buffer is circular. That means if we don't read from this buffer frequently enough, the hardware pointer could achieve the end of this buffer, start rewriting it from the beginning and rewrite the part of the information which we haven't read yet. And in such case, we will have a data loss. This data loss is called overrun. The underrun on the other hand could happen only during the playback process and it happens then the PCM starts starving for new data. Then application doesn't write data frequently enough and how we can actually keep track of such errors in the Alasa middle layer. For such purpose, we can use the two variables which we have already discussed. That's the hardware pointer and application pointer variables. If the difference, for instance, during the capture process, if the available frames, the difference between the hardware pointer and application pointer is larger than the buffer size, that means that the hardware pointer made the whole circle and overwritten the part of data. That means overrun. During the playback process, if the difference between the hardware pointer and application pointer is lower when some stop there's hold, that means that we will start starving for new data soon and that will mean underrun. That's not the only way how Alasa middle layer detects such issues, but it's one of the ways how we can keep track of them. I don't have a break for Q&A here, but probably we should make the one because things will get complicated soon. Any questions? No questions at all? Is it clear what I just explained? Please send pluses to the chat if it was clear and you understand what's going on. There is one question looks like now. Does Alasa support only PCM streams? PCM device is just an abstraction for the hardware on the board which translates an analog signal into the digital signal. So that's the main abstraction we record the sound from. So actually I'm not sure here, but I believe that's the only kind of stream available for recording and playback some sound in the Alasa layer at all. So yeah, Alasa for sure supports MIDI. I don't know about others. Yeah, Alasa for sure supports MIDI. I've never heard about TDM by the way. Yeah, sorry, I worked with it. The scope of this talk is mostly about the PCM devices and their abstraction in the kernel and the Alasa system is obviously pretty huge. So there are other things there which is not in scope of this talk. Yeah, sorry if I couldn't answer your question. Yeah, hopefully you will still get something useful from this talk. So let's, I left the most exciting information to the end. So yeah, please, yeah, let's go ahead. How the Alasa MIDI layer communicates with the PCM device? It was mentioned on the one of the previous slides. So let's check your attention. Could you write to the chat your suggestions? Any suggestions? Come on, no suggestions at all? Okay, Alasa MIDI layer uses the callbacks to communicate with the PCM devices. To define callbacks for a PCM device, we have this scary structure called SNDPCM Ops. We can set the callbacks for our PCM device with the SNDPCM set Ops function. Obviously we don't have to define all of them. There are some generic implementations of some callbacks which could fit the majority of hardware. But some of them should be defined anyway. So let me explain the mechanics of the callbacks a little bit. Then the LC user space leaf, for instance, opens the sub stream for capturing or playback. It runs the IOCTL to the Alasa MIDI layer and the Alasa MIDI layer calls the open callback which we defined for our PCM device which was opened. Similarly with the hardware parameters callback when the LC user space leaf library calls sets some hardware parameters. The Alasa MIDI layer passes the hardware parameters to our driver so we could set it to the hardware via callback. Let's take a look at the callbacks in more details. The first callback is an open callback and the open callback is obviously being called when the sub stream is open for playback or capture. So we receive the sub stream, the pointer to the sub stream here which was opened. And in the open callback we usually set the hardware description. A hardware description is a structure which contains accepted rates, counts of channels, period counts and sizes, possible buffer sizes for our hardware and it is set here. So yeah, also we can allocate our private structures here. For instance, if we want to keep track of the current opened sub stream internally in our driver we will allocate some private structure here. Also we can set the hardware constraints in the open callback. The constraints is other important feature of the Alasa. I will not talk about it here but it's pretty well documented and it's basically allows you to say if you set this frame rate, my hardware could record this amount of channels for instance. If you set another frame rate, my hardware will be able to record another count of channels and so on. So it allows you to set your custom restrictions for the hardware, your custom constraints. The callback on the other hand is called then the sub stream is closed and in the closed callback we usually free our private data which was allocated in the open callback. It has the following prototype and both the open and closed callbacks are not non-atomic so we could sleep in them safely. Another important callbacks are hardware params and hardware free. The hardware params is called then the application sets the hardware settings for instance, buffer size, period size the desired format and the desired rate and so on. This callback is not non-atomic and it could be called multiple times because the software could set hardware parameters multiple times and usually in this callbacks we do some hardware setup and sometimes we do unmanaged buffer allocation here if we decided to use unmanaged buffer location. There are generally two different kinds of memory allocation in the LSD system the unmanaged and managed. The unmanaged memory allocation means that we pre-allocate the pages for our hardware in the initialization of our sound card method and after that every time the hardware params is called for one capturing or playback process for every process with the PCM we have we allocate some pages for our hardware for instance using the DMA so we can define the custom behavior of buffer allocation. If we choose managed buffer allocation we can call the S&T PCM set managed buffer all the function once and it will allocate the pages for us which is pretty convenient and when you're dealing with the majority of sound hardware it could be enough for you. So two kinds of allocation. In hardware free which is called just before close we usually free the resources which was allocated for the hardware. This callback is also could be called multiple times. The prepare callback is called every time the S&T PCM prepare function is called in the kernel or in the also user space library and usually the S&T PCM prepare function is called when we face X runs. So this method is aimed at recovering from X runs. Here we also can set some hardware params as rate format and this callback is not atomic we can sleep in it and it also could be called multiple times. The difference between this callback and the hardware params callback is that prepare is called any time we have an X run and we call the S&T PCM prepare. That's the main difference between them. The trigger callback is called every time that some event is happening with our PCM device. For instance, then the PCM device is started, stopped, paused, suspended, resumed and so on. This callback is atomic and yeah, it's also pretty useful. The pointer callback is one of the most important callbacks and it is called then the alchemy delayer wants to know the hardware pointer from our hardware which points to the other buffer. So this callback returns the hardware pointer in frames. Don't confuse this hardware pointer with the HV pointer variable which we discussed. This hardware pointer is related to the actual hardware. Usually we read the hardware register here and return the pointer value in frames. The IOC callback is not so important. Usually it is predefined. I just decided to describe it because the behavior of it is not really obvious. This callback allows us to redefine some of the PCM Ioctals. You could not define the new Ioctals using this callback. Actually you can redefine only three of them which are presented on the slide. And usually the alchemy delayer provides the generic implementation for all the Ioctals. So you do not have to implement it yourself but it still could be useful sometimes. That's all about the callbacks. Obviously there are many other callbacks there but they are also pretty good documented and they have good documentation. You will find all the links in the useful resources at the end and you can, if you're curious, take a look at them. How the PCM device could be tested? For testing the PCM device, you can use one of the very powerful tools which we already used a little bit for listing the PCM devices in your system. These tools are called iRecord and APlay. They are extremely powerful tools. You can literally set all the hardware parameters with them. You can record the sound in interleaved or non-interleaved access mode as I described. You can set different formats, different frame rates, different amount of channels and so on. So for instance, I have a virtual machine here and let me run it. On this virtual machine, I have three sound cards and the two of them are related to the loopback model which is pretty useful and it represents the software loopback. Basically, all the sound you send to the one PCM device appears on the other card abstraction. So you can send the sound here and you will get the output from the sub-stream on the other device. It's pretty cool. Let me record some sound from it. So we type a record, we specify the device. You can use it like that, the number of the card, the number of the device, for instance that one. After that, we need to specify the count of channels, for instance four. We specify the rate, 48 kilohertz. We specify the format, for instance, science 16-bit integer in the little endian. We specify the duration, for instance, five seconds and we specify the file name. Oh, all right, what's going on? Oh, all right, oh, my indentation is broken. Sorry, again, a record, minus D dev, once I have a script for it, like that. So basically you type a record, minus D hardware, you specify the card you want to record some sound from, you specify the device, you specify the amount of channels as I already said, you specify the format and the frame rate and the duration, five and the output file. After that, it records some sound. Let's take a look, yeah, five seconds. Also it has pretty convenient features, for instance, you can specify the high parameter and hit pause. So that means interactive. You press space and you hit pause here and you can release really powerful tools. Any questions at this point? There is one in the chat, Yvonne. Yes, the first of all, look back, the third one is my PCM test driver, that's true. Something like this, come on. Yeah, the second one is my PCM test driver. I will use it for the third demo. All right, any other questions? Nope, all right, let's go ahead. Now we'll discuss another component which you can define on your sound card. It's a component called controls. Obviously the user might like to control some hardware parameters, some parameters on your hardware device and for such purpose, he can use controls. There are three main types of controls which we can define. It could be the integer controls, for instance, the amount of distortion, the volume which could be set as an integer value with the minimum possible value, the maximum possible value and the step which it could be increased with. You can define a switch, the Boolean type of control, for instance, enable or disable the hardware loopback. Or you can define the enumeration type of control, which basically allows the user to choose from a few predefined options. Here I should mention that all of the controls should follow the naming convention. It's not obligatory, but you should. The format is described here, but also in the documentation. And yeah, how the controls are defined. To define the controls, we create the SNDK control and use struct. We pass the interface this control is related to. For instance, in this example, we say that this control is related to the sound mixer. We have a name for the control here. We have an index for the control because there could be multiple controls with the same name. We set the access rules for this control. For instance, in this example, we can read and write from this control. Also the controls could be volatile. That means that the hardware could change it without any notification. And in such case, we have to pull the, actually pull the new value every time we would like to read the value from the control. So it couldn't be cached. Also, we define a free callbacks for our control here. After that, we call a few functions and add our control to the cart with the SNDK control add function. Let's take a look at the callbacks because that's pretty important thing about controls. Did you notice that we don't define the type of control here? That's because we will define it in the info callback. The info callback is called every time the user application likes to know the information about our control. For instance, if we have enumerated type of control, the user space application might like to get information about one particular option. For instance, the option number two. And in such case, we should return the option number two to the input it into the SND value structure. In this particular example, we put it into the name field of value dot enumerated structure. We will take a look about on the structure on the one of the next slides. As you can see, I highlighted that we perform some validation here as well. To avoid possible out of bounds accesses and such things. On the other example, you can see the control with the Boolean type, which is actually the control with the integer type, but with only two possible options to choose. The minimum value is set to zero, the maximum set to one. So we can actually set zero or one. That's a Boolean type of control. Another important callback is a get callback. It is being called then the user space application would like to get the value of the control right now. So in this callback, we usually literally read the value from the chip from the hardware device and put it into the LM value structure, which you can see on the right. The structure looks scary. It contains a union for different types of controls, but yeah, that's what we are working with. Here I provide two examples for the integer type of control and for enumerated type of control, which looks pretty similar because they are pretty similar. The put callback is called every time the user space application would like to put a new value into the control and we should validate it before sending it to our hardware because we would like to avoid all possible issues here. To view and manipulate controls, you can use one of the utilities. The first one is called a mixer, which has a console interface. The other one has a pseudo graphical interface and it's called a mixer. I guess I will skip this demo because we literally are running out of time right now. So let's discuss the another component, which I left to the screen. So there is one question about callbacks in the chat, if you would like to answer that before you switch topics. How callbacks gets registered to sound framework also and when. The callbacks for the input element are registered and basically we keep the pointers to the callbacks in the SNTK control new structure. Before we register our sound card, before it appears to user, before it could be used by some user space application, in the initialization method of the card, which is called from the probe method of the underlying interface, we should register all of the elements of the card and add all of the controls, all of the timers, all of the PCM devices before we register our card in the system. So callbacks are registered in the SNTK control new structure and the corresponding control is registered in the init function of the sound card. How does your user space know that now you have callbacks present and ready to support requests? You're ready to support requests then the sound card is registered in the system. Did you answer this question? I think they'll come back if they have a question. Oh, yep. Yes, thanks. Okay. Great. Yeah, and anyway, the best thing you can do after the webinar is to go and check it yourself because I'm a little bit nervous and I could be mistaken, so please take a look at it yourself because if you're curious, of course, yeah, it's obviously better to check yourself as well. I'm just showing you the direction you can look into. But usually, yeah, you define all of the controls in the init function of the card. All right, we discussed put callbacks. Ivan, oh, yeah, sorry. Thank you. What's going on? That chat is visible. So if you can minimize the chat when you get together. Oh, sorry, sorry for that. Oh, that's okay. I thought it's visible only for me. That's okay. All right, we'll skip the demo with controls and go to the most interesting part. As for me, that's the timers. Timers are not documented anywhere at all. I am 99% sure that you could not find any articles in the web describing the timers in ALSA. So that's pretty exclusive information here. The ALSA middle layer allows you to export the hardware timers. So the other kernel models and the user space applications could bind their custom events to them. For instance, the kernel module could bind to the timer, set the desired amount of ticks and after this amount of ticks, the timer will fire and do some actions and you will actually use the timer of the sound card. Let's take a look at how we can view timers in your system. For this purpose, we can use our favorite PROC ASound, PROC AFill sentry, which is really useful. You can take a look at it in more details because you will find a tremendously huge amount of useful information there. And we can cut the timer sentry. And we see a pretty interesting thing here. We can see that ALSA middle layer automatically allocates the timer for each substream registered in the system. That means that we actually, when we play some sound on them or when we capture some sound from them, every time they prepare a new chunk of data, the timer related to this particular substream fires. So you can actually bind the events in kernel modules to the events of substreams, to every time the substream processes, every time the hardware processes a new chunk of data for a substream, it fires your timer, which is amazing in my opinion. And for instance, such driver as software loopback, SND loop could bind to the drivers, to the timers of our sound card and keeps track of time using the timer on other sound cards, which is pretty useful and funny as well. I wrote a demo kernel module, which will help me demonstrate the timers in the ALSA subsystem. They're generally, oh, okay, I will show it after a little bit of explanation. To create, to export the hardware timer to make it visible, we create the SND timer hardware structure, which gets some flags, the resolution, that means the resolution in nanoseconds, it means that your timer ticks with the interval of some particular count of nanoseconds. For instance, if it's 100 nanoseconds, your timer will trigger every 100 nanoseconds. After that, we could set the amount of ticks, which could be set by the kernel model or the application, which would like to bind their event to this timer. So for instance, if the maximum amount of ticks is 100, that means that we can schedule our event for 100 ticks in advance. So it will tick every 100 ticks. We define a few callbacks, which actually are triggered when the user space application or the kernel model wants to start the timer. The desired amount of ticks is set in S6 field of the SND timer structure. And we can actually tell our hardware timer that we would like to fire an interrupt after some amount of ticks. Yeah, how we can bind to timer? To bind to timer, we can use the structure which is called SND timer instance. So we create the structure in our kernel module. We pass the desired card index, card ID. The timer is based on, we pass the ID of the timer. And after that, we can literally bind to this hardware timer, which is amazing. I will show it to you right now. There is a module in the kernel, which is called HR timer. And what it does, it actually creates an exports the timer, which is virtual and based on the high resolution timers which are not hardware dependent. So that's the virtual driver for the virtual driver. And if I, this module is loaded in my system by default. So when I cut the Procosound timer's entry, you can see it is available here. It is called G3 because it's a global timer. There are different kinds of timers in also. I'm not sure about the difference between them, but you can specify the interface your driver is related to. For instance, these timers are related to PCM devices. They start with the PCM, with the PCHAR. So the high resolution timer is a virtual timer in the system. We could find two. It has the resolution of one nanosecond. And we can schedule our callback for any amount of ticks up to one billion. So that's actually what my small kernel model does. It creates the SMD timer instance structure. It sets the parameters which are related to this global high resolution timer. They create the new instance of the timer with this name. They set some flex to it. And it's at the callbacks. The first callback is called callback and it fires every time the timer fires. And the second one, a second callback is called and it fires every time the event is happening with our timer. For instance, then the timer is started or stopped. After that, we open the timer with SMD timer open. We pass the slave ID, which basically identifies this particular timer instance. And that's it. So that's basically what this demo model does is creates the demo FS entry. You can use to like schedule the timer for desired amount of ticks. I will show it to you right now. So here we have my model. I will insert it. All right, here we go. And we have the debug entry in the bind timer folder. We have the timer entry here and we can set the desired amount of ticks we would like to trigger our event with. For instance, one billion, which is a maximum value. Okay, six, nine. And here, oh, sorry, not cat, but echo. And every second it fires a timer and preets the amount of ticks we set for this timer. Which is pretty funny. I provided the sources for this model in one of the slides, you will find the link to it. And you can play with it in your free time. Yeah, it will be a good exercise because as I already said, it's not documented anywhere. All right, we finished with timers. Now let's talk a little bit. Oh, we are running out of time. Can we make a few demos? Sure, do we have time for it? A few more demos? We have about 20 minutes. All right, let me make the small demo in the first five or 10 minutes and after that we will have Q&A session. I don't think we will have a lot of questions here. Can we do so? Go ahead, yeah. Yeah, great. How to debug sound-related issues? How to debug x-frans? There are a couple of ways how you can debug the sound-related issues. The first one is tracing. And that tracing is generally a good option for debugging any kinds of issues within the kernel. For such purpose you can use ftrace if you have an issue on the kernel side. Or strace if you have an issue in the also-userspace library. If you want to track some code paths, you can use strace. You can use this also-specific debugging here Prokofas entry which is called x-run-debug. I will show it to you. Or you can use the king function of all debugging methods is printk because, well, you can use printk for everything. How to do tracing? Well, the process of doing tracing for sound subsystem is not different from any other subsystems. So you, for instance, could use the function graph tracer for it, which will show you the exact time of the function execution, time of function calls. And you can find a reason why you have delays. For instance, using the tracing I figured out that in one of my drivers I had a delay in hardware pointer function. And basically the tracing helped me to mitigate such issue. All right, the another thing you can use to debug sound-related issues is x-run-debug. To enable this feature, you need to enable three kernel configuration options, PCMX1-debug, Verbal, Prokofas, and debug options. And let me show it to you. I will restart my QMO right now. To simulate the overrun error, I will use my PCM test driver because it has a feature of injecting the delays into the playback and capturing processes. How will we do so? I will connect to my virtual machine with SSH and I will put some negative delay. So my driver, my virtual driver will produce sound samples instantly fast. Oh, sorry. To do so I can, I go to the one of my kernel parameters, we see tests, parameters, inject delay. And every time I will record some data. What's the format for record in a second? So we specify the file, the count, or the time, for instance, five seconds and the count of channels. So yeah, here we go. We have overruns here because we are producing the data instantly fast and some part of the data gets written. Let's try to enable this extra debug options. To enable the extra debug options, we should write the value, this is that value to extra debug proc of S entry. The value is the sum of the values which corresponds to different debugging features. For instance, we can enable some additional logging. We can enable the stack trace every time we have an overrun. So let's take a look at it. To do so, we set an echo to the proc of S entry, proc sound to the desired cart number. It is the cart zero. Oh, no, that's card one. Card one, desired PCM device capture and extra debug. Also we need to increase the log level to see the logging messages. And now if we record some sound, it will get us a stack trace for every possible issue happening in the sound subsystem during our capture process. We can stop it here and take a look at the log. You can actually, and here we see that we have an X run on this particular substream, on this particular device of this cart, the cart one, device zero, and the capture here. We have a stack echo trace here, so we can actually send it into the decode stack trace script and get the line where the issue happened. As we can see, it happened in the hardware pointer. So the alchemy layer detected the overrun there. That's really convenient. I wish I knew it before I started developing sound card drivers. So that's a really useful feature and it has a lot of options. You can play with them in your free time as well. Another additional options to debug your sound card drivers is to use also self-test. They have a self-test which tests your sound driver using different frame rates, buffer period sizes, different hardware parameters. It can test mixer controls as well. So it actually can be very useful for detecting the timing issues in your driver. Or you can use the SND loop, the virtual software loopback driver to compare the output which you get from your hardware device. If your hardware device has some loopback features, for instance, the hardware loopback. So you can use the SND loop driver as a template driver which you know works correctly, which doesn't have any timing issues, which works fine and you can compare your drivers with it. Also, we can use the SND print K in France. We can use them anywhere and always, but to use the PCN specific logging functions, we have to increase the logging level in the message like that and enable the config SND debug option. Let me say a few words about my virtual PCN driver and I will be finished for today. That's the driver I developed for testing the AlcaMedia layer. Actually, it has a few features. For instance, generating the random or pattern-based capturing data. So we can read them in the user space and compare with the desired, with the expected pattern, the expected order of samples. So we can be sure that we don't lose any data in between the hardware and the software in the user space. Also, it can check the playback for containing the pattern so we can check another way of data. It can inject some errors and delays into the PCN callbacks so we can test how the AlcaMedia user space applications behave when they face different errors, different x-runs. And yeah, so please take a look at the sources. It could be used for writing another virtual drivers because covering the system with a test is always a good thing because when we have more tests, we have less bugs. Also, you can look at the documentation for it and here you can find the user flow resources. I used to prepare this talk and also you can find the sources for the timer driver I've shown and the record and playback scripts I had. So the QR code is here, the source, the links are also here and the best information source about the Alca is the Alca sources obviously because you don't have a lot of documentation written and you always have to read sources to check it yourself. Thank you for your attention. Do you have any questions? Oh, everybody see the question in a box in the chat. All right, any questions? Or was it so clear that, oh, all right, any questions? Do you explain the clocking situation in Alca? What do you mean by clocking situation? Usually we bind, are you interested in the mechanics how the hardware notifies the Alca means the layer that's another period lapsed or what do you mean by the clocking situation? Can you write in the chat please? Sorry, I might not be aware about the clocking situation and also usually then another period lapses, usually the hardware has internal clocks and every time, yeah, what does the hardware driver define the clock source or is it possible that the client middle layer or client application does define this? Usually the drivers have to define the, they don't define the clock sources themselves. Usually the hardware have internal clocks and every time then it basically synchronizes the periods, the new periods of data with this internal clocks every time. So they don't like export this clock sources into the kernel space. So another applications and another drivers could not bind to them if it wasn't done explicitly. I mean, if we didn't define the timer as I shown, so the timers are the only way to expose the hardware clocks into the kernel space and bind to them. So the system clock is the separate. Are you asking about the system clock separately? I guess, so that system clock drives all of the timing in general. Yes, on the middle layer side on the driver side, you usually bind to the hardware clocks on the sound card, but you do not export it to the outer space except as using the timers, but the timers are running only then the playbook or capture processes happening with the hardware. For synchronization of the semi-the-layer, we use system clocks. For synchronization of the driver, we use hardware clocks. So yeah, but you cannot access the hardware clocks from the semi-the-layer if you didn't like explicitly exported the timer. Anyway, you can reach out to me after the webinar in one of the social networks like LinkedIn or you can easily find me in the mailing list of the kernel because I'm only even a little off there. And I will be happy to discuss any questions with you after the presentation. Maybe we could look into these questions together and figure it out from the sources, from the documentation. Yes, yes, that's how it works. The hardware clock on the card fires an interrupt. The driver notifies the semi-the-layer that another period of data is prepared and the semi-the-layer copies the data from the hardware to the software buffer at this point. So you don't export the timer on the board to the user space somehow. The only, well, they are exported by default as timers as I already shown. The timers are exported for every sub stream but the problem is this timers are not running then sub stream is not running. And well, it has the resolution of the period. So you cannot set this timer to any amount of ticks more than one. So the timer will fire after every period elapses. So it's not really precise. Just like the thing which allows you to bind to the events then, then our period elapses. Yes, yes, that's true. So yeah, usually the sound card is a clock master here. So yeah, any other questions? Any other questions for Yvonne this time? Cool, all information was so clear that we don't have any questions. There are quite a few questions I think earlier, Yvonne and you were able to answer all of them. I tried to answer them. Sorry if I'm not aware of something. Oh, I think you gave enough breadcrumbs for people to go look at things and find the demos are good demos and showing all the tools that they can use looks like. There is one more question. Yes, yes, yes. Changes that guy from mentor made to SNDL loop was to make it possible to choose the SND timers as a sync source. Yes, yes, exactly. So you can actually synchronize the SNDL loop only with the timers which were exported. You can synchronize it with the timers of the substreams which I already mentioned that it will be synchronized with the timer which fires every time the new period of information is ready is processed by new period of samples is processed by the hardware. But that's the best precision you can get. So you combine only two exported SND timers which is not really convenient because I also would like to have the way how we can bind to different timers in your system for instance, network clocks and so on but we don't have such opportunity unfortunately. Maybe it would be a good thing to implement such thing. True. Thank you, Iban. Oh, another question. Oh, okay. The SND timer, then yes, yes, only like that. Unfortunately again, it would be great to have some kind of timers which you can export to the whole kernel space and generic kind of timers. You can use it so you can use them everywhere but as far as I know, we don't have the one. Sure, do you aware of such kind of thing? So the timer subsystem, are you talking about set timer, get timer type of situation? No, I mean that you can export the hardware timer on some hardware, a hardware device on the sound card for instance to the whole kernel space and you can use it anywhere. Right, I think the only thing I know about is the system clock that is exported through that. We have system clock, the semester clock that drives the system timing. And obviously the reason you probably wouldn't want to do that is a random called driving timing, random sound card driving timing for the entire system. So there are some issues with security issues that are timing off issues. So you have to, system clock is maintained and then your system clock can be set by NT network timing. So it's all set up during boot up of the system. So I think that is the reason why you probably wouldn't want to have multiple timers and clocks controlling timing on this. So it's fun to do this, fun to have experiment like that with the system sound clocks and getting using hardware timer on a sound clock. But I think in general, it might not be for security reasons and other integrity reasons probably. It's not going to be useful. That would be a reason to not do that. Yeah, but it would be a fun experiment. So if anybody wants to try your welcome, please CC me when you will send the batch. I want to take a look at it. Thank you very much, Candace. Thank you, Yvonne and Shua for your time today and thank you everyone for joining us. As a reminder, this recording will be on the Linux Foundation's YouTube page later today and a copy of the presentation slides will be added to the Linux Foundation website. We hope you are able to join us for future mentorship sessions. Have a wonderful day.