 Welcome to my what's the clock talk about linux clock system internals first of all who am I so I worked in the linux and embedded firmware field for the last 14 years for my previous job I was a principal engineer for porting linux on the custom ARM SOCs and I worked with SOC design teams so I was a software guy with the hardware guys. For the last 5 years I'm a general admin Libre in south of France and one of my upstream task is to write support for amlogic SOCs in linux and Ugoot. In this task I was contributing for the amlogic clock driver for the last 3 years and a half. So what's the clock ? First I will speak about how the hardware handles the clock. So I'm not a hardware guy or electronic engineer guy so it may be incorrect or partially correct but it's only here to have the keywords and to understand the underlying hardware constraints and designs to understand how the clock framework has been designed in linux. So hardware what is a clock ? A clock in hardware is only a signal. A signal can be 0 1 ok. So a clock is a continuous signal with two edge rising falling and between the edge you have a width between the same edge we have a period and when the period repeats you have a frequency. This is a clock. So normally it's 0 you should have the same period of a time, the same width and the same frequency. The clock can have a duty cycle so you not always have the same length between the width at 1 and the width at 0. And the time from 1 to 0 or 0 to 1 is not always instantaneous. You have a time to set up before and a time to hold after to make sure the clock is stable after and before these times. When you have multiple clocks or when you take a clock and you derive it in multiple clocks you can have different phases. We define phases in degrees. Each clock is not ideal and you can have jitter. Jitter is when you slightly have a change in width and period and excess can be very harmful for the stability of the system. So yeah you expect a perfect clock like you see on a logic analyzer. But yeah the reality is not the same on a scope. You see what effectively goes on the line and often it's not really that beautiful. So the structure, a clock in the system is the heartbeat of the system. Everything is synchronous to a clock and often a system takes a source of clock, an external clock like an oscillator or a crystal. And generate like a clock tree. It will divide, multiply, gate or whatever the clock to give each function of the SOC a particular clock with a specific phase, a specific duty cycle and so on. So in order to generate and provide the clock in the system you have multiple components. So first you have an external clock, we call it a crystal. You have external oscillators but you also have internal oscillators in SOCs, often used for very low clocks like the 42 kW clock. In the SOCs you have PLLs, dividers, gate, mixes and clock synchronisation when you have multiple clocks for different sources to synchronise them. So what is PLL? So a PLL is used to take a clock in entry and generate a different clock in output. So it's really a complex system. It's mainly used to take for example a 24 MHz output clock and generate a multi GHz internal clock to clock the full system. So if you are interested you can go in Wikipedia to see all the electronic and physical theory on the PLL. It's really complex. Ok, I won't go in detail in there. So here is the parallel. Often you will have a multiplier and a divider component to generate a clock and you often have a pre-divider and a post-divider before and after the PLL. A gate. A gate, there is a lot of gates in the system in general because you don't want all the system to be clocked at the same time. So you want to disable some components of the system, some functions. So you put gates in front of the function. So the clock doesn't go anymore to the function but you can't cut the clock when you want. So they use latches to actually stop the clock after a period, or between two periods for example. So you also have mixes. Mixes is you can have multiple clocks sources for a function. So Arduino has multiple choices for mixes. You can have very simple mixes where you take the clock. So when you change the mix, it's directly changed to the clock. So you can change the middle of between two periods. So it will put a glitch. Or you can really have complex digital gliss-free mixes like this one, which are more expensive in hardware for very sensitive functions. So this is a sort of an example of SOC. You see an external crystal oscillator which comes in the package. It goes in a gate because you could cut the external crystal oscillator. So shut down this table for reset for example. This crystal oscillator can go into multiple PLLs, not only one. The output of the PLL, for example, can go to the bus clock. And this bus clock can go to each function, internal or external functions, with a gate before them. And for example, the same PLL clock, that is not divided for them, can go to the DDR controller and clock the memory. For complex functions, for fast functions like HDMI, MMC and so on, you could have a separate PLL that use the same source clock, which could generate a much higher clock for HDMI 4K, for example. And this could go on a mix. And this function could select between the bus clock, the first panel or the second panel for the third clock. It's a really classic design. So in the Linux world, we used to call the group mix divider and gate as a composite clock. Because in the systems, you see a lot of these groups of clock before functions and so on. So let's see the software. How and how it was implemented. So historically, Linux drivers handled the clocks by themselves. For example, they had a UR driver or CPU frac. They had hardcoded in the driver. The device can do, for example, 100 kHz for EITC and so on. And they had to calculate the clock locally with inside the divider or the mix, knowing the input for this one. But each driver managed it inside. You could not really link a source clock with a function. It was really hard to port it on multiple SoC. In the same family, you could have different PLL, different bus clocks and so on. So it was really complex to handle at the end. So, RMK started. All this started in 2k4. With the ARM integrator family, in a really, really basic clock code, which was only here to set the PLL. So we call it VCO. VCO is the core of the PLL. VCO is what generates the clock. And the PLL globally is the group with the pre-divider, the post-divider, the filters. So this was really designed for one function to set the PLL of the system. So here we call VCO the multiple PLL of the system. It was really basic at the time and only for one architecture. So, 2 years later, finally it was moved to a common clock.h. It managed to handle the basic function of the clock. So you want to enable the clock for gate, for example, or PLLs. Get the rate, actual rate. Calculate if the rate was achievable. Set rate, it changes the parent of the clock for a mix. And get the possible, the actual parent. But it was still not managed in a common fashion. So every machine code implemented their own clock code. For example, the integrator did an implementation, it was basic. Home app did an implementation. And each platform driver used them at some point. It was, because the implementation was not the same watch, the usage was not the same at all. For example, you see the Mac integrator. Clock did see, it was really basic. It was only, they kept the VCO setup from the early code. And only added dump functions for enable, disable, set nuts and so on. For example, Home app did a complete implementation. You could change parent, set PLL, the clock set rate. To go through the tree to set the rate to the parent and so on. So they did a great job. And it was not enough. So they added some custom functions. Like clock use, use count, which was needed for the case. Which was missing from the initial clock.h. So there were clashes, because it was okay for platform only drivers. For Home app only, for harmony. But often you will have generic IPs. Then you need a clock input. So they could set the rate, change the parent. And since the implementation was different for each platform. You need some different limitations and different behaviors. So you have some if, there, and so on. And worst of all, you have different duplication of clock logic. For example, all the rate calculation, progression and the parenting system was made for Home app. But not for other, or differently, or maybe badly. For example, all the clock running was maybe done correctly for Home app. And not for the other platform. So it was not right. For example, I am part of the one who added a fake clock to satisfy a driver. Which is that something I want to speak about. So to solve this inconsistency. Okay, my toolkit will work for TI at the time. Introduce the common clock framework. It was like 5 years later. So for 5 years they handled this stuff. But at one point it was not manageable. So the result was the consolidation of many different strict clock definitions. And different framework implementations. And the goal is an implementation of the well-known clock API. So they kept the clock.hapi, which was okay-ish. But they took all the different implementations and made a common one. With all the good practices and all the good algorithm. So the idea of the common clock framework is not a standalone framework. Like other stuff like HCI, PCI, etc. Where it was really around a driver for all device. No, here it's a real framework. It's a library. So drivers are responsible for populating the framework with a clock topology. Plugging the ops, the callbacks, physically probing the hardware. So when a driver puts the clocks to the framework, it stays in common. So you can group, you can link clocks from one model to another driver. It is the goal of the common clock framework. So with the library, you need to provide some clock ops. It's the base of the clock control drivers. So for each clock of your system, you will provide some, not all of them, some of the callbacks. For example, prepare and prepare a monetary. For example, init is monetary. But not all clocks can handle all the functions. For example, only gates can enable, disable, etc. So, for example, for dividers, you will need a recal count, recal write, run write, set write. For mix and set parent, get parent. For PLL, you need almost everything except the parenting. And you can have wild clocks, for example, which needs all the calls, for example. Or you can have very basic clocks, fixed clocks with only one callback, get write, for example. So when you give all your clocks with the list of the parents, the framework will be the clock tree. It will link the parent of each clock with the name of the clock to provide to generate a tree. It will cache the current parent. It will calculate in the tree the write per clock. It will cache the clock by reading the registers and by calling get write for each clock. If the get write, if it's not momentary, it takes the clock of the parent, for example. And you will have a complete tree of clock writes. And when you want to change the clock, you do set write, for example, or get run clock, it will work the tree regularly to match the request by reparenting, for example, if possible. If asked, if you have flags, you say you cannot reparent, or you cannot reparent, for example. And you also have enable disabled. For example, if you do disable for a clock, which doesn't have enable disabled calls, it will propagate to all the parents. It will not disable all the exam because it has some contours, some enables and disabled contours. For example, you won't cut the main gate because all the previous enable will also propagate to all the clocks up to the root clock. So the enable count will only decrease by one if you do disable. So, since the beginning, it has quite evolved over the time. So you have clock in notifier, for example, you can be notified on multiple events, like clock change, parent change, pre-clock change or pre-parent change, so you can act on the system. For example, if you use the clock in notifier on M-logic to do DVFS, where the clock path, the CPU, is quite complex. So you need to, when you change the clock rate on one path, you are notified before changing, so you can switch to a safe clock until the clock timer did the change on one part of the tree. And afterwards, you are notified to post change and you can put back to the change tree. You have device 3 support, which is a big thing because before device 3, it was quite complex to use. The added clock accuracy support is not really used. First, you can specify accuracy in part per pillion, and the accuracy will propagate to sub-clogs, for example. You have clock phase, you can specify, change and read the clock phase for a clock in degrees. You have clock cycle in numerator, denominator ratio. There is clock exclusivity for a consumer, because until this was merged, when a device set a clock, for example, in a tree, that use a PLL, for example, when the set rate of a SPI driver set a clock that change a PLL value. If another consumer use the same path, for example, the set rate of another consumer would change the PLL, that change the clock of the SPI, so it was wrong. So now you can tell the clock framework, ok. My clock, I want to keep this rate, I am exclusive on all the tree of this clock, over up to the root clock. So you cannot change any clock on the tree. So if another clock want to use a set rate, it will either keep one of the clock in the tree same rate, or use another rate, or fail. And you can set rate invariant with a range, a mean and a max. So before device tree, so the mapping between a device and a clock was fixed. Which was complex to write, honestly, before DT, when you needed to set up all the devices and the relationship and so on in C. It was really a complex and long task, honestly. And the MAC code is linked to the device and clock statically. So it was, there was nothing dynamic or you could have dynamic rate, it was in code. So we need to rebuild the kernel to change so it was static. And each clock was associated to the device structure. So you could have a number of clocks for each device structure. So, but what he was really missing is how to link a clock output of a controller, for example, a clock input of a consumer out of the clock controller and driver. There is no simple way to link, for example, a SPI controller and a SPI device. Ok, there was no simple way for that. So often it was not discriminable and it was described differently. So device tree provides a way to link a clock output to a clock input. Whatever the source of the clock, whatever the consumer of the clock. A producer can be a device and a consumer can be a controller, clock controller. He can be everywhere. So you can link clocks between clock controllers, between devices, between devices and controllers in everywhere. Before that, it was nearly impossible or very complex to do. So with device tree, you can declare multiple clock providers. You can have multiple clock controllers in the system. You can declare simple clocks, clocks provided by devices, and special clocks generated by special devices like PWM clocks. You can link clocks between devices and you can set the constraints in device tree for parenting and rate between clocks, for example. So this is an example of clock bindings, for example. Like you see the system is here, you have an oscillator going to the system. This oscillator clock can go either in the UART directly in the PL and the UART can take an input, take an input to clocks. The output of the PL is in the oscillator input. So this little diagram is how you express it in device tree. And everything is generic. When you set up all the specific data of the clock in device tree, and you link them with, like you see, classic device tree. Ok. So still, the clock control information is still a very heterogeneous. You have a very different way to write a controller. Because in the last 8 years, each maintainer introduced me a way to declare clock and so on. So you will see every clock controller driver is quite different. I've a lot used these COMPETIT clocks and so on. Because initially they thought the clock framework was not smart enough to handle the link between the component of COMPETIT clock, which is false because, for example, for the MLG controller, what was did by Max Turquette, he did the initial controller, is we declare every node of the tree. Every node. Every node with its particularity. So if there is a gate, we declare it. If there is a divider, we declare it. And you have flags. You can tell it's read only. You can tell it's fixed. But you need to declare everything. And then if you declare absolutely everything, the clock framework will handle it correctly. And we work the tree and set the rate of parenting correctly. And it works, actually. It's pretty good. And if you have an issue, you simply add a flag to a clock to tell, or this clock should not re-parentate, for example. So I think the MLG clock controller wasn't most modern, as the most simpler. There is absolutely no logic, amlogic-specific logic in it. We simply declare all the clocks. We handle the amlogic-specific clocks. But there is no logic in the controller driver itself. We simply declare all the clocks. The size logic for the... One logic is for the CPU-DVFS because you need to respect the stock designer's clock path because the CPU, you need to keep... You have the white frequency, then use the white path for the CPU because you can have really high frequency, like 2 GHz and so on. One of the limitations of the clock framework is the tree working, which is recursive. So I know Stephen Boyd would like to go with it and avoid it to be recursive. But yeah. Worker has started. There is an implementation, but the switch is quite big. So it may make some time to switch. It doesn't handle some now very important properties because in the old times, you only had very slow frequencies or only the high frequencies for the CPU and the whole system, you only had low frequencies like multiples of 100 MHz. Now you have clocks that provide the HDMI output of very fast buses and when you go high, you ask high GHz clocks, you have complex PLS with complex settings and filters and you have jitter. And you have, depending on the path, depending on the divider value on the multiply value and you have different jitter capabilites and this is not an error at all. The clock framework doesn't know which path is more. So this was, there is a theory to solve this to actually tell okay this path, this clock path you want to handle it. I will handle it in a specific function for example. Still in discussion, there is no clear way to handle it right now. On now, the new ARM systems or the new ARM David systems ARM V8 systems you have part of the system which is handled by the firmware in the SCPI or SCMI over and this firmware or this TFI firmware needs some clocks for example. And there is a really no simple way to tell okay, this firmware use this clock for an internal device. So how to tell the in DT for example, you cannot. So there is no proper way to describe. Today we set the clock as critical for example in the controller but it's not the right way to do. There is no handoff mechanism from firmware to device. For example, for DVFS you could have your firmware that handle the DVFS and then when Linux is fully booted you can say okay, now there is no more firmware that handle it and it's me. But in the meantime between the the firmware boot and when the the handoff could be like when the module is loaded you could have the clock clock framework that cuts the gate and cut the clock for example, or change the rate. So there is no simple mechanism to do that. And like I say, there is no dynamic clock path prioritization. There is no way to prioritize the clock path for HDMI for example, when you have HDMI 2.0 4K you need to provide up to 6Ghz to the 5Ghz for example. So you need to to select your clock path how the sub designer designed it. Ok, so this is all and if you have questions don't hesitate.