 Hello, folks. Welcome to a deep dive. We'll get going here in a few minutes. If you're watching this recorded, make sure and check the description below for time codes. Yeah, so check that out. Check out the time codes. Thanks in advance to David for putting those in there. And somebody wants to help out with those. I think David's on vacation a few weeks, so we can talk about that later. Yeah, so I'll say hello's and we'll get going here in just a bit. Folks are passing the link around. Thank you, folks. Hi, G3 holiday. Hi, Theronius PX. Something like that. Hi, Yanni. Hi, Dexter. Hi, Bruce S. Hi, MD Roberts. Good morning, unexpected maker. I think I've got all the chats up. Good evening, Christian. Hi, Dewester. Hey, Mr. Certainly. Oh, let me pause my videos here. Everything looks good. Hi, Piata. Hi, Stevie. Now it's a Friday. Hello, Minnesota Mentat. Yeah, we've had lots of streams today, which has been great. Perching on my chair here. Hey, David. Hi, Jonas. Hi, Bruce Siegel. Hopefully it's in that right. We'll get going just after two here. I like to give folks a little chance to pop in. And as always, it's pretty casual. Hi, wagon loads. Hi, Hamslabs. I hope you're well. I'm doing well, yeah. It's a cooler day today, which has been really nice. I have the windows open. The fan's not on. It's only 78 degrees Fahrenheit in here. I think yesterday it was like 87. It's not pleasant. 90 in New Jersey, you know, thank you. I was just telling my spouse, I was like, I really, you know, I'm kind of, I'm ready for some cooler weather. And I didn't, I hadn't, when I said that, I didn't realize that this weekend we were going to have cooler weather. So not too, not too sad about that. Bruce S has just received the Morpheus 240 ESP32S2 board because I have to work on the circuit python port. At this point, point is more of a copy something else process. Yeah. Jonas says, I more or less randomly dropped in, but hey, circuit python. Hey, welcome. Yeah, yeah, we'll talk about circuit python. If you have questions, feel free to drop them in the chats and I'll answer them. I love doing that. Hi, Dave. De Wester says it's the mid 80s here in the upstate South Carolina. Mid 80s isn't nearly as bad as 90s. G3 holiday says 93 here. Wagonload says it's 77 in your room. That's right about, right about where we're at here too. Oh yeah, praying for rain, lots of fire in the BC interior. We did get a little, Randall says, isn't this a muckbang stream? Is that an auto, weird auto correct? Yep, we got a little rain. I made sure last night before I went to bed to try to close all the windows that would let the rain in. Chicago has chance of thunderstorms and is 84. Muckbang equals eating show. Oh, cooking and eating. I mean, I just had lunch. I haven't done that yet. Yeah, thank you folks letting me know what that is. Jonas says just started to do some basic Circuit Python with my son with a bunch of Adafruit products. Awesome. I hope it's going well. We try to make it easy. Minnesota Vintet says again much love to the timekeepers. Yeah, that's that's usually David, DCB. I try to drop time codes in but I'm not nearly as thorough. Yeah, we're cooking up some Circuit Python. All right, it's two or three. I've procrastinated enough. Let me do the whole day of Circuit Python. It's been awesome. I'm glad you enjoyed it. I was watching the earlier streams, too. I enjoyed seeing everybody's perspective on all of it. So I don't have a whole lot I was thinking of for special for Circuit Python Day. But yeah, let's get started and we can see where the stream takes us. So hello, everyone. My name is Scott. I go by Tannute Online. I work for Adafruit on Circuit Python. If you don't know what those two things are, Adafruit is an open-source software and hardware company based out of New York City. I live in Seattle so I work remotely for them and tend to do this. Oh, the cat's so silly. Yeah, so they do open source hardware and software. And then Circuit Python is a version of Circuit Python, which is a programming language, really a great programming language for beginners. And it is designed to run on microcontrollers, which are these little inexpensive computers. They actually live in lots of consumer products as well, but we take those same chips and we make them more accessible in what are called kind of dev boards, development boards. And so that's what we run Circuit Python on. You can make all sorts of projects out of that. So those are those two things. If you want to support me, you can support me indirectly by going to Adafruit.com and purchasing something there. They pay me to do these streams. They pay me to work full-time on Circuit Python. So it comes back to me. If you want to chat with me and a lot of others outside of these streams, you can join our Discord server by going to the URL adafru.it-discord. Adafru.it is like Adafruit with a dot before the IT. That's our URL shortener. This is a deep dive. It happens every week, almost every week on Fridays at 2 p.m. Pacific, but occasionally it's shifted to Thursdays at 2 p.m. It typically goes for two hours or more. So we're about five minutes in. We've got just about two hours left. Questions are welcome. We've got plenty of time. It's super casual. Ask them away. I see. Wagon Loads has a question here. We'll get to in just a second. And yeah, so I think next week is on Friday as well. So we'll see everybody next Friday. If I do change it, we do have a deep divers role on Discord now that I will ping and let know that if it if it switches. And I also do I also do have I usually do blog posts like Thursday Thursday or Wednesday depending on what it is. So keep an eye on Adafruit blog for that as well. I usually remember. A couple days I've done it the same day. Last up, a piece of housekeeping. Let me switch to the cat cam here. Spook is the kitty kind of in the top left of the image there. He likes to move around. He's like kind of underneath. It's kind of hard to tell, but he's like underneath the door of his carrier, which is kind of funny. But he is epileptic, which means he does have seizures from time to time. It's not very common. He's been doing really well. But if I'm just like paying attention to him and not the stream, that's why. So just a heads up with that. And with that, I think that's housekeeping. Thanks again to DCD for doing 10 codes. And let's answer wagon loads is question, even though I saw some folks in the chat answering it as well. So the first question here is, does circuit Python run on the ESP 80266 or the ESP 32? Bruce replied currently only processors with native USB, so no. But maybe if Billy workflow gets there than yes for the ESP 32. Bruce says I did just get in some ESP 32 C3 boards and other another thing to stare at. And I have collected a risk five boards mostly unused. Epileptic gerbil. Interesting. Is it well controlled? Are they on medicine? So we did have ESP 80266 support until until including circuit Python three. But we found that people, the way that we were advertising circuit Python was you plug it in as it shows up as a drive. And that's just simply not possible on the ESP 80266 or the ESP 32. And so what we did is we at that point, we made it a decision to be very strict about the native USB requirement. So by native USB, I mean, like a, it's a specific peripheral on the system on the chips that run that can manage a lot of different types of devices, USB protocols, so to speak. So the ESP 80266 and the ESP 32, they have this chip that does this conversion between USB protocol and a UART, which is just like serial. But because that's how they provide USB, it means that they cannot provide mass storage, which is what it makes, what makes it look like a drive. So that's why we, we stopped supporting ESP 80266 and why we still don't support the ESP 32, the regular one. As Bruce pointed out, I've been working on the BLE workflow stuff. So there's a high likelihood that as we get the BLE side of ESP IDF, which is used kind of across all of the ESP chips, we'll be able to actually do what we've started pioneering, which is this BLE workflow. What we mean by BLE workflow is this workflow where you can transfer files over Bluetooth low energy, which is what BLE stands for. And you can also see the serial output. It's still really early and it's only been implemented on really early and only been implemented on the NRF 52,840. The challenge with the BLE workflow stuff is that there's all of this host side of software. So the thing that, that sends the file, the thing that reads the serial output, like we have to create all of that as well, which is actually a lot of work. I didn't fully appreciate it. And but I, if you think about the way that we developed the USB side, like all of the OSes already supported mass storage because that's what a thumb drive does. All, almost all of the OSes did the CDC because it's like very commonly used, which is the serial link. So with USB, it's just a matter of getting USB going and then like the OS works with it. But that's not the case for Bluetooth low energy, which has been a lot more work. That's been, yeah. So, so the device side I think is in a, is in a reasonable state right now, but we're not going to really be able to use it and push it until we have apps on the host side as well. I've been working on the web Bluetooth version, which will be good for Android and for Mac, Chrome will be able to use it. And then we're working on two different iOS apps. We're using, and by we, I mean, folks, Adafruit. Antonio is working on Glider, which is an app that we are working to really be kind of bare bones, really bare bones for it. So it's Glider is just going to integrate with the iOS files stuff and just send files back and forth. And then we're also working, so Trevor is working on PyLeap, which is an app that is circuit Python specific. So Glider is just files. It doesn't know anything about circuit Python. PyLeap will be a much more integrated like pull, pull Python code from Adafruit, Adafruit learn and transfer it over and like see the serial output as well. So that's where we're at. I am not good at any of those host side things. So I'm kind of doing smaller tasks right now. Jonas asks which devices do you expect to support? So the PyLeap workflow in 7.0 is currently active for all of the NRF 52, 840 boards. And I believe all of the NRF 52, 833 boards as well. We have basic support on the Microbit V2, which is the very first board that does not have USB on it. So it's the first board that has Beely only. So any of those NRF boards will do it by default. So what will happen is for boards that have USB as well, you will have to, it'll blink yellow and then it will blink faster in blue and you have to hit reset during that time in order for it to broadcast out. I wanted to make sure that you were able to control when it was connectable. So on those boards, that's kind of what you have to do the first time. And then once it saves the information, basically once it's bonded, it will start up and broadcast so that devices who have bonded previously can connect again. If the board is Beely only like the Microbit, it will do that process without having to do the reset because that's the only way you'd be able to use it. WagonLoads asks, what about on OTA devices? So OTA is short for over-the-air and we kind of only talked about that a bit with the ESP32 S2, which is using, which typically uses over-the-air to reload flash firmware over Wi-Fi. So that's a little different than what we're talking about. So there's like, kind of, there's like three pieces of software you could think about on a circuit Python board. There's the bootloader and then the circuit Python firmware itself and then the files that circuit Python runs. So that's like your file drive. OTA typically is referring to the thing, is a function of the bootloader and that would allow you to like upgrade circuit Python itself. We haven't really looked into that on the NRF side. So that's to be determined. We've got to figure that out still. Hi Keith EE. Oh yeah, I should plug the, hi Microdev, hi Naradak, and Johnny. Hello. Can the boards you're programming today access the internet? So the NRF 52 840 boards cannot access the internet directly, but the ESP32 S2 boards can. Yes Microdev, I can do that. I've got to, let me cover the reddit first. So I can load, hope that answered your question about over the internet. I wanted to plug, let me switch to the desktop. So Keith EE is also a mod for Reddit. I don't think they might be saying that, but we are actively doing an AMA. So if you have questions, your Reddit user, you want to ask questions. Here's this link, I'll put it in the chats. And kind of a bunch of us are monitoring that. I went through and did a lot earlier. So I might actually just scroll down and I didn't want to, I didn't want to do the AMA, AMA while I'm on my stream, because just like the hack chats, I tend not to, I tend not, if I answer it on the stream, I won't type it out. So I wanted to make sure and split those two up. I don't see anything new. Oh, narrow dot. Thank you for replying. Okay, so we have, I have a stack of two, two things we want to talk about. Keith EE says a follow up from the AMA. Microdiv, I'm not doing quite your stuff quite yet, because it's going to be a bigger discussion. Keith EE asks, can I ask a follow up to the question from the AMA? AMA, one user asked about threading and the response was that it was difficult to do true threading for a number of reasons. Assuming that at some point more Pythonic style threading gets implemented, it would either circuit Python or MicroPython run into Gil-like issues. So for those who don't know, the Gil is the global interpreter lock. And it's a Python thing. I don't know exactly where that lock happens. But I think if you look at how threading works on the RP2040 with micro Python, we don't support it and sacrifice them. But in MicroPython, you'll see that you still run into issues sharing memory. Because, like, even though you have two CPU cores, they're using the same memory. And so if you're ever doing an allocation or memory manipulation, you have to worry about that being a shared resource. So I think generally, yes, you'd have to worry about it. I don't know. I don't know exactly the Gil details. But yeah, I don't know a lot about all that stuff. Damian is a better person to ask. Damian or Jim. Ask Patrick Dovey says, I'm so behind I've not listened to the TalkPython to me episode yet. And now there's an AMA I've not seen to so much to do. Yeah, the AMA was something that Keith was asking me about. But then like, I was like, Hey, actually, next week's Circuit Python. Circuit Python day, wouldn't it be cool to have that as well? So yeah, check that out. And folks don't know, we do actually have a Circuit Python subreddit as well. If folks are there, I tend not to check it sporadically. But there is one if reddit's you're doing. Okay, let's let's start to dive a little bit since microdev. I know it's super late for them, I think. So let's pull that up. So where do we want to start? I think I'll leave that open. So I remember to check it later. So microdev has this pull request. And I want to go over the problem we were trying to debug last week, because I figured it out. So remind me remind me to cover that. So there's this PR from microdev microdev has been doing an awesome job. Helping us move a couple pieces of functionality that micro Python put an assist module into their own modules as C Python actually has them. So that's been really helpful. And I was looking at it. And there is a really good question. So if we look here, we can see that there's like deletions in pi stuff, and then additions into how we set up our make files. And then the way that circuit Python is structured for those who are new is we have shared bindings. And this does the like shared bindings does the like translation from Python land into C land. And then you can see here that like register just calls the shared module register and register calls the unregister shared module. And then we have the shared module folder that actually has the logic. And I had commented here that there's this at exit callback T global here. And I said, no, I think, I think you'll want to have this an MP state so that it gets collected on garbage collection. So it isn't freed. Doing it as a global risk the GC freeing the memory you need. So I think we want to talk about that why that is. So usually in a C program, it's okay to have state. Well, it's it's okay. It's okay to have global state, especially if you're not going to like, let anyone else touch it. But there's an extra challenge with circuit Python and that memory is allocated. Memory is allocated by kind of the circuit Python, micro Python mechanics. It's it's a mechanic we share with micro Python, but we have some customization to it that micro Python hasn't taken in. So if we look at register function here, we see that there's this realloc call. And there's also this M malloc call. So these are these are things. So what is memory allocation? So there's kind of two, two styles of three, three styles of memory allocations, there's the ones like the things at the top that you just know you're going to always have. So like this size t callback length variable is it is a global variable. It could be static. So it's not necessarily global, but it's kind of like it always exists. And the place in memory that that variable is stored is kind of is dictated at build time by the compiler. So the compiler will say, I've got all of these variables that I know I need to allocate kind of globally upfront. And that ends up in two different sections in the linker file. But there's the ones that are zero by default. And there's the ones that aren't. And so in initialization, I think that we saw it last week as well. There's kind of two sections. There's BSS, which is all of the variables that start as zero. So instead of, yeah, there's all the variables that are start as zero. And then there's all the variables that start as nonzero. And that's the data section. And so the way that the data section works is that all of those variables are kind of like squished next to each other. And then in the flash, you have a copy of like all their starting values. And so the very one of the very first things you do when you start up is you copy all of those values out of flash into RAM in the section that you're going to store all of those pre initialized variables. And then after that, you're going to have a section the BSS section, which is more variables that have that global scope, but they all start as zero. So all you do is you, you know the range, you know how long that is and you just set it all to zero. So it saves you flash space to split those two things apart. And then there's two other places that memory allocations can happen. There's the heap, which is the like, I don't know when I'm going to need it, but I'm going to, I want to have some like dynamic memory. And then there's also the stack. And the stack is all of the memory that I'm using as I call functions and store things temporarily. That's kind of what the stack does. So the stack is typically the highest end of the memory range and it grows downwards. So what we were seeing last week is we were reading the stack pointer, which is like the low value and reading all the way up to the top of RAM to kind of get an idea of like where in the call stack we are. But the heap is something that can be managed either by standard libraries or it can be, in our case, it's managed, CircuitPython actually has two ways of managing more dynamic memory. One is more common than the other. We have one that is kind of super supervisor level stuff, which we don't use very much, but it allows us to store like dynamic USB descriptors or like small pieces of state like that where user code may not be running, but we want to kind of hold on to something kind of longer term. And then the remaining pieces of memory between the top of BSS or the supervisor allocations all the way up to where we think the stack will be the like lowest point of the stack. So the tallest stack, all the extra range we call the heap. And the heap is just where like, if you create a variable in CircuitPython or you import a function or something, you can do all of that. I got distracted by the fish comment. All of that ends up in this heap. And there's a process called figuring it, like trying to figure out where in that space is free and where you can put new variables. That process is called allocation, which is also usually truncated or called malik memory allocation. And then realloc is reallocating some existing memory. So the worst case for reallocating memory is it will free the memory at house and find a new spot for it. So yeah, I think while Bruce said that one of my first streams was on memory. And in fact, that that is true. Before I was doing these regularly, I did do a couple deep dive streams. When I was doing some tuning, some tuning of the memory stuff. Sorry, let me divert. So ask Patrick W is talking about fish and says I use oh my Z shell. There is also oh my fish, which is what I use. So yeah. So wagon loads asks, can you defrag the heap or the stack? So the stack, you don't really the stack doesn't really have to worry about fragmentation. But the but the heap definitely does. And there's a lot of academic papers about the heaps. But the challenge, the challenges that we have for circuit Python and micropython is that they're pretty small, pretty small systems. So so in essence, the ability to switch to move memory around if you were to defrag it, you have to have some layer of indirection so that the references into that table stay the same while as the locations in the memory change. And so basically it has a memory cost to it. So circuit Python cannot defrag micropython can't either. So yeah, so this is memory allocation. And so what it does is it it starts from the last place that left off, basically when the last thing you tried to allocate it kind of starts there, and then looks from there until it finds a free area that's long enough for the allocation you're trying to do. The change that I made in circuit Python and was one of the topics of my early, like the original deep dyes, was that circuit Python, if it knows that the memory is like almost certainly going to stick around the rest of the Python script. And that tends to be anything that's in imports. So if you import and you allocate a lot of functions, like, you're not going to delete those. Those are all going to be really long lived. So circuit Python has these mechanics to say like, if I import something, let's just shove it. What it does is all it does is it changes the allocation process. Instead of starting at the start of the heap, it's kind of starts from the other side. And the reason to do that is that if we know it's long lived, we can pack all the things that are long lived together, like, more densely than the stuff that we don't know the lifetime for. It's like a very crude version, kind of a manual version of what's known as a generational garbage collection process, where if you have that table of indirection are able to move things around, you basically from time to time, you'll recompact it. So you'll, you'll take all the stuff you allocated and that's still available that's still in use, and move it all next to each other, and then start allocating below that again. So circuit Python can't do that. But what we can at least do is look ahead and say, like, if it's an import, if it's a function, it's like, probably going to live a long time. So we'll put this in this other section. So the other process of the other part of the, the garbage, or of the managing heap is figuring out at what point things are no longer like memory is no longer in use. In regular Python, this is done through a process called reference counting, which basically means that for every piece of memory that you're you have, you're keeping track of how many people are are are using that memory. And when that number goes from one to zero, when no one else is used, when no one is using memory anymore, that's when you free in circuit Python, because we're trying, and this is all for micro Python. So in micro Python, where you don't want to have extra spend the extra data for everything that you've given out, Paul says happy circuit Python day. Thanks, Paul. Instead, what the way that micro Python does it, and this is true for circuit Python as well, is they use a mark and sweep process. So when you go to free up the memory that you can, there's a mark and sweep process. So this can happen. If you import GC and call GC dot collect, this this will trigger this process. But otherwise, this process happens if you're trying to do an allocation, you've looked through all of memory, but haven't found a spot, then it will go through this process as well, and then try to allocate it again. And if it can't allocate it after doing that collect, then it raises the memory exception, the memory exception error. So the way that micro Python does it is that it's a two step process, it's marked and then sweep. So you say for all of these blocks that we think are for all of this region that we think are in use, we're going to have a little a checkbox that says whether we're still using it or not. And the way that it fills in those checks boxes is by looking in certain places and then looking to see what that memory refers to. So for example, imagine that you have like all of your you have a dictionary that stores all of your variables at the top level. So what you do is you say, okay, here's all the variables that my Python code currently has. Let me look or mark all of the memory that that is being referred to by those variables as in use, because they're in variables. And then what you do is you look at all of those structures, say it's a dictionary and say I'm and recurse. So so if I'm using a dictionary, everything the dictionary uses is also in use. But then there's also some there's stack memory. So there's like variables that are not globals, but they're used by function calls. G3 holiday says, does Python have garbage collection? I haven't ever needed it. So I don't know. Yes, it does automatically. And this is, I should say that the reason one of the reasons I'll push back on people about Python being bad for time and critical things is because this process can happen at any point. It's not exactly true, but in practice, the way that Python code is written, it could happen at any point, because people don't understand when the things are doing necessarily cause allocations. And if you cause an allocation, that means that you could cause a garbage collection process, which is what we're talking about. So the question is, is where do we start? So for every pointer that we know we're using, we look at all of the memory it uses to make sure that we keep that as well. So it will look on the stack. So from the stack pointer all the way to the top, because there's local variables for functions that could be referred to there, it will look on most ports, it will actually look in the registers of the CPU as well, in case the references are there. It will look in the and then it will look at the very top level stuff. So there's this MP state variable, which is kind of where we where we started where micro micro dev was asking about. And it will look in that MP state variable and and collect what they call root pointers. So although those root pointers will be collected, which means that they'll be they'll be marked as in use, and then all of the things that they point to could also will also be marked as in use. And so the reason that and so after everything is marked, anything that was allocated but not marked as in use will will then be free, which means that in practice, the values of the memory will just stay the same. But at any point, the garbage collector could choose to put something else there, which would then change the value of the memory. So that's called like a use after free problem, where you're using memory after you thought you freed it. And this is why I want to do rust at some point, because rust has better memory sharing semantics so that the compiler can check things ahead of time. But that's not the not not how circuit by the marks, that's not the Python world. Where you and that's good for beginners, beginners shouldn't have to worry about memory management. You know, they get there when they run out of memory and circuit Python, but they shouldn't have to worry about like the details of it from the get go. And that's why that's why Python's great for beginners is so much of the Python language, you don't need to know upfront, you can just learn it as you need it. I think that's one of the major things about Python that's so great is that you can start from nothing, you can learn print statements, and then you can learn the things you need to learn, you don't need to learn, or you don't need to learn a lot of stuff you then need to ignore. As somebody who taught some Java, it's very explicit and it makes it that you have to teach people things that don't make sense because you can't go into those topics. I remember the very first day of learning to code with Java is like learning public static void, main, bracket, bracket, args. You don't teach what public means, you don't teach what static means, you don't teach what void means. It's a lot kind of all upfront, whereas Python is much smoother on RAM. You can learn things as you need them. Yeah, so bringing you back to this ad exit thing that Margaret was asking about is that the MP state structure will be collected. And let me just see if I can't find where that happens. I have a little bit of glare. I'll make it bigger in that I'll help with the glare too. So basically, I think the problem is with as as it's written, micro dev is using a global and that global doesn't get and it doesn't get marked. Globals don't get marked unless you explicitly have them get marked. Whereas if you just put an MP state, it will get marked, I think. So where does that happen? So main not see I think has a collect a collect is the the word you want to find. So here's this GC collect. I think this is a top level hook in micro Python. So here you can see here's how we collect the registers. So we actually get like the register values. And then there's this call of collect route. So this gets the the VS VFS mount tables. So this is like accessing the file system. We can and then we have a bunch of hooks for different modules. So we allow different modules to hook things as well. And then here's the stack pointer hook to now I think that I'm pretty sure MP state is also and that it must be collected in the GC itself. So we're in pie and then we're going to pull up GC. Let's find collect. So here's the sweet process of sweep happens after you mark. DCD says what if there are more than 10 registers in collect call to get registers, then we would have our problem and it's entirely possible we do. But yeah, that's it could be a lot. Yeah, you hate magic numbers. That's wise. Yeah, it really should be a macrosion on a purport basis. Okay, so that's the sweet process. So here. Oh, here we go. So GC collect start, which is what we called from GC collect. And so we can see that it's doing some locking for threads. And then what it does is there's this MP state context. And it's finding the offset of dictionary locals and queue string, and then collecting from there to there. And then the root pointer section is MP state VM, permanent pointers is the thing that we added. And the pi stack as well, root start, root end. Yeah, so I think this will cover the I think this will cover if you just put it in MP state VM. Which I think if we look it's MP state dot H. And here's permanent pointers. Okay, so that but that's in state mem, MP state VM, loaded modules, and then root pointers here. So I think you would want to end these root pointers. And then here it has this warning that the root pointer section ends here. So all this stuff won't be marked automatically. Anyway, so that's why I think there's kind of two ways to deal with this, like, I'm going to allocate something. And then I need to keep it around, you can either hook into this collect stuff that we've done, or you can add it to the MP state VM stuff. And I think we do kind of a mix of both. I kind of like the being that the more explicit of like, here's the things that we're going to collect all the time. Great, micro dev says thanks for this memory dive. That really clears things up. Awesome. Well, hopefully that will that will get you unblocked on the exit stuff. I really appreciate you picking that up. One thing, okay, next thing, let's move on. We have a nice bow on that. Thank you, micro dev. The other thing I wanted to talk about is last week, we had this really, really weird bug, right? So last, last week, we were struggling, struggling to use pyocd to figure out why assertions were failing. And I was like, my mind was just blowing. And I was just like, this doesn't make any sense. Like everything's being copied over, but it must be being corrupted, blah, blah, blah. Like all good things, I took the week off or the weekend off. And I came back to it on Monday. I wasn't going to spend a lot of time. But I was like, you know what, like, let me take a look at this again. And somehow I realized, you know what, I should make sure that the flash memory matches what the Elf file says. And it turns out, it turns out that the uf2 bootloader for the nrf chips was broken. But it was broken in a way that if the uf2 file size was more than a megabyte, meaning you had more than 512k of code or stuff to flash, it would just stop working. Or like, it would flash up to that 512k, but then fail. And so it would fail to copy that stuff. And that's usually at the top of flashes where all of those default values for data sits, which is just like bad news. So basically, we were like partially copying nrf debug builds over. So I filed an issue on the nrf bootloader. And then TAC was quick to fix it. So if we looked at closed issues here. So I filed this issue two days ago and TAC like, I said, oh, there's there's two other things like I wrote a script. So the script we had last week, right, compared to flash to RAM, and it was correct. But then it was still failing. And then I also realized, and this is probably part of it, that I realized that if I used pyocd to flash the builds, it worked. Because it doesn't have this bug. So when I looked at the data section in the Elf file, I realized that the values that pyocd was reading out for the uf2 flash version didn't match. So I posted there and this and TAC replied quickly saying like, oh, yeah, I can do that, or I can reproduce it. And found these issues. And there's like some reason that it's protecting some of the areas. And so it's been fixed. So if you have an nrf board, and you're doing debug builds, and you're using the uf2 bootloader to load them, make sure you have 061. This is a bug that's been there for a while. So that was like, such a relief to figure out because it's been plaguing me a while, I think. So that was really good to figure out. And then, oh, the other thing. So I was, I've been in this weird state of like, I really should do this like non circuit Python work for the hosts apps. But I'm like so bad at JavaScript. And I'm so like out of my league with all the mobile stuff. I was like, I can't really do that. I've got to wait for other people to find issues with the BLE side of circuit Python and go from there. So I was trying to pick up more bugs, and I was still having this problem where GDB takes a bunch of memory. And it's which was the reason I was looking at pyocd in the first place, which is just like incredibly frustrating. But we were Dan kind of said like, yeah, but the arm ones that I'm working like I don't have this problem. Like so it's it must be something with your setup. You should really just use the arm versions and who don't want to use the arm versions. I like, I like being on the cutting edge. But I was looking again at the GDB issues and somebody said, like, oh, this is not the two things I was suspecting is GDB or GCC, which is the compiler. What I hadn't thought about was LD, which is the linker. But in some of these issues that says, oh, the linker is actually broken. It's generating this debug data that's wrong, which is causing GDB to to use a ton of memory and working correctly. So turns out they had fixed it. And I updated to bin utils 27, which bin utils is what includes the linker. And so that bugs been fixed as well. So that's it's been a good week getting past these these really weird issues. Any questions? I've been talking and talking and talking. Let me just pause since there is a leg. Tada Moji from Dexter. Yes. 100% Yeah, the linker was busted. It was outputting bogus is outputting bogus debug data. But yeah, it's a total relief to get past that. Now I can just fix some more bugs with the tools that I've come to come to know and love. So I did notice. So I use arch. And the bug had actually been fixed for a while, relatively a while, but they hadn't updated it. But the arch folks are really responsive if you just flag it as out of date. So in fact, it's now officially updated. So this has been utils 237. I updated it yesterday manually. And then you can if you use arch, there's this flag package is out of date and the packages are generally really responsive. So thank you to Anatole for probably said that wrong. But thank you for updating the new sales for me. 2.361 has the issue. 2.37 doesn't seem to. So yeah, that's been great. I can actually debug stuff. So the thing that I was working on most recently. Hands lab. Nothing like when your tools are worked. I know hands. Pi OCD is cool. But the problem I was going to keep using Pi OCD, but they don't support the same D 51, which is where this next bug was on. So that that was a bummer. Have I seen waukey? It's a web based simulator with a really nice gooey chain circuits. It supports Arduino, Pi Pico, and they're going to release ESP 32 soon. I haven't. You can connect a virtual logic analyzer. That's pretty cool and handles Pi O. Also does micro Python and they just started working on circuit Python. Let's take a look before we get on to this next bug or Arduino projects. Micro Python project. I wonder how they're how are they emulating this? Is this all of the G unicorn stuff? But they even have this here. I can contact with one of the devs if you're interested in. I mean, they can use me as a resource, but I don't know. I've always been kind of a downer when it comes to simulators. Like them, the Mako people were pretty hot on simulators and I'm like, there's a difference between the simulator and having it in your hand. So I've always like tried to steer towards just like focusing my effort on the bunch of hardware devices in the emulator tool. Yeah, I think this is cool that somebody else is working on it, but I'm I get so distracted already. As you all know, I get very distracted. Like I have a a Fitbit REO on my desk here that I'm trying to hack to. So like I have enough stuff that I get distracted by that I and this is all web stuff to and web stuff is definitely out of my out of my comfort zone. So yeah, I'm all for it. If they have questions on how to get it running, I'm happy to answer questions. Keith, he says I used to be super in favor of simulators, but that was because at the time it was so expensive to even buy a deaf board. Now, it's a lot easier to justify grabbing one for me. Yeah, like in terms of like where I put my time, like imagine somebody who doesn't have a computer at all. And so this is why I'm like Billy workflows, a big deal. It's like you can spend twenty dollars on a Billy Circuit Python device, twenty five dollars and then connect it to the phone that you have. When we guys point out, I could see some cool opportunities for automated testing. Yeah, I think automated testing would be very it's something we need. We needed two years ago as we start to see like some of these issues that were hunting down for seven are like things that used to be working and now no longer work. They're regressions. So having a test suite would would prevent those regressions or reduce the likelihood. But yeah, this sounds like the person that needs to help me. PIO debugger, that's pretty cool. Wagon load says my first Arduino cost me thirty five dollars. Now they're in the five dollar range. I know, right? Todd Lut says, hey, Scott, check out this cool shiny thing. I know I'm a sucker for shining things. And I start a lot of a lot of projects and don't finish many of them. Yep, I think that's neat. I would point people to it. I wonder what. Editor, this is actually Monaco. Interesting. So the Monaco editor, I think is what VS Code uses. But yeah, Johnny says, seems to be nagging about unit tested testing planted a seed. You nagged me long after I wanted to do on hardware and testing. But it's just a really hard problem. Like in the first year or two of doing Circuit Python, I was doing a project called Rosie CI, which is like a a Raspberry Pi with a bunch of boards hanging off of it. And I worked on it for a while and it found issues. But it's just like such a pain keeping running like it just wouldn't work. Simon says, how's the cats? I really must get playing with my ESP 32 S2 Calgon one. Kitties are good. Oh my gosh. Can you see spook? Let me see. Cat check. Hold on. This is worth moving the camera for. Come on. He's like cuddling with the shredder. That's hilarious. Hold on. I got to take a picture. That's so funny. I keep the ESS. What was the power system for Rosie? That sounds like it'd be a mess if that alone. Let alone managing all the USB devices. So it was connected. The USB was connected to through a powered hub. It was connected through a powered hub. So power shouldn't have been the issue. Sorry. I'm going to I'm going to post this cat picture on Discord. Because it's that cute. And you can't see it on the stream. That's hilarious. I can't scroll the other one. Yeah. It's hilarious. WagonLudz says, Can you put a Fitbit on your cat and be notified when he has a seizure? So it crossed my mind early on when he first had them. That was like six years ago now. He's like six and he had his first one when he was one. Basically I was around so I could keep track of him. And he just ended up having so many that it was just kind of like he has him. He has a rough life. Let me tell you. It crossed my mind. The challenge the challenge with putting anything on the cats is power supply. Right. So like, yeah, I guess a Fitbit's pretty well contained, but then you're still having to like charge it every night or every week. That gets kind of annoying. Diminishing returns I would say. But it has crossed my mind. So the more recent thing and the reason that I have the scale torn apart was I we've been wanting to better track their weight, the cat weight. And I had this idea of putting the litter box on scale or two. So that every time they use the litter box, I automatically weigh them. And so that was kind of like the impetus for me to like try to get some scales to hack. So I'm figuring out how to run the these are. I got Fitbit Arias. The originals. Because it's actually what we have as well. So we we've been wanting to update the Wi-Fi on that. So I got two more from the the Goodwill nearby as well. So I'm working on hacking those two. The scales. He has such a rough life. Okay. Enough cat. Cat stuff. So let's see. Do I have the issue open? Yeah. It's still open. Oh and it's still broken. So I can fix it. So the issue that I was trying to fix is come on. I'll fix that. Would the scale auto adjust to zero after exit? Yeah. Basically what I would do is I would just weigh it perpetually and just know that like the different caps would be. Yeah. Phone guy says I don't know if the we balance boards and communicate things with Bluetooth. Yeah. Actually that's what I was going to pick up as the we balance boards when I was doing all of my thrift shopping. I saw a ton of those but didn't get them because they were so big. So I went to Goodwill and they didn't have any of the balance boards. And so I was almost going to be like, ah. But then I found the scale section and they had like eight different electronic scales. And they had two of the ARIA which I already had one of. So I now have three of those. So yeah it was just like it was a trade off between like how many times do I have to go places to find the we balance boards versus just buying the two fit the ARIAs that already were there. And the we balance board is iSquared C then. It is I think. But I've already found the iSquared C bus on the scale. On this scale. So I have two of them. I should be able to just put two under the litter box and maybe that will just work. I don't know. I do really enjoy being in electronics, being able to hack stuff like that. Scales are kind of hard to find online because they're bulky. So like you really do want to just find them in the store. So I actually got kind of lucky that two of the ARIAs were there. Okay. So there's this switch to per event. And it was for this bug. Oh, I realized you can't see. So there was this bug from December of 2020. So we're a little behind. But it basically says if I do pulse in I can't like auto reload stops working. On the circuit playground express, which is a CMD-21. And here's the test code. So it was great. I just started with this. But I put it on a MetroM0 because MetroM0s have an SWD port. Dave, who's been awesome, says I can't exactly get it in these versions. And then Robert says, oh, no, it still happens. And I said three weeks ago, I was like, is this still in 7? And he was awesome enough to test it again and say, oh, it still doesn't work. Auto reload still doesn't work. Also remote test. Randomly exits after passing through the while loop, which is an error, I think. So I was picking up the SamDs, the CMD-21. And what I found was that the SamD-21 does this weird timekeeping thing There's kind of two. So in 6, I guess, in 6 we switch to the way that the core timing of CircuitBite then happens. It used to be that the Cortex series CPUs have what's called a system timer or assist tick timer which is like this timer that's like right next to the CPU that tracks, that counts at CPU rates. It's great because it's high resolution, but the problem is that it's usually based on the clock that's clocking the CPU and if you're doing sleep then that means that the clock that you're using for timing gets stopped as well because the sleep usually involves suspending the clock to the CPU. So in 6 we moved it with all the sleep stuff we moved it to all of the timekeeping to the real time, the RTC which is either the real time counter or the real time clock. In most cases it's a counter meaning it's just a number but in the STM32 case it's a clock meaning you get date time like you get hours, minutes, seconds sub seconds days, that sort of stuff it's kind of a pain. I prefer counters not clocks. Because like all time goes forward you can do math later to figure out all of the clock stuff. Okay, so the STM221 is weird in that so we moved from the SysTik timer to the real time counter there's a clock mode too but we just use it in count mode and there's two ways it's used sleep so you can tell the counter hey, wake me up or send an event or send an interrupt when the time is this number so there's like a comparison that you're checking against so that's used for sleep and then we also occasionally have these sorts of things that we want to do periodically so we have this notion of a tick and a tick is defined it was one millisecond which is, what, a thousand times a second but because a lot of real time clocks or counters are run off thirty two point seven six eight kilohertz crystals which is thirty two times ten twenty four that's how you get an even second is you pick a frequency where the binary you can divide it using binary a second, exactly so the tick definition with six changed from one millisecond to once one thousand twenty four per second and then if there's more resolution than that we have this notion of a sub-tick so the problem with the same D-21 is that the only way that the periodic ticks can be triggered was there's no interrupt from the real-time counter done on a periodic basis there's only an event and so the way that we were originally doing it is that we would say like oh if we're gonna sleep we'll just set that number but if we're doing tick counting we'll just like every time we run a tick we'll say okay wake me up in another tick which is kind of a pain to do and then Dave Putz who had fixed some issues with the pulse-in stuff had fixed the fixed that stuff by turning off kind of like that ticking while pulse-in was happening which made it work better but didn't the problem is that ticks is used by the auto-reload so when you do a write to USB we don't immediately auto-reload we wait 700 ticks so you do a write we enable ticks and we count down so if pulse-io is doing this stuff where it's turned to ticks off it means that the auto-reload won't be able to work which is what this bug is so what I was switching it to do is that I separated out the periodic thing because I realized late in the game that there was a way to get an interrupt on a periodic basis from the SAMD 21 it's just indirect and maybe I should pull up the datasheet and maybe I should cover the basics of a system on a chip first so um here's the block diagram so if you're looking at a datasheet I highly recommend using a PDF reader that has all of the sections in it um and most datasheets will have block diagrams so a microcontroller is much more than just a CPU it's not like it's not like your computer where you just like plot the CPU down and that's I mean they're more complicated than CPUs as well but generally they're just thought of as CPUs so microcontrollers are really known as like systems on a chip what that means is that there's not only a processor so if we're looking here like this dark blue box is actually the only thing that is like actually the CPU like there's a whole lot of other stuff in this diagram that is not the CPU so if we're taking a quick tour and have I covered this I'm not even sure so um there's the processor so this is the SAMD 21 um this gray area is the high speed bus matrix so this is your memory bus um or at least it's the like first stage of the memory um and it has access to RAM so there's RAM here both the controller and the RAM itself there's a cache and then NVM so NVM is non-volatile memory so this is flash so we have RAM flash uh DMA this is the direct memory access and in the chart here it actually has this M next to it this is master which is terminology we were trying to get away from um and then S is for slave um I'm not sure how they would for the memory bus what would we call it instead it's like the initiator and the responder something like that uh the idea being that like these M's are the ones that say hey I need this memory and the S's are the ones that say like oh that's me let me get that memory for you um you can see USB is up here as well so this is USB has high level high speed access um and the device service unit so these are the things that you use for debugging um and there's an IO bus here too so you can do direct IO and then there's these bridges so these bridges here these three bridges bridge B bridge A and bridge C are um what are known as I think they're known as peripheral buses that's the APB um APB is a peripheral bus and they don't necessarily get clocked at the same speed as the CPU and the high speed bus but because the SAMD 21's 48 I think they generally are I generally think they are all the same clock speed um but you can actually see here that like this bridge B bus is um what connects to the peripheral access controller DMA um the NVM controller and the USB uh USB stuff um and then there's bridge A which has a separate access controller as well along with the clock kind of this is all the clock and oscillator control power management extraordinary clock controller more clock stuff real-time counter so here's the RTC watchdog and external interrupt controller so this is kind of interesting here in the one of the constraints of these memory bridges is that only one piece or one thing on this bus can be read or written at any one time um so this might Dave tested this out and uh it's possible um good night Dave thanks for hanging out um it's possible that like because we're doing something to the real-time counter we're unable to access like the external interrupt controller um ah if you need to look up what Scott's going over this is a nice guide to going along with the dive the advanced micro controller bus architecture I should copy that I'm not sure I've read it I hope I'm right I think this is really fascinating um because so much of what people think about in terms of computer performance is centered around um the CPU speed that like people don't learn that there's other ways that your computer can be constrained um and memory accessing is one of them and I think that's one of the reasons that like the uh like the newer Apple M1 is so much better than others is like it does a better job at managing memory than then which allows the CPU to run faster as well but like you really need to know the performance of a system um you really need to uh understand the whole system um not just the uh not just the CPU speed okay um let's just quickly go over this since we've done like half the diagram there's progress people are learning that SSDs are good yes SSDs are spectacular NVMe is awesome um I am like on board with keeping my I'm on board with keeping my computer up to date okay so bridge c so remember that like only you can only read or write memory or registers right registers are memory as well um you can only do that on one bus at a time so for every clock cycle I think it's every clock cycle you could only do one address on it and there's like some pipeline thing so there's like this idea of latency that they like there's the value to read and then the value that you read kind of like it's a couple clocks to go to and from things so there's a latency component in addition to how many things can be in like how many things you're actually reading at once anyway it is a memory buses are a constraint as well that's the one of the morals of the story um so bridge c has access controller again circums so circums here are the things that do i-squared c spy and art timer counters timer counter for control so these are just fancy counters um the ADC so this is the thing that sampling stuff the analog comparators the DAC that's output um peripheral touch controller the PTC and the inner sound controller i2s um so there's a couple other interesting things and one of them's written kind of like shown better than the other one is dma so there's this like tiny little gray boxes and so what this is indicating in the block diagram is not moving memory per se but what is indicating is signals about the memory state of these individual peripherals um so that's what this like thin line connected to dma is is what what these peripherals have so peripherals are are the name that i would use for these like different hardware blocks so these are these are i i think i'm sure i've said on deep dives before but i think of them kind of similar to an i-squared c device right like you have a microcontroller and on i-squared c bus and there's a temperature sensor on it like that temperature sensor is running separately of my microcontroller um but there's a way for them to communicate so i think of these peripherals in a very similar way uh except the way that they're communicating is over this apb bus um instead of over like i-squared c so there's a lot more wires and they're on the same chip but uh and they may be powered and clocked by the same system um but really they are independent things wagonload says can ram and microcontrollers be accessed similar to peak and poke and c64 basic um from c you certainly can and micro python has an api for it as well uh we've talked about adding a memory api to circuit python but we haven't added it yet um but there is very little memory protection in microcontrollers so there's no reason you couldn't yeah um and i guess what we should also say is that like the so the registers are registers are a name for memory that are kind of like the api for all of these peripherals and they're what's known as called memory maps so uh this high speed bus matrix will decide like everything on bridge c starts with these bits and then everything after that gets split across the things connected to it uh so there's and there's like memory yeah there's a memory we can find it here if this is interesting to folks um but yeah so what dma what these thin dma lines can do is say like hey dma like i just read like a uart byte and then what the dma can do because it can control the high speed bus matrix is it say oh now i need to go fetch this value from bridge c at this register and then then it will write it back to ram and it looks like dma has ram access separate from the high speed bus matrix as well um so dma is really cool dma is a way to get concurrency in a microcontroller uh completely off of the of completely without the cpu's uh it's all interesting dive deep uh yeah so dma is really cool it means that the cpu can say like okay we're gonna we're gonna transfer data to this thing or from this thing uh based on these signals go do it and let me know when you're done um so that's dma is really cool it's very powerful and it stands for direct memory access not dogs might attack like dexterous suggestion uh but yeah dma is really powerful um and that's one of the reasons it's important to think about uh memory buses as uh memory buses as um resources because if the dma is using bridge c that means the cpu won't be able to use bridge c at the same time um so there is some sharing that happens and especially on the sand d 21 like um um you could potentially really kind of bottleneck yourself across a particular memory bus um and then i guess we should also point out that these boxes on the outside this port so this is the the port peripheral is the thing that um derives the pins to the outside world so all of the signals on the inside of the chip are like really small and really um like low voltage things so those signals get transferred to the port peripheral which kind of lives on the outside of the chip generally i think um and it gets boosted up to like the signals uh that you would get from the outside world or vice versa and there's ways that the port peripheral like can choose between them um the last thing and i think the reason i brought this diagram up is this event system thing so um they don't show it at all but they're they're kind of too in the same way that there's these dma boxes back to this dma thing there really should be another thing that shows interrupts so there are also wires that go from all these peripherals for every interrupt and they go into uh the processor here and there's a piece of the processor called the nvic which is the something something interrupt controller so it's taking all these wires from the peripherals and it's able to say these are the ones i care about these are the ones i don't care about if one of the ones i care about uh goes high or whatever there's different ways of doing it but signals an interrupt occurred then it has a process to like switch the the cpu to something else um so that so there's interrupt lines from these peripherals that go into the nvic and the cpu um but then there's also this event system and this event system is not shown very well here um controller control e yeah maybe um the other thing i like to think about in terms of it can be like clock like the the thing that originates the clock although on the microcontroller like this it's not um not necessarily that um so the event system is is a lot like interrupts except the events go between peripherals not to the not to the cpu um and the event system itself has wire has interrupt wires that go into the cpu so um popping off the stack talking back about the the rtc so if you pull up the rtc um we can see there's a block diagram in the different mode so it does have counter modes and a clock mode but we're using it in counter mode and then it has interrupts oh nested vector interrupt controller so the nested the nested part means that um interrupts can have priorities and you can actually have multiple interrupts active at the same time um so I wanted to find was okay so here's the thing on interrupts so this says you can get interrupts so remember interrupts are signals sent back to the cpu saying like hey you should do something there's overflow compare alarm um alarm is the term if you're using clock mode and then sync ready um um so we're using this compare match to in counter mode to know if you sleep to know when to wake up and we were previously using that for the periodic ticks as well um but if we look at events here there's overflow which uh there's an interrupt for as well we use that we use that because the number can run out because we're clocking it pretty fast so we keep track of how many times we overflowed as well it's basically another counter extending like the resolution uh but there's also this period this periodic event that can be out by the RTC and uh what we would really like is we would like to get the periodic thing as a an interrupt because what we want to do is we want to get an interrupt every 120 1 1024th of a second uh to call supervisor tick which counts down until we auto reload and a couple of the things I could do um so in the cmd 51 can time trigger an interrupt right so so if your time is a fixed point in the future like I'm going to sleep for 10 minutes I'm going to sleep for 2 minutes then we set the comparison register and then that will generate an interrupt but if we want to do this like every so often periodically we want to wake up thing on the 51 the periodic register or the periodic interrupt can interrupt but on the 21 that's not the case however if there's an interrupt we can convert interrupts into we can convert interrupts into interrupts wait we can convert events into interrupts so this event system all of these wires end up from all the peripherals go into the uh into the event system peripheral those are known as generators and then there's the opposite side which is users so users take in event signals and do stuff based on that um but we're not actually going to use that second half if we look at here you can see peripheral generates event events and then they get specific channels and directed to users haha so how long does the interrupt take to fire and load into state and what's the resolution of it so how long an interrupt takes to fire is dictated by the CPU I think that it's about I don't know on the order of tens of cycles so the at least the Cortex M0 what it's got to do is it has to save all the register state kind of like a C stack to memory which takes time and then it jumps to uh and then it reads the interrupt table and then it jumps to that portion of it um so the periodic one we're doing is about once a millisecond it's like slightly faster than once a millisecond um so it's not a problem right like if it's a 48 megahertz clock and we're doing it once a millisecond which is like a thousand times we have 48 thousand cycles per time um to do it and we don't run the tick all the time either um so yeah it's comfortably slow yep but it's super cool like um this is not the sort of stuff you think about if you're doing like web development I enjoy like this is the machinery of the CPU that I like as I said in the reddit like I really like being close to the machinery of it um okay so one thing they don't tell you here in this diagram is that not only can the event system kind of create these things but it can actually do interrupts as well so let's see do they have another section so there's different ways yeah here we go so there's different ways of doing it um there's edge detection re-sync async and the async path is selected the channel generate interrupts so it basically just passes the signal through um mark asks is that a cpu design naming scheme or is this the circuit python vernacular generators versus users creating and submitting events or interrupts I know the term generators from programming language design so generators and users is terminology from atmel for the cmd event system um for events specifically um the interrupts stuff that the envic is something you're going to see across all like the cortex m stuff and maybe since we've gone through this maybe since we've gone through this block diagram maybe I'll pull up like the imx and blow some minds because it's a lot more complicated um but maybe we yeah if if folks like the block diagram say so and we could take a look at some other ones um this story is almost done so the the event system can generate its own interrupts um so it has overrun channel which it basically means that um multiple events came in without the user consuming them um and then there's also this event detected so this is a way to convert an event into an interrupt uh which is awesome and exactly what we want um so that's what this PR is doing um the build's still broken so I've got to fix the build some more but Dave did test this and said that um there's still an issue with the DHT11 on it um which I'll probably have to break one out next week and figure out how to test it um because we might actually be having like memory bus contention issues or something um that's causing this I don't know like writing the register can take some time like we have to re-synchronize but like we should be interruptable so yeah we're not we haven't quite figured it out I guess let me show you the code so changes here so here's like endvic control so there's kind of two places and I don't know if I've covered this but there's kind of two knobs two places you can control interrupts there's does the peripheral send the interrupt out of the peripheral and then there's also does the CPU pay attention to the interrupt that's coming in um so the endvic here is uh the CPU side of the interrupt line and then so we were previously doing this trick with audio dma on the samd 21 so that's why there's some event system stuff here is there's like an event signal from the dma peripheral that we want to use as an interrupt um so we were doing that trick and now I'm moving it out into um port and I'm removing this as the like end end start start end pull stuff that um was the original fix um because like there's this read continuous mode um oh yeah so that's the other piece I should talk about I guess so a lot of these fixes that they were doing is because because we use so this clock this real-time clock is now the source for like time monotonic so anytime we do a call that says hey get me the current time um we would have to go through this process that was really really slow um because the clock and its circuitry is only running at like 30 32 kilohertz or something like that um there's this process to say hey I'd like to read I'd like to read your counter could you synchronize it for me and then once you do that it then will tell you when it's live but the problem is that that takes a long time um relative to how fast the CPU is going and so that was the original theory with why the pulse in stuff was not working and I thought that this change would fix that because what David done in this original start pulse thing was to set continuous mode and the nice thing about continuous mode is that um you can tell the um and the cat wants to um what you can do is you say hey read some clock I'm going to read you a lot just always stay keep always keep this register I can read updated um and so what I was trying to do in this PR is just like you know we're going to read that a lot why don't we just have it always continually update and then we don't have to do synchronization um so that's what I was doing is just I set it continuous but setting it continuous is a little bit of a process um because you have to synchronize you synchronize before you set it to continuous then you do a request you you wait for that request as well um which is pain and it takes like he he said it took like a hundred microseconds or something like it's not fast um so here's how I turn the event on and after I enable it I have to set this continuous mode and then in get count here you can see I removed the read request synchronization stuff so in theory get count should actually work faster now um which actually could be like a non-trivial performance game we'll see we'll see people are like oh same d21's faster now um um anything that involves reading that clock will be faster um so I moved the interrupt after tick stuff into the same place again um I got rid of so this is in the the RTC handler is the interrupt thing that's called in the RTC so that's the other thing about interrupts on the on these strips is that it might so I said there was places that they're controlled there's the peripheral side and the CPU side but um the number that can go into the CPU is limited especially on M0's so what some peripherals will do is they will say I'm only going to run one wire from myself to the CPU but I'm going to have three different things internally to myself that could cause it and so what you'll see usually is that the um the interrupt happens and then the first thing it does is it checks this int flag and so this is like asking the the real-time counter like oh which which thing triggered your interrupt and then you can clear that individual thing um and this is how ticks used to work we just say like oh let's set the next tick um but that's no longer the case and now we move the event handler here so we say if tick event is set and the interrupt is active then we'll do tick otherwise we'll do the event audio stuff and if we're starting ticks and we haven't allocated a channel so if it's equal to the like plus one of the valid numbers we'll turn on the event system we'll find a channel we'll set it to interrupt from this peer to register and then we'll disable the overflow one because we don't care I used an NTP to set a RTC and then compared the RTC to NTP 24 hours later and found out the RTC was gaining 7 seconds a day yeah that can happen um it depends on the quality of the clock source into the real-time clock or counter um good real-time counters and stuff will actually allow you to like shift or skew the count one way or another so you can say like every thousand counts drop one or add one and that's how you can like tune it to be a little bit better um the sandy 21s if you don't have an external clock are terrible um really terrible um but the 32k crystals are reasonably good hmm so mark says I've seen this a few times now and I don't really understand it the volatile keyword means it's going to actually check the memory location value and not start getting a register correct so the compiler could say for a non-volatile value it can know I read the value and I'm the only one that can change it so I don't need to like pay attention to anything else um whereas if uh if you're um if you're volatile that means that the memory could change out from under you so anytime you want to know the value you've got to re-read it um which is kind of what you're kind of what you're saying yeah thanks minnesota mentat I will have a good weekend definitely like going to be done in 20 minutes because I've been talking a lot okay so new interrupt after ticks which is just I moved all the code back down here um we're setting the comparison register to the next value we're synchronizing it and then we're setting the flags and we have to reset continuous mode because it clears the flag in the first place which is annoying and then we do a check to make sure that we didn't like take so long that we should have woken up um yeah so uh our folks interested in seeing another diagram was that cool linux 203 points out um temperature can affect real-time clocks and some compensate for temperature dcd cisher okay so we do samd 51 um yeah bruce bruce says yes tell a compiler that it can be changed outside the compiler's control so don't optimize it out of existence let's do 51 first imx is really complicated 51 would be a better stepping stone or rp2040 let's see if they have a good diagram so here's the so the diagrams do vary quite a lot so here's our rp2040 um the diagrams vary with the vendor but what we can see here is we can see interrupts are shared two processors proc 0 proc 1 sio which is the single instruction io stuff so that's shared between the two processors although I think the model is that you have two different ones um that goes out to the ios so this is like the equivalent of the port directory um the or the port directory the port peripheral on the samd 21 um and you can see here that they've called out the fact that they're like gpio treated treated differently from q spy swd which is debugging and the crystal stuff so crystal signals are analog signals usually um and you can see clock generation um and then they have this bus fact fabric box this is going to be the equivalent of that like uh high speed bus and dma hangs off that along with usb and then memories off there and you can see how they split the like different s ram banks and that's interesting because s ram s ram has the same constraint where you can only access um particular banks of memory kind of all at the same time um rom so the rom here is where the like faster mass stuff is and um the u of two boot litter will all live in this rom this is read only memory the xip slash cache this is uh memory that is used so this is starting to get to this in the same vein of memory buses being a restricted resource um something you learn in computer science is uh that the kind of further away from the cpu the memory is like it's orders of magnitude slower um so this processor here is running at 120 megahertz or 125 megahertz but this q spy is four bits at a time running at 60 megahertz tops so you're taking like it's just math right and what is the math like four bits 60 megahertz which means you're doing 30 you're doing one byte at 30 megahertz but you're loading four bytes at a time so you're doing like eight is it eight yeah it's like under just under eight megahertz of words so word is like um 32 bits so if your processor is trying to run one 32-bit instruction or or if it's trying to run a 16-bit instruction which our ARM Cortex M2s can do it's actually when it's taking it can only load that at like 16 megahertz which is outrageous um so what they do is instead is they have this cache in front which means that if it's memory that the processor is already read once it doesn't have to actually go out to the q spy chip that's like somewhere here off the diagram that is like relatively so slow to read um and this is why in circuit Python we have a way to designate stuff that can end up in SRAM so like functions we know we're going to use we can actually spend some SRAM to put it in SRAM and therefore this cache can have more space for stuff that is used dynamically okay so um that's the basics of caching there's pio here uh 0 and 1 and then that goes out to GPIO um oh interesting it says that is why those old mainframes were built in a circle so the bus was as short as possible yeah so like in mainframe days the clock speeds depend on wire length and that's still I guess that's still true today the longer a wire is the slower it goes generally um okay so then we kind of lastly just have this peripheral bus so this this must imply that like whereas the same one had multiple kind of peripheral buses my guess is that this means that they're all on the same peripheral bus and so this is all your spy you are all that stuff is here um so that's the rp2040s diagram it's relatively simple um similar to the similar to the samd21 I mean the diagrams not quite as detailed but like I think they would be more similar if we looked at it in the same way um so let's go on to the 51 and generally what you'll see is they get more complicated as um the kind of like the cortex m0s cortex m0s like the most basic cortex m but then like m4 is going to be more complicated and then after that like the m7s like the imx's are going to be even more complicated handling the rtc it's kind of like a leap year adjustment bruce says that's why high speed traders put their data centers next to the stock exchange close to the stock exchange speed of light signals are still faster if they go a shorter distance um okay so here's the samd51 data sheet and look at that diagram that's a lot more complicated than the cortex or than the samd21 so what do we have here we have the starting from where we started before we have the cortex m4 the mpu and the fpu so the mpu is the memory protection unit uh fpu is the floating point unit this does your floating point math cystic counter is marked here and then core site so this is for like low level debugging nvm again so this is flash again but this time there's a cache in front of it which samd21 also had i don't remember dma controller is up here now and then sram and then it also has this ram ecc block which is if you wanted to error correct it uh high speed bus again and serial while and then the device service unit um and then where do we want to start so bridge a so remember these all share memory access times and now there's orange lines for the event system which makes that clear um and there's a whole lot of let's not go through all the peripherals otherwise we'll take all day but this looks very similar although note that there are circums and timers on this peripheral a um oh and you know one thing um yeah there's a separate reason so the samd51 was cool because you could you can dma to the port peripheral which you could not do on the samd21 um which is kind of a neat trick um and i don't know how that's shown in this diagram it might be in a separate place oh here we go there's this port designation here um to the bridge b bus so that means that you can because dma can access bridge b um you can actually dma straight to the port which is cool uh about the 51 uh okay bridge d again um pretty straightforward like a bunch of peripherals you can see the dma boxes are here again um so there's bridge a bridge d um um direct access to quad spy so that's your q spy controller that's your flash chip integrity check monitor which we don't use but i think it's a security thing um public key cryptography which we don't use um circums again counters again see they get kind of like move different places um and then bridge c is intermixed next to this other green one bridge c is uh ethernet can uh ccl is something we don't really use but it's um like glue logic stuff almost almost like uh almost like an fpga but very very very basic um yeah so there's like what four buses here um car controller usb is up here off actually it's on one of these buses but has direct sram access um any questions about this diagram i think it's it's a good place to start because it's very similar let me see if i have pramx already downloaded i'm just looking in my in my downloads folder so let's do the 1010 you'll see that a lot of these um memory primitives are the same because they're arm systems so when people license thing from arm they get the cpu core but they also may get the memory stuff as well um arm does more than just the cpu love the factory says can you use dma between neopixels and ice grid see you on any circuit python targets um so neopixels are weird because their timing is weird um in order to use dma you need a peripheral that can do the weird timing um and the pio on the on the rp2040 can do that weird timing so you could probably you could probably dma ice grid see the neopixels on the on the rp2040 um it would be weird um but you could probably do it but generally like the samd21 stuff we actually use the cpu to do the timing for neopixels um so you can't um like it's that there's not usually peripherals that do the timing of of uh neopixels directly and then if they can so one trick that is pretty common is that you can produce the timing for neopixels by exploding their bits out so you can do a spy output but the data coming in is like a one would be one byte of like a particular length um so you would uh yeah in order to not have to explode the data into larger to use it as is you need a you need a peripheral that can do the timing right like a pio can so the pios are really cool i i hoped we see more things like that and more microcontrollers in the future um yeah pios are really cool for sure um hmm euthanite can put it packets directly in s ram there that's nice i guess it sends a signal when it has a full packet um are these microcontrollers optimized for bus length between cpu and ram or are there other items that have a higher priority for bus speed i i i think that generally everything on this high speed bus matrix has uh kind of equal access um although what we can see here is that there are some things that have access to ram outside of that um so i guess i guess the answer would be other things can take priority um and the s ram controller will actually decide who gets access who can actually prioritize it and i think it's actually configurable as well um there is um so here's a memory mapping overview so this is if you ever wanted to know like what the actual address of something is you can look at like this memory mapping overview and then somewhere there is a chart about how they're connected i don't know where it is and i'm past it anything no pinouts no so there is also a chart usually in these data at least the sandy data sheets that have like all the things that control the high speed bus and all the things that can be read via the high speed bus and like not all of them are necessarily connected together um which is why like the dma on the sandy 21 could not access itself um that's like all configures and stuff do you want to go out yeah nwk says cypress psoc tips have a lot of peripherals that are connected to the dma and the digital programmable logic ray is also connected to the dma controller um and the gpi is connected to the programmable logic so you could do something similar to what pio does is there a point i think um okay okay i like these diagrams let me pull up this one oh there it is black diagram oh it's not i should pull up an stm one so there i remember seeing one that was yeah spooks ready for the weekend um the quality of these diagrams varies a lot and i should find the stm ones because the stm ones are good too but there was actually one that showed the different speeds that different buses run at which i thought was really interesting um this one it's not that one i was thinking of um yeah so this is a lot less interesting that's the simplified block diagram this is the reference manual here's the memory map i'm just seeing if i can dma max form this is kind of cool this is just the cpu block so you can see the cortex m7 the itm is an instruction trace ppb i don't know what that is dpu might be um breakpoints npu is memory protection fpu is floating point icash and dcash so these are cases where the cpu is so fast that even the memory buses are slower um let's see romtable tpiu is like a fancier trace debugging stuff so it can actually spit out everything it executes um ahb so these are um the high speed buses and then the axi bus is even faster i think um and then flexram is the imxrt's allow you to kind of decide how to use ram banks um itcm is uh instruction memory tcm is tightly coupled memory so you can do instruction memory so the m7 is a harvard von neumann architecture i don't remember what there's like two classic ways cpu architectures one is where data and instructions are loaded the same way and then the other is that data and instructions are loaded separately so that's why you see here that there's a instruction coupled memory and then d tcm is data tightly coupled memory and then oc ram is like ram that's generally available but is run slower that's why the arrow kind of goes back out for axi this is where this all sounds familiar but i don't know i don't know this datasheet as well as i do the other ones um i think that maybe they have like a you know like a list of charts it might have been an stchip that i was thinking of do i have that in my i don't know if i have those on this computer but oh here we go here's a datasheet for an l7 let's see what this has we're at h7 h7 is kind of a top of the line uh top of the line stm oh yeah look at that it's going to be kind of hard to read okay where are we at let's start from the cpu like we did last time so this is a stm 32 h7 b0 xb um um let's start from the cpu and work our way out so here we have an arm cpu cortex m7 um oh mark asks i saw efuse on there like the thing that playstation 3 had to stop people from flashing their firmware backwards yes i think so i don't have a hard stop this will be the last thing i talk about if you do david um itcm so again this is the instruction memory d2 dtcms uh jtag debugging trace debugging again uh there's a ahb here that has access to this ahb bus metrics yeah harvard architecture versus von neumann i couldn't remember which one switch um mdma so this is like a fancier dma um chrome art which is 2d dma so this is usually used really useful for graphics lcd tft this is the thing that can control like a 24 bit rgb thing on its own um wwdg i don't know what that is it might be watchdog but i could be wrong jpeg decoder then there's a 64 bit axi bus matrix um and this is only running at 280 megahertz but i think that um there are other h7s that run faster and so you'll see like this one is fast but then and this bus is fast but then like as you go outwards that slows down um so let's follow it so here's flash and ram um octo spy so instead of q spy you could actually do eight lines um and then here's a 32 bit bus matrix so even though this is this is 280 megahertz but 64 bits at a time greg says window watchdog thank you greg and hello um and then so this 64 bit bus can talk to it 32 bit bus so you basically get half the data right it's clocked the same but it's half the data um you have two dma's here so we had a fancy two fancy dma's at the axi bus level and then two less fancy ones here um our usb is here so that's what otg is um stcard and then this there's more ram so it's interesting so this axi ram will be faster than this ahb ram um i'd like to use pio for audio output rather than dma pio you could do for depends on what you mean by audio output you can do i2s really easily um okay so there's a bus here 280 mega hertz to 80c stuff um and then here's you see like a bit ahb slash apb bridge and it you probably can't read this this says apb1 and then it says 140 mega hertz so we're down so we had a 200 280 megahertz 64 bit bus it's the fastest fastest bus matrix that gets then 280 megahertz 32 bits and then that 32 bits at 280 then talks to this thing that is only 140 32 bit and that's where you get all your timers your uarts your spy kind of all this stuff that's really slow hdmi csc uh that stuff's all slow and then we have another hb apb bridge here that is again going to 140 um and this is more spy and uart and timers again we have um interesting here from this axi hb bus we actually have another 32 hb bus down here um dap is like the thing that does debugging oh and there's even an sp ram there's there's or s ram there's 10 kilobytes of s ram on this like slow bus here um there's a dma mux there's another dma down here more s ram uh including the backup s ram so that's the one that's usually low power so that's still at the 200 mega 280 megahertz but i think as your latency will get longer as well as you get further away here's your dpio from here you've got power control and clock control which doesn't need to be very fast and then one last ahb apb bridge to another 140 megahertz bridge and this is where you got your security stuff and your low power timer um uarts and voltage reference and this is clocks down here as well and then supply supervision so one thing um that this would be cool to have in these diagrams as well as like clock domains um so all of the like clocks are the thing that synchronize the signals and to save power you basically turn off clocks you don't clock things because power is consumed when you change transistors um or even just have them powered up so uh typically along these memory bus boundaries you'll have clocks and power switches as well so that you don't have to have this part of the chip on at all and that will save memory so these diagrams are cool microcontrollers are cool um so yeah that's uh oh yeah and there's an open mv h7 cam yeah these cool these chips are really powerful very cool all right I am toast stick of work to me hopefully that was a good circuit by the day stream check out the reddit if you haven't already if you have more questions you can check out the reddit today um we do have the circuit by the sub reddit if you have questions as well um but the the primary place to go is to look at to join our discord server which you can do by going to avafru.it slash discord um hop on there we're in the chat um circuit python dash dev channel is kind of like if you're getting circuit python building you're adding a new board that sort of stuff that's the place to go um if you have questions about this low level stuff that's a great place as well um if you want to support me please support Adafruit by going to adafruit.com purchasing hardware hopefully the thing you want is in stock but it's been quite the trick lately so uh hopefully there's things there that interest you uh if you don't know what kind of what direction you want to go I always suggest going to learn.adafruit.com starting there um we should be uh okay so Keith says we're going to keep the AMA pinned through tomorrow the answers there are amazing if you want something interesting to read yeah and Jim from MicroPython who we had a few weeks ago on the stream um has got really thorough answers to the AMA as well so that's cool um thanks to Jim for doing that um I think that's all my housekeeping thank you to DCD again for taking timecodes and I hope you have a good vacation if folks want to volunteer to do it next week please let me know and we'll figure out how to do it um and the link to the AMA I've got it right here nope and Keith Keith beat me to it I'll just leave this up because the cat left the room he left me hanging and so we won't end with the cat pet but uh thank you all please support Adafruit try CircuitPython if you haven't uh and uh have a great weekend we'll see you next week on Friday and on the Discord of course