 Recompiling FFMpeg got me somewhere. I was in the middle of compiling OBS realizing, wait, I don't think I need to do that. I should have the old version still. And it started up. So, I waited a little longer than I actually needed to. Thanks for sitting tight, sorry I'm starting late. This video is smooth again. This camera here doesn't work. Yeah, everything's going well. Oh dear. I don't think we actually need this other camera. You know, I just, I'm wishing I had started, tried to start OBS a little sooner. That's what I'm wishing. Well, looks like this camera is going slow. You wanted to see my leg. Hi Michael. Hi, unexpected maker. Hello, David, Randall, Beata. Hopefully you all are still here. And hopefully the background noise isn't too bad. All this compiling has kind of bolstered my, bolstered the temperature of my room. So, I'm just going to leave it on. I think I can turn it down though actually, to be a little bit better. All right, is it going? I see we have some more folks on YouTube, but I haven't gotten hellos. But it looks like it's okay. I'm just moving my windows around a little bit. Sorry, I, like, I don't have my act together. Can you all hear me? 26 minutes late is too long. Oops. Hi. Add me. David says, glad you were able to get it going. Audio good. Sounds good. Okay, thanks. I was a little worried. I did have to, like, change my audio settings too before I started too. Sounded good. Okay, great. Michael says, so I'm kind of a new subscriber. This channel has multiple creators on it. BGS says loud and clear. Awesome. Thank you. Okay, so let's do intro and we'll answer Michael's question at the same time. So, hello everyone. Thank you for waiting. I'm supposed to start at two and I tried to start OBS and nothing happens. So I had to fix that. My name is Scott. I work for Adafruit, which is the channel. So the channel is for Adafruit. They're an open source hardware company based out of New York City. They do open source software as well. I do the software side mostly. And I work remotely for Adafruit. And I'm the lead developer on Circuit Python, which is a version of Python geared towards beginners on microcontrollers. Microcontrollers are these little inexpensive computer, like self-contained computers. So RAM, CPU and peripherals and stuff, all on a single chip. Makes them really inexpensive to get into and you can use them in all different places to do all sorts of like interfacing with the real world. It's kind of in the realm of like physical computing that people do it to. So I'm just one of the folks that works for Adafruit. And so the folks that end up on this channel are different, do different things. So this is deep dive. Deep dive is when I tend to stream, I tend to stream two to four, like two to four Pacific time. So about two hours, no label on this. This is the IMX 1011. This is the like trace board that I made. And yeah, I haven't put a label on it. And during deep dive, I love to answer questions. So if you have questions about Circuit Python, microcontrollers, that sort of thing, please let me know. Adafruit questions are great too. And yeah, so I tend to cover very technical topics. I am working kind of in the core of Circuit Python, which is actually mostly, it's all C code. So I'm usually in the C code land that underpins the Python side of Circuit Python. So yeah, I tend to just talk about what I've been working on and answer questions regarding that or other stuff as the, and see where the wins take us. So do it, folks have any questions as we get going here? Otherwise, I'll basically pick up where I left off in terms of what I was working on. So it's been two weeks since I've done deep dive. Last week, neither foamy guy or I did it. The plan is that next week I'm going to be at my parents' house. So I won't be streaming next week, but foamy guy is available. So foamy guy will be streaming next week in the spot. And for those of you who haven't watched foamy guys that tends to do higher level library Python stuff, which is probably the most valuable part of Circuit Python is the libraries, because those are things that support all the different sensors and things. So check deep dive out next week for foamy guy. And I'll probably be back the week after that. Yeah, this is like cat cam that's like real slow. I don't know why it's, I don't know why it's working. You had Michael's question. I think I answered it. Yeah, so this channel has multiple creators on it. That is the case. Adafruit pays multiple people to make content on this channel. Yeah, so I won't be streaming next week. And then I can just talk about kind of what I've been up to the last couple of weeks. Where we left off on deep dive is I was working on, I think I had finished the IDF 5.1 update. So the ESP IDF is the vendor provided code to interface with the expressive Wi-Fi chips. And we're updating the version of that in Circuit Python so that we could support new chips. And I know unexpected maker is going to be interested in this. So that stuff's merged in. Adafruit or the Circuit Python main branch is now IDF 5.1. And then the next thing that I did after that was I wanted to re-enable the RGB matrix. This is something that MicroDev had pointed out. MicroDev did the bulk of the work for the IDF 5.0 update. And he had pointed out that in order to get it going quickly, he had disabled the RGB matrix support. And you can kind of see in this cat cam that this is an RGB matrix on my desk here from when I was testing that. In fact, I think I was actually trying to get it working on the stream. I don't remember. But I found this issue. Unexpected Maker, you'll be interested in IDF 5.1 status is what I was thinking. So I found this issue, an RGB matrix, and people had filed a bug about this as well. Where after a little while of using it and iterating on your code, it hard faults. It crashes into safe mode. And I did it. Let's switch to the desktop. Yeah, so it crashes into safe mode. And I figured out what the issue was. No RGB literally unplayable. Yeah, so it's not enabled yet. There's a pull request for RGB matrix stuff. I pretty much moved on, although maybe it looks like Dan actually looks like Dan got to it. I was going to say it hasn't been merged in yet. The invalid pin issue and circuit by the nine alpha. I think I saw the issue go by but I haven't looked at it yet. Yeah, Dan Dan just merged it 20 minutes ago. So RGB matrix is enabled in main as of 20 minutes ago, which I didn't even know I was too busy trying to get the stream going to see that. So thanks to Dan for that review. And I also had to update the ESP protocols stuff. So Dan merged that as well. So there was two pieces to this puzzle. Let's just take a gander at these two PRs because I did one PR and actually we should probably do an eight to X merge into in domain as well because these there was the crash itself the safe mode crash is something that was present in a circuit by the eight as well. So I fixed it there but then I had additional fixes that went with the nine point updates. So one simple thing I fixed an eight is that there's a list of the address pins and eight but that it wasn't updated for revision. So I fixed that protecting against a little pointer here. Get this as big as I can. And hopefully my my lighting is a little better. I actually got a second key light. So I've got two key lights now. I should be a little less shadowy although you can see that I still have like this window giving me a lot of light as well. Yeah I'd expect to make or go ahead and post a link to the issue and maybe we can take a look at it right now. That's totally totally something I am game to look at. So so this bug here is that it was using this allocator impulse stuff. So you what you can see here in the new world is there's allocate memory and free memory. And those are those are two I'm trying to my brain's thinking is that right. Yeah I think that's right. I should maybe I should test it again. But so there there was this there's this problem or this this trickiness with RGB matrix because we share the proto matter library with Arduino. And it requires a few it does a couple allocations itself which is done through this like pound defined for malloc and free. And so there's this tricky problem where so this free impulse and this allocator impulse get passed down in the proto matter for doing allocations. But the original version when it freed it would allocate memory to the supervisor stream paused. Looks OK from. Oh the problem was in certified on dev restream is giving me a need help button. I was just marking it as fair. Yeah. Yeah. Let me know it looks OK from restream side but I might be trying to upload too much. So originally the allocator implementation would say allocate memory and then return it if it managed to do it correctly. And then the free it had to do this call from allocation from pointer. So in circuit Python 8 and in main right now there's this make mechanism for doing allocations where they get the memory gets allocated into the micro Python heap. And then after micro Python VM stops it moves all that memory to the start of where the heap was so that it can outlast so it lives longer. But but moving memory is really complicated and like proto matter it doesn't expect that to happen. And in fact this code wasn't expecting it to happen either because this allocation from pointer doesn't work if the pointers been moved. So the problem was that it was leaking. This code is leaking these allocations and the supervisor memory manager thing had a fixed number of allocations it could do. And then after that it wouldn't it just wouldn't be able to allocate anymore and cause problems. So the the main cause of the crashes that were leaking allocations supervisor allocations and then we're running out and crashing. Now there's a separate problem where so so that's caused by it's doing an allocation this proto matter RGB matrix code only holds on to the pointer when it really needs to hold on to the outer object. Which is the allocation itself that the underlying supervisor memory allocator gives you because that outer thing doesn't move but the pointer stored in it does change. And so the problem here with this way of like I only store the pointer and then later when I need to free it I find the allocation from the pointer this doesn't crash or give you a warning if you don't actually find anything. So it acts OK like it did successfully free button in reality it didn't free anything and therefore it's leaking supervisor allocations. So that was the main thing that I was fixing here so now you can see that I add this basically like cash of allocations so I keep track of like OK I allocated something with this pointer and here's the allocation for it. And then when free gets called I looked up I look up in that cash and then free it properly. That works to prevent leaking allocations but it doesn't actually it doesn't actually it doesn't actually fix a like the problem that memories moving. So like if Proto matter tries to use a buffer kind of like and the way this code tries to get around it is it like tears Proto matter down and starts it back up. And yeah so there's a period of times that still exists after this change where like Proto matter could use memory it really shouldn't because it's been moved. Todd has a question in YouTube that says I need a short little two wire with male slash female connectors to embed in a 3D printed chain link. Anybody able to assist in the engineering. I would just do like a JST plug. But I have not I don't I don't know 3D printing that well so I would take a look at like 3D printing on 3D Hangouts on Wednesday morning is a good place to ask about that. But hopefully other people here will will chime in if they have offering to pay to I don't have any experience about that. Yeah the eight different discord so there's two chats above me here. The middle one is the discord so you can you can go to adafru.it slash discord to join it. So you can ask there as well. There's a 3D printing channel. I'd expect to make her says I have to say whoever thought of the name Proto matter should get a raise. So if you go to the Proto matter repo they cite where they got it from. It says I used Proto matter in the Genesis matrix David Marcus on Star Trek 3. So it's a Star Trek reference which I did not know until I actually read the read me here. But yeah it's a it's a Star Trek reference. Okay so that's the changes that I did an eight and the goal is to just like prevent prevent it from crashing or reduce the likelihood that it crashes. And then I had additional fixes in the idea five main main update stuff. So I had to update Proto matter because it didn't compile with five one so I had to do some changes there. There's a new code that Jeff had added for using queue strings as compression tokens basically. So we have like all these translated strings and we also have queue strings and he added the ability to find like a sub string in a translation and map it to a queue string as a way of compressing it. The problem was that that means that we're using the queue queue string pool stuff outside of the VM. If you have like an exception or other message that that you need to decompress they need to decompress when the VMs aren't running. So I had to add the ability to basically reset the queue string stuff once the VM was done. Otherwise the queue string stuff has a like linked list of tables for queue string lookup and it adds dynamic ones to the end. And that's where you start and then you work your way back. So after the VM you're looking at like this one that's in the micro pipeline heap. And if you do that after the VM is gone you could just find all sorts of garbage. So you have to actually reset it to the static queue string pools. Mike Jones says accessing memory when something shouldn't because it's not there anymore. The challenges of managing dynamic memory. Yes. And maybe that's maybe this is a better segue that I should talk about the future the future. Let me just go through this. So this is turning it back on queue string reset. I'll get back to that in a moment because yes it's it's difficult. No FPS limit. Oh so in this it turns out that frame buffer display did not have like the more modern refresh function. I was like really frustrated with it and I was like didn't we fix this and yes we did but we did on display not not frame buffer display. So with circuit by the nine refresh on frame buffer display will work like the other displays refresh which is basically a refresh is when you call it unless you give it these target frame rates which was like something I added originally and then we're like this is like really hard to use. So now it should work as you expect and it will work like display so I changed that. And that was really good because it was like driving me a little bonkers. Like I had this animation actually I can turn it on and maybe you can. I don't know what this how well this is working. It's still like really slow and it looks like this is the hover cam here but if I switch to it it doesn't work. I can show you this animation turn it off and back on again. I did find my LED plastic that makes it look really good when it's dusty. Now my jeans are dusty but I had an I made an animation that did dots in both directions on the display as a way to like debug the mapping. Hey there we go. Let's lock. I think it should just come up. Hacktoberfest shirt. So I have too many things on my desk as always. I have more space and I still have too many things. Oh dear. That's the USB hub that just fell. This is the live TV equivalent of things. Okay so we're still pointing too far down. Let's take a look at this matrix. We are upside down. All right let's plug this in and see what it's doing. I think it might be mismatched and it's just not working. Yeah anyway let's talk about the memory stuff that's more interesting. Yeah the refresh is getting updated which is good. That's most of what's the changes in here are along with this Q string reset stuff and turning it back on and then that combines with the other fixes to that. So yes let's get back to... Beata says I recall some of the memory talk way way back when Max for an even fully 32 bit. I think they had similar sorts of problems. Yeah so I spent a lot of time last week thinking oh what is the right solution to this? I talked about this fix where I'm giving memory to protomatter but then I'm moving it and I was googling on my phone during a lunch being like oh how do I... What's the right way to do this? How do embedded projects manage kind of compacting memory? So like it's a class of garbage collector where at a point where it does a collection it will compact things as well. But if you move pointers you have to have this layer of indirection or you have to know all of the places to update and it's just a huge huge pain. So I was like looking and looking and looking trying to figure out how people do this. And then I realized we probably should probably should just not do that. We probably shouldn't do that. And we've been talking a little bit about this feature that's in MicroPython 1.21 which just came out yesterday. And let me find it or actually let's just pull up the release notes. I will show my desktop. So released yesterday MicroPython 1.21 which is awesome congrats to them. But the thing that they added in 1.21 that's very interesting to me is this MicroPy GC split heap auto. So in 1.20 they add the ability of doing split heaps which is the current way that it works in CircuitPython and it works on some ports in MicroPython this way. But basically right before you start the VM you say what is the biggest amount of memory that I can get is one big chunk and that's going to be my MicroPython VM heap. The problem with that is that if you are doing other allocations you may have like a small region or there's some microcontrollers where you have like multiple memory regions. So like an ESP has both the internal RAM and the external PS RAM. And in CircuitPython we just designate external PS RAM basically to CircuitPython and that's our big heap chunk. But we've started to introduce this like reserved heap stuff. So like in CircuitPython you could say like only take or like leave a set aside some other chunk of PS RAM to use in IDF's memory allocations. And that's great for cameras or other other like drivers that need it SSL stuff can go there too. Which is yeah so especially on the ESP you have this problem or this challenge where not only do you have a MicroPython VM but you also have that's doing memory management. But you also have this outer real-time operating system that has managed memory or dynamic memory in and of itself. So in MicroPython 121 they've taken this split heap idea this split heap support and basically added the ability to add additional memory to the MicroPython heap as you need it. So instead of getting one big chunk or two big chunks that you do up front what they do in 121 on ESP is you start. They start with a 64k chunk and then yeah the MicroPython heap has been reworked on this port to support the large variety of memory configurations. It now starts at 64 kilobytes and automatically grows as needed. And the way that it grows is that it asks the outer heap that the ESP IDF's heap hey give me another chunk that I can allocate MicroPython stuff into. And it a consequence is that you only need to manage as much memory as you're using. So this means that boards with spy ram have much faster GC collection times if only using a small amount of RAM. But if you want to use a lot from Python you can but also correspondingly now the IDF can use memory as well. So this is really cool because it allows for better sharing of memory in Python and outside of Python. And if we go back to what the challenge with ProtoMatter was is that particularly for displays like RGB matrix displays circuit Python does these really tricky tricks so that the display stuff stays alive even after the VM is gone. That's why we have this like move memory sort of stuff which is as I said really hard to maintain. So I got really excited and basically said like why don't I've decided and I kind of like worked on it some is that in circuit Python 9 we'll do what MicroPython is doing on ESP 32 but we'll do it on all of our ports. Because all of our ports may need this ability to allocate memory that lives longer that lives longer than the MicroPython VM. So by having a split heap that gets chunks allocated as you need it as you need you probably have space outside of that for any allocations you need to do that should live longer than the MicroPython heap. That's the theory. I've like half implemented it I ripped out all of the supervisor memory stuff. But because we're not updated date with MicroPython 121 yet I can't really like piece it all together for nine yet. Mike says I just got into MCUs no expert and was reading it to avoid dynamic memory in MCU programming because of the headaches involved. But I guess it's unavailable if you want to use things like MicroPython. Yeah so I think there's actually a lot of while there's potentially some really good room for optimization like thinking about tiny USB a lot of our static RAM allocation is buffers for USB. But imagine that you have user code that turns off a portion. So like there's a there's a MIDI buffer or a mass storage buffer. Right now tiny USB is all statically allocated. So if you want to maybe use mass storage there's going to be a 512 byte buffer that is just constantly always in your in your RAM even if you've turned it off and you're not using it. So by moving to the system where we have dynamic memory that's properly managed outside of the Python VM we can be much more it's much simpler for us to just like put stuff in there. So yeah. Circuit Python in general is much more dynamic than a lot of C programs because we we don't know what user code is going to do. So we have to kind of like prepare for all cases. We have to be very we have to be a lot more dynamic than even a lot of vendor SDKs expect us to be. So I started that branch and we can find which branch is this branches view is really helpful because it has like recently activated ones. So here's my switch to split heap. It doesn't build yet on everything but I could go back and fix it. It's a lot of this commit is wrong. This diff is going to be wrong but I I just straight up deleted the supervisor memory allocator stuff and then had to fix everything that used it. And I'm I'm using this memory allocator called TLSF which the expressive folks actually use and maintain their fork of. They use it in the IDF five. So on ports ports besides expressive will do we just use TLSF ourselves. We use TLSF ourselves directly instead of on expressive will just use it via the IDF. So yeah now now when we do things like allocate pi stack it just gets done with this new port Malik call instead of doing the like allocate memory which is the old API to the supervisor stuff. And you can call port Malik even when the VMs running and you'll get memory that can stay where it is. You will fragment your big region but with split heaps micro Python will still be able to like bridge that region. You only risk that you can't do like a super large contiguous Python allocations at that point which is a trade off and it'll be very interesting to see. Like what projects have more trouble and that sort of thing. But yeah so this branch that I've got going is like halfway there we really need like micro Python 121 with the split heap growing to be able to actually fully switch to it. But I ripped it all out right now in this branch I'm just allocating a very small heap so I was able to like test that it does work. But and yeah like I deleted lots of stuff. Everything is going to have a fixed stack. We used to have this ability to like change the stack size on the fly but that was only on some ports. And I don't think anybody actually uses it. So I just ripped it out. I don't want to have to maintain it anymore. Because it's tricky. Reserve PSRAM is gone because now you don't need to reserve it. You just hope you can get it when you need it. And we're switching over. We used to do on express of this by Ram use mem map. We won't need that anymore because we'll just allocate as we need from from the IDF. So on the IDF on the express support you see we just call heap caps malloc instead. Which is great so we have better better integration with our tosses that run underneath us and. And yeah they they don't lose a bunch of memory either it's kind of like whoever uses it can use it until it's gone. And then on all the other ports. They they have to give like top and bottom for them heap and then the TLSF will be used directly for those ports. But yeah there's all these warts that go away. So like there's this structure that tells you like what code file to run the next time the VM runs and that had to use all this fancy allocate memory stuff. Previous trace back string had to happen that way. Custom USB stuff was really tricky. Where like there was this like save to this temporary buffer call and all that. So hopefully it's going to clean up the memory management a lot. And still allows you things so like now now all of these things don't have to store this like wrapper object. They can just store the pointer because the pointer stays valid. So it simplifies things a lot. And I don't think the things that stored these allocations like I don't think they usually check to make sure it hasn't been moved. Yeah you can see I deleted the header file completely. So yeah that's the that's the future of memory management. It should be easier for us to maintain. This doesn't redo all of the display stuff. I'm still thinking about that. Because we have a limit right now on like how many displays you can have active at once. And so people have to recompile circuit by fun to do it. But there's really no reason that we actually have to have that limitation when we can more freely dynamically allocate things that live longer. We don't need to set aside this fixed static memory for all the displays at that point. So a follow up to this switch would be like changing how displays work so you can just make as many displays as you want. And then like the terminal will only show up on the first one. And potentially I haven't decided this but like potentially the first one is the only one that would stay alive past the VM. So yeah if you have thoughts about that let me know. Yeah 384 lines of this like complicated moving stuff I just deleted. Which is cool. And these are the default versions of Port Malik. They just use the TLSF stuff. Yeah. So that's coming but it depends on us getting all the way up to date with microbyte on 121 which is just released. So after doing all this I was like I got to help Dan. Dan is working off and on on the MicroPython 120 merge. So it includes split heap support but it doesn't have the code that auto grows. So I think we'll go even a little further and get to 121. And I also want to like upstream some changes that I've found as I've been fixing stuff. So Dan's been working on getting things building and I've been working on getting tests fixed. In micro in the merge version of circuit Python for 120. So I've got the CI going and nothing builds. So that's where I'm at is I just got the boards actually building because I had to get NP NP Y cross building. David says recall a while ago you were working on bare metal circuit Python on the Raspberry Pi. Now that the Pi 5 is out with dual displays do you see this on the timeline? Does Adafruit have other multiple display products? So I think the Pi 4 can do two displays as well. So I don't think that's new. David asks will this new method reduce the compiled size? I haven't measured it. Dan and I briefly looked at how much code size the TLSF stuff takes and it's about 3K I think. Is this yeah if we we could just look at my CI run and see. Okay so tests failed. Okay so yeah that I have to fix the Unix port on this prototype branch before I know. I'm hoping it'll be about the same. But time will tell and we're currently going through a lot of compile size churn anyway with the micro Python updates. And then another to do for circuit Python 9 is updating the compiler itself as well. So there's going to be like at least a some like degree of noise in terms of compiled size. Adafruit has one multiple display product and that's the mask with the two displays. And I think for that board we we set it to two. I was thinking about if you could this is a bigger rework of display IO. But if you had like a terminal and it could it could go across the two displays would be pretty neat. But right now all of the groups and all of the tile grids assume they're only in one display. So there's like transforming transformation stuff from the root display that gets propagated down. It's not not simple not simple code. I do hope going back to you David I David Buchanan. I do hope that the new memory allocation stuff will actually allow us to to have fewer static buffers which will give us more heap space. And so your heap usage should be a little bit more flexible in nine than it was an eight which is actually hopefully helpful on the same D 21. But that a lot of the buffers it's like 5 K of RAM that's statically allocated and that's mostly USB buffers. So it's potentially a pretty big rewrite of tiny USB. Or at least like making it configurable to do dynamic memory rather than static memory. But could be helpful for us especially because those buffers are can be like you know half a K to a couple K. Yeah I probably won't spend time on getting the Pi five running bare metal circuit pipe on but hopefully somebody else will. My my my core interest still is in the five four hundred because it self contains with the keyboard. That is still my my ideal use case for bare metal circuit pipe on. But now that we have USB host in circuit pipe on and this dot clock display stuff. So not the RGB matrix stuff but these round displays which I'm not sure I should. But on a stress if we have these like round displays and there are this like 40 pin and there's no frame buffer on the display. So the you have a big frame buffer in the PS RAM that gets spit out every frame. Like between that and the USB host we might actually start to be able to like make self self contained circuit by the computers. Kind of like I imagine for the pie four hundred but not using the pie four hundred. Any other questions. Let's see I thought that these builds would succeed. But all of them failed. It looks like everything is red. So we'll take a look at that. I thought Dan was getting them running but he's doing it locally not on the C.I. And the C.I. was like surprisingly tricky. Build board build board build release files get settings from make file is invoking make with print C flags. So this is something we should be able to reproduce locally. So this is trying to build the shared bindings matrix. So this is like if you build the board directly you're not going to find it. But if you try to build it from within like we have a helper script no rule to make target print C flags. So there's probably something that was deleted. So most of what these fixes have been of like OK what's missing or why did it used to work and now doesn't why doesn't it work now. So this is just main so what if we just do print dash. Yeah so here's under pie make rules and we can see that it's at the bottom and if we blame it it's probably something. So this is this had to do with the that disappears. Here's print config. If we go and look at this. Oh so yeah like it looks like Dan factored it out. But then this pie make is probably it probably got deleted from there because that's the one that we share with circuit Python or with my Python. He puts it in pie make rules but I think we could probably put it in. I wonder if we could put it in the circuit Python make file just so that it doesn't get accidentally dropped just like it did. So maybe we can put it in make environment. That's not what I want. Definitions those are make rules. Oh yeah this is the right place to put it. So let's see if this works now. Hey oh we got out. But OK so now we stage it and we see if it works. This copy paste handle make file tabs for you. I don't know looks like I did get a tab there. So that looks right. So we'll stage that and now we'll commit it add back make file print push it. Yeah so I'm like I was very excited about this allocation stuff because it'll make it much easier. Like we won't be bounded about the number of allocations we can do now. Like right now we have the static array of these supervisor allocations that limits how many you can do and it's just like a whole bunch of complicated code for moving it. And one thing that TAC was talking about so he added support to tiny USB for a USB host chip that's connected over spy. And this is something more just previewed for a new feather wing that is a like USB host feather wing that has this chip on it. And he was talking about needing just or wanting like a malloc free for tiny USB for that support. So switching to TLSF will give us that ability which would be great. Okay so now now that we push that new thing we watch the CI run. It's kind of like I reproduced it somewhat locally. And I guess I could I could run that release build scripts which hosts chip the max 3421. I think I remember tiny USB. Yeah max 3421. So he's been working on it and testing it from Arduino and then later we figure out how to do it in circuit by phone. And this is the perfect example of like we really want dynamic memory because we want support for it but we have no idea if we're actually going to use it. And with USB host we also that's just like displays that's something where we want it to run outside of the Python VM as well. So yeah the thing I'm going to figure out is like both displays and and this USB host stuff can use like I squared C and spy buses and those typically still get allocated. Like the board ones are special they get allocated statically but the but most of them are dynamically allocated in the microphone heap but then if they like they they might be used by a display or they might be used by this USB host stuff. And so it's tricky to know what the right way of. But you just always allocate these things outside the VM heap or do you still do some sort of like cloning out of the VM heap when you need it or something like that. I haven't figured that out yet. All right so it's going to kick off all the board jobs now because we didn't break MP Y cross which is great. The Mac one is still broken so maybe while we're waiting for the other ones we should take a look at this and you can see here that it's like triggering the 400 board builds. So this one's a little bit interesting because I should probably actually use my laptop like I could I could actually build this on my laptop to get it fixed because my laptop is a Mac. And this is these errors are clearly like compiler related. Yeah so that could be another like iterate via the CI which is annoying. Take a look here. I wonder if the docs build will work. I'm not sure it did last time. I was pretty excited just to get to this point though of like being able to narrow down like which boards are building successfully or not. I know Dan's working on that but getting it done in the CI is another. Helpful thing. Do those plans sound good to folks. What feedback do you have for me. I do feel like we're making good progress into the like Circuit Python 9 land. Like the IDF is up to date. One 19's merged in one 20's close one 21 is now out. So we can go a little crazy and go straight to that and it would be up to date. And I do want to have a discussion with the Micro Python folks about like getting all these sort of random things upstreamed. Yeah. Be out of says that is the same chip that is on the Arduino host shields I have. Yep hoping to review the max 3421 with tiny USB support. Thanks for mentioning it. No problem. We intend on supporting it in Circuit Python but we don't yet. And yeah I have like the spark fun host shield to test with. I haven't gotten that far. What would you think of a logo to MPY compiler. I have no background with logo so it's kind of hard for me to say. I would say if you're going to do that just go go to go to Python don't go to MPY. Just go to just go from logo to the equivalent Python code. You can always run MPY cross on the on the Python code. If you wanted to get really fancy you could write a Python script to load logo dynamically and run it. That would be kind of wild. Oh our boards are still not building. Let's take a look. So presence has a bunch of build failures. I know RB 2040 should be working. These are probably things that Dan's worked on. Let's see if any of them do succeed. Before we dive into them. I do actually have to like court. I'll have to coordinate with Dan too. I don't know if he's working on a anything specifically. Like if he I can pick up a different port or something. From what he's doing. So interact and Broadcom are going. I think I think the trinket and zero builds that should work. Builds locally I think. Unexpected maker what board of yours doesn't work. I've got like half hour. Maybe I could do that concurrently with waiting for the CI to run. Although I don't know if expressive boards working. In 120 either. Okay so that compiled. Let's let's do let's do an expressive. See who do I want to test on. Probably one of the expressive the C6. I don't know what whether it has it or not actually. No it doesn't. All of them when referencing boards dot XXX pins. I think I've got S3's on my desk here. What. Don't pull out my mic. I really like these new debt boards from the expressive debt boards with dual USB are really nice. They're really helpful for you debugging. Okay so this is a expressive S3 dev kit C. Unexpected makers in. Chatting in discord but not YouTube. They reported on tiny S3 feather S3 Safari I assume it's the same with all at least S3 boards. Comes back with invalid pin. Okay so we clean. That's not what I want. This is circuit by the name. I'm going to start over because I think I've got to update the IDF to whatever this merge is using. Did we get any boards that don't okay. I'm not sure if this has the 5-1 stuff in it. NRF's going. Alright I think. Let's just switch. Ah yes. Paul. Invalid pin. Analog in board dot battery. Version. 90 alpha one. A A A D. You know I think I might have fixed this actually. Because they changed the indexing of the ADC units. So this is specifically. For analog in. I think I fixed it. It's not going to find it. A zero. There it is. It's here dot clock. Overscan. But does that have my changes. Looks like it does. Fix ESP 32 with 5.1. Okay so yeah let's just try it. I'll give it a shot. Oh I want to get it out of the merge. Fetch. Let's do main. Do not care. Oh come on. Update everything. Make fetch. Also models. UBUS pin is not an ADC pin. It's just a digital read. Yeah we'll try it. I'll give it a shot. I'm going to do it on the espresso board. But I'm assuming it would have the same problem. I'll do it on S3. But there was a problem with. There was an issue with analog. Like it used to be the unit was one or two. And now in five it's zero or one I think. I think I did fix that. So I've had this problem. And I think I found an issue with git. I tried to file an issue. They don't have an issue tracker. They have a mailing list where no one replies to you when you send an email. It's been very annoying. But test UMPen. If you delete the remote. It should work. The problem is that it's trying to fetch from the wrong remote. Commit the changes or stash your changes. ESP camera. It's so annoying. I feel like the sub module stuff has just stopped working as well. It shouldn't be. I don't have any changes. Or VBUS. Yeah, I don't know. Yeah, if you're trying to do an analog read on the wrong pin, that would do it too. But there wasn't. Yeah, there was an issue with like in circuit by thumb eight, like pre IDF five. If a pin didn't have an ADC, it set the value to zero. But then in IDF five, they made it zero and one. So I had to change the, the no ADC pin to not be zero. But then I, but I think I fixed it. That's garbage. That should be close enough though. Ports, expressive. ESP. F export. I don't know why. Issue from you editor repel import board followed by dirt board. Let's take a look at our. Hey, oh, we have NRF boards that are compiling. Okay. Yeah. So I think I'm going to take, I'm taking Monday off. When I come back on Tuesday, it's fixed, fixed the builds, fixed the boards that don't work week or 120. It is interesting. There's some here that still don't build. It's probably just a size issue. Yeah. Like most of them work, but like German, German failed by four bytes, four bytes too big. That's real close. Four bytes. I hate it when that happens. That's like, if you just do a tag and your version strings shorter, it'll fit that. And that's, that's what's nice about running the CI. This one's significantly larger. NP Y cross not found. Interesting. It shouldn't have NP Y cross. Like there's this setup NP Y. Oh, I put it in the wrong place. Oh, I wonder how many things that's right. Okay. That makes sense to me. I was doing a lot of stuff around this in GitHub actions across download, make it executable. Why is it? Upload order artifact. Upload artifact. Is that right? I wonder if this needs to be, it looks like it successfully artifact NP Y cross was downloaded to circuit by then NP Y cross. And then we do successfully managed to change it to execute. But then there's a syntax warning and then the build fails make see NP Y cross. Oh, we're doing another NP Y cross build, which we don't need to do. Now I'm getting distracted again. This is the challenge of doing two things at once. Like now I want to update the script to not make NP Y cross at all because there's already this thing that's getting a cached version. Like it built and linked it. But now it put it in this other place. It built it under there, which is weird, but like it would be nice to save time. I wonder if that's only for frozen stuff that we build NP Y cross again. That's what it is. Let's stop getting distracted. I don't want to fix the script. I'm in the wrong branch. I'm in the branch to do all the ESP stuff. Okay, so we have a build and me just about out of time for today, but it's the way it goes. And that's the way it goes when you compile OBS yourself. That is the right one. I wasn't in the right. I like to have TO dev serial by ID USB silicon labs, which is the UART to serial converter. And then after it finishes, I hit reset and I load circuit Python and I do dev serial by ID USB. It didn't start. That's not good. It's not there. There it is. There's three. Circuit Python stuff. And it's not a debug build, so it's not showing up on the left. Now we have the repel though. So we can do import board. Board.io0 Import analog.io A equals analog.io.analog in I actually don't know if zero is valid. So io0 invalid pin, but does that actually, is that correct or not? We can go to ports, expressive. It may not be io0. Ports, expressive, peripherals, S3 pins. Yeah, so pin zero has no ADC. And so let's try pin one. So analog in pin one does not work as well, which it should. Assuming we want to use ADC unit one. So let's try 11 now. Board.battery. Yeah, so 11 works. I think, let me look at this. There was something between the two units. ADC index equals zero. So that's the problem. Is that that's not right. It should be ADC index is whatever the no. Like I, yeah. Okay. You found a bug. Good. This is an easy fix. Peripherals. It's this no ADC. Didn't I fix that? Where did I fix this? Maybe I missed it. I totally fixed this once already. Sock num ADC units. Did I not fix that? Where did I fix that? Well, I don't remember. Maybe I missed it. I found the issue. I fixed it and then failed to get it merged in. Because I don't think I have any open PRs. S3 pin mapping for red B. But that's not going to have it in there. Well, I missed it. Ports. Expressa. Peripherals. Update ADC and I2S APIs. Yeah, it's fixed here. That's what I would expect it to be. What brain? What am I on? This is main. I could have sworn I switched to main. Where am I in history? Enable RGB matrix on IDF51. C9D. October 6 today. And yet, Ports. Expressa. Peripherals. Pins. H is wrong. What am I missing? Am I on RGB matrix? I'm very confused. How would Git help you find this fix? Well, like I fixed it already. Why is this file not right in the wrong? Okay, so it's correct. This must be like a cache. Oh, you know what? This is in my circuit pipe on H directory. That's why it's a little better. Okay, let's close that because it's confusing me. This is where we want to look. So here we look. Pins H, no ADC. But I think the problem is that I did not update analog IO, analog N to say no ADC here. I think that's my problem. Is that I have the magic zero. Okay, so let's give it a try. Going a little wild in there. Just about ready to wrap up. If you have any other questions, please ask them now and we'll happen to do it. I should be able to get a PR out for this in the next five minutes. It's a super simple fix. We just got to test it because I knew what the issue was. Okay, so we want to test analog N on board IO1. Wi-Fi is doing stuff for board and for analog A equals analog IO, analog N, board dot IO1. Hey, it works now. A dot value. It's not connected anything, so it won't break. But it'll allow you to now. Okay, so that is the problem. Let's pull up our thing here and hit stage. I hit the stage button. You can't see it, but get status. Okay, so there's no issue for this. You see unit check. IDF5 made. Yeah, lots of stuff. That's totally the sort of thing I expect to find. So let me know, file an issue if you find more stuff like that. Jeff has gone for a few weeks, so I'm not going to have him know that. Sweet. Awesome. Well, that is a good way to wrap up. The CI is still turning, turning, turning through. Oh, look, most of the STM boards built. CI is still turning on giving us an update on where we're at. My guess is a lot of these have frozen modules. You saw me test it. It's not going to break anything else. We'll keep an eye on here. Yeah, so we've got a ways to go on the 120 merge. But we'll get there. All the Raspberry Pi boards failed, even though I thought Dan fixed those. But yeah, so that's what's next up for me is, this is like, submodule problems. Great. Great. All right, so let me get back here. Thank you, everybody, for joining me on this deep dive. Next week, I am off, but Foaming Guy said that they can do it. So I'll stop by this time next week for Foaming Guy's deep dive into the world of Python on CircuitPython. My name is Scott. I go by 10Nute Online. If you want to support me, you can do that by supporting Adafruit. They do a great job supporting me. And do that by going to adafruit.com. If you want to join the chat the last all day, all week, join our Discord server with the URL adafru.it-slash-discord. I tend to hang out in the CircuitPython dev channel. I do go over the help with CircuitPython channel from time to time as well. Thanks again to DCD, who usually takes time codes. I really appreciate that. I have to make sure I remember to, I'm not sure I updated last week's or the week before. So I'll take a look at that as well. And yeah, please try out main. Obviously it's a little rough, but we're going to get there. We're going to get the MicroPython merges in. We have the IDF in and then we'll polish it up and do that. So yeah, no cats. So no cat cam. This does work, which is cool, but it's crashing for some reason. The matrix is crashing for some reason. Oh well. Thank you all for hanging out there. Thanks for hanging with me as I figured out my OBS problems. Hopefully in a couple of weeks I'll remember to turn it on or try to run OBS before I need to get on immediately. And we'll have a little bit more time to fix it myself. All right, with that, everybody have a great weekend. I'm going to take some days off next week, but I'll be around through the core of the week, the middle of the week. So I'll chat with you then. And have a great weekend and I'll see you probably in two weeks. I haven't looked at my calendar. I should be able to do it in two weeks, but I got to look first. And I'll let you know if you want to get notified about changes, you can join the deep divers role on Discord. If you're on Discord, just ping me and say, hey, I want the role. And then we use it very sparingly, but just as a heads up, I'm not streaming today, that sort of stuff. So there's a role there on Discord for you if you want to get notified by me or a filming guy about it. Anyway, thanks all and have a great weekend.