 Alright, good afternoon, good evening, happy Friday, and welcome to the Deep Dive. Let's get a couple of our windows straightened out here. Looks like I do have sound to the OBS. Let's check on, looks like we are good there. Oh, is it going to be out of graph over in the YouTube? Nice to see you this evening. And I changed the scene back, yeah we changed the scene back. We should be good to go. Alright, so good evening to everyone. Hello, thank you for joining us here on the Deep Dive. My name is Tim. I go by Phone Me Guy on GitHub and Discord. This is the Deep Dive live stream program. This is a weekly live stream where we are working on things related to CircuitPython. From week to week it differs. Sometimes we are working on the C code, the core of CircuitPython, which is what I have open to a random file right here. Sometimes we work on libraries, sometimes we work on documentation or infrastructure or other things related to the project at large. Tonight we are going to be inside the core looking at some display IO stuff. Let me do take just a quick step backwards though in case we do have folks that might be joining us for the first time or watching this in a video after the fact if you don't really know what I'm talking about. This program is going to be about the CircuitPython project. This is the website for the project right here, circuitpython.org. This is basically an implementation of Python that runs on tiny computers called microcontrollers. These devices will plug into your big computer. They'll plug into your actual computer, your desktop or your laptop. They'll show up like a thumb drive. There will be a Python code file on that drive. You can edit that file and save it. And then the microcontroller will actually execute the Python code that is on that file. Different ones of these microcontroller devices come with different form factors and shapes and sizes and capabilities. Oftentimes a lot of them have what are known as IO pins which allow you to connect other hardware to it. Other pieces of electronic hardware like lights, buttons, buzzers, speakers, sensors, GPS modules, Wi-Fi radios, all these kinds of things you can connect using these IO pins and then you write your Python code in order to interact with all these different pieces of hardware. That is kind of like the really, really high level, quick version of what we're looking at. We will, of course, be way down in the depths during this program in the C code. So if you are brand new to this kind of stuff and you want to learn more, a couple of places you can go are circuitpython.org and then also join us over on the Discord which is, I believe, still linked down below, ADAFRU.IT slash Discord which is down on the screen there. During the actual stream, the chat right below me here is the live broadcast chat but there are a couple of different circuitpython related channels over on the Discord as well. So if you just want to learn more or get involved in the project or just say hi, come and join us over on the Discord. That's another place to learn and just get in touch with folks. How's it going, Devo Dessa? As well as DJ Devon. Nice to see both of you as well this evening. So today where we are starting... Oh, I will say as well, circuitpython is an open source project. Anybody is allowed to use it. Anybody is even allowed to port it to your own microcontrollers so third parties can make their own devices and then add support to circuitpython for those devices. All that is free of charge. You don't have to pay anyone in order to do that. It's an open source project and it is primarily funded by ADAFRUIT. This is their website, ADAFRUIT.com. ADAFRUIT is hardware and software company based out of New York. They are the ones paying the folks who work on the circuitpython project. There's a core team of folks who work on the project full-time who are paid by ADAFRUIT to do so. There are other folks like me who are paid by ADAFRUIT part-time to work on the project and the surrounding stuff that goes along with it. So huge thank you to them. And then of course if you would like to help support the project one of the ways you can do that is just by heading over to ADAFRUIT.com and purchasing hardware from them. By purchasing hardware from them you're helping keep the whole ADAFRUIT machine and ADAFRUIT factory running which of course is allowing them in turn to help continue to fund the circuitpython project and the folks who work on it. So thank you to everyone who does that. More specifically where we are jumping in tonight is back onto a core display IO issue. We looked at this I think maybe a week or two back. We did some digging on this I think on this program and then again on my own live stream. And I've got a pretty good idea of a potential fix for this. But it needs to be cleaned up and it needs to be pre-commit run and it needs to have all the comments removed and it needs to just get ready to be an actual like PR that's really ready to consider. So that's what we're doing tonight. This is the original PR. This is the issue 8013 was submitted a little while ago and basically what this issue boils down to is that when you have tile grids, tile grids are one of the core components within display IO that you can draw on the screen. When you have tile grids that are hidden, they take extra time to render. They take a non-zero amount of time to get drawn even though they are hidden and therefore not actually being drawn. So we are diving into the core trying to figure out why that is and then come up with a way to fix it. Where we are at right now is I do think that I understand why it is and I do think that I have a potential fix in place that I still need to do some more testing on and then clean it up a bit. So let me show you that bit. So let's get to where first draw. Okay, so yeah. We have lots. I added a boatload of different print statements in here while I was trying to learn about what all of these internal fields are for. I'm actually just going to comment this one out as well to start us off. We are currently inside of tile grid finish refresh. So when you refresh the display, oftentimes in circuit python land, typically the default is auto refresh is true, which means that it just calls refresh itself. You don't actually have to do it yourself. But you can turn auto refresh off and then call refresh on your own if you want. Either which way though doesn't matter which way, whether it's auto refresh or regular refresh. When that happens, this method is one step in the process inside the core. Specifically this is on the tile grid and it's called finish refresh. And some of the logic inside of here, in particular this bit right here, first draw, it's creating this Boolean first draw and then the value that it's setting it equal to is whether or not previous area X1 was equal to previous area X2. So previous area is essentially two points, X1, Y1 and then X2, Y2. And those two points represent the opposite corners of a rectangle, specifically the rectangle that this tile grid that we are inside of here occupied on the previous render frame. Not the current render frame but the previous one. So that's what those two points on previous area represent. So this is saying is X1 equal to X2 in the previous area. Which essentially like in English the question that this answers is does the previous area have a zero height? Or excuse me, width actually, it's width not height. So it's saying is X1 the exact same as X2 which means they're aligned on a vertical line. So therefore there is no width. That's kind of like in English what this statement is doing. It's storing it into this first draw. And what is happening is that when first draw is, let's see which one is it, true or false? I think it's this one. I'm not 100% sure. I think it's this one. So that would be when first draw is true because this is saying or first draw implicitly it's like first draw equals true. So down here, this is the one that takes time. When we get through, when we get into this function, when the core is inside of here, when it calls this, this occupies time. And so what we want is to not occupy time if our thing is already hidden and if we have already drawn it once after it became hidden. So the root kind of like sort of logical issue that leads to this is just that first draw is the Boolean that it's using to determine whether or not it should actually get drawn. And the width of this thing, it's not actually, it shouldn't really matter I think. Basically it was written to kind of assume that this was maybe different than it really was. So first draw the concept, the idea of like, is it the first time that we've rendered this shape since it's been here and in its current state? That concept, that is good. That's pretty much what we want but the value it's getting set to is actually unrelated to that conceptually. And so what we want to do is kind of rectify that. We want to make it to where, well, one of two things. We either want to make it to where first draw is going to be actually answering the question like have we already been drawn or not rather than answering the question of like, do we have zero width or not? And then that would be one potential solution. I think if we could do that then the rest of this would behave as expected. Another potential solution could be to kind of leave first draw the way it is but add some logic outside of here either above and outside of this or even perhaps over on the group side which would be the group that is the parent that contains this tile grid perhaps could have some logic over here even that maybe checks that value and then decides whether or not to actually spend time or not. So that's kind of the idea of how we're going to try to fix it I do need to figure out exactly what state we're in because I think that there is like I think I started this fix in it somewhere so let's compare this back over to main let's figure out just where we're at so these commits down here are the ones that exist in our working branch hidden refresh areas and they do not exist in main we're just going to take a quick look through these a lot of them are going to be prints in fact this one is entirely and completely unrelated this was a different thing I was working on what I'm going to do is actually kind of revert this change by copying this and going into shared module display IO init to put it back to how it was because that is not actually meant to be part of this PR for real that is around line 296 which is right here back to how it was I guess we'll just go through it doesn't live update but we'll have to check after was that the only thing in init? yep looks like it okay display so a lot of this is going to be prints like this whole thing here technically it's a variable and a print but it's actually just a print essentially this will eventually get get removed but I am going to leave it for now so inside group here this guy looks like indented a bit and then the new bit is down here where it says okay if display IO tile grid get rendered hit okay this was my first attempt at the solution and the solution that I elected for with this attempt is more so the second of the ones that I talked about it was I implemented logic somewhere else somewhere other than that first draw variable and this is the place this is where the logic ended up being I basically said here if not tile grid get rendered hidden so if that's not true then do the work basically call refresh on it if that is true if we have already rendered hidden then we don't bother rendering it again so that's the potential solution one of them that's inside of this pool that's inside of group at 459 so we're going to want to remember that in a minute because we may turn that off and on a bit to do some testing that's the other thing I need to do here still is more testing inside tile grid we have what kind of differences so I added a function tile grid get rendered hidden this is just returning self rendered hidden inside of tile grid set hidden we say if we are becoming if hidden is false it does this and then if hidden is true we set rendered hidden to false so when we become hidden we turn rendered hidden false and then when we actually get rendered after having become hidden then we will switch this back to true that's kind of the idea of what's going on here and the rest of that is going to be down here a bit this is just to do down here we have a boatload of prints and then even more prints that are commented out and then right here inside of refresh areas we are saying if we are hidden then we're setting rendered hidden to true because we are rendering here like by the time we make it to here if we do the stuff inside of here it will cause us to get rendered essentially so we're saying yep now we have rendered the hidden version of our self therefore we don't need to do it again the next time the display changes in fact we don't need to do it again unless or until something else changes you know if the XY coordinates change if the hidden state changes if the group that we're inside of changes if those things happen then we need to evaluate again whether or not we need to be drawn but if none of those things happen then we just skip we don't spend any time or effort on rendering ourselves and then I added essentially a boolean here called rendered hidden so the current state of this does have the fix in it I'm going to go ahead and make a build and we're going to test the fix with the original reproducer code because I have not actually done that particular test yet the fixed one with the reproducer code I have test the new version with some more specific stuff related to how the new version works but not the actual original reproducer case so we're going to want to make an expressive build I have what is this s2 s2 tft so let's take us back up to the expressive and then we want to export dot space sp that one export yep do that and then make port I don't have a full name let's do this whoa let's do that make third so the fix right now boils down to inside a group we check if rendered hidden we check if that's true or not and if it's not true then we do render but if it is true if we have already rendered then we don't bother rendering and by not bothering we save the time that it was taking that's the idea at least so one thing to consider would be do we want to have it called rendered hidden maybe it would make sense to actually have this be called first draw like the name of that variable was and then maybe we would change this instead of being first draw it could be I mean instead of being you know previous area X previous area X2 it could be you know get get first draw get the value from self hidden by parent you know I think I need to edit the code inside of there as well actually to change the value of rendered hidden because right now I think I updated it inside of set hidden but not set hidden by parent essentially it boils down to do we want to keep this exact first draw thing here or not is there anywhere else that it's used because the thing is I don't think that it does as far as I can tell it doesn't do what as far as I can understand the intention was we do have another one here and get refresh areas so we had one and finish refresh we have one and get refresh areas as far as usages we also have this down here full change I don't know I'm of two minds like on one hand I want to just leave first draw alone and not change it at all because I don't fully necessarily understand like I'm not positive if my conclusion about it is correct about the way that it's not actually serving the purpose that it meant to I'm not I'm not positive that that's the case so I am inclined then to try to fix it by changing something else that way in case I'm not right about that you know we're not modifying it we're not changing it but on the other hand if I am right about that and first draw is actually not doing what it was intended then it probably does make sense to change it so that it is doing what it's actually intended all right we've got this let's plug in I don't think I've got the camera set up yet did I ever figure out the thing you were working on with ESP spy airlift co-processor uh yes uh that's an ongoing thing after the wage yeah yeah yeah I did I figured out the specific stuff I was trying to figure out and I have a proposed solution in place the next step is still some more testing that I need to carry out on that proposed solution um but yeah I did figure out the part of it that I was working on and I did uh also get a fix proposed fix we'll possibly work on that some tomorrow but we'll see I have an idea for something else to work on as well tomorrow that I might do that is a more project than than like circuit python itself like a project I want to make with circuit python more so than something that's part of circuit python so I may be working on that tomorrow yeah that is also totally correct there was discussion around that in the weeds on the weekly meeting let's get to bootloader oops got the name of it for a minute I've been doing too many rp20 40s I want to just do rtab but that's not right for this one family visiting and exhausted I exhausted them working on house stuff taking a nap tired have some free time to enjoy the stream nice not sure if I'll be around tomorrow not enjoying at least part of the regular routine for now nice that's always nice get a little chunk of time to get into normalcy when you're in a situation like that I can definitely definitely agree with that it is nap time over here as well my wife and probably both of the cats are off having a nap right now as well um okay what is our current code pie it's probably Igmo Machine there you go that's a there's a sneak peek I won't I won't say anymore about it for right now but there's a bit of a sneak peek of what the project we might be working on I think tomorrow is this line group okay yeah this is kind of the original reproducer but I've done a lot of commenting and doing other stuff I'm actually going to try to grab it back from my reproducer is that in the PR I know there it is it's pretty similar to this anyway but I want to get a fresh copy of what's posted so let's just step through this code here as well just so I can make sure I understand and remember what this is doing so pretty standard stuff initialize the display we're using a built-in display so we just grab it off board set the auto refresh to false okay so we're not going to be doing auto refresh make a main group make a line group loop 70 times make 70 rectangles these rectangles are this size they have three width they have half the screen height they're going to be white we are appending each one of those rectangles to main group so we're going to have 70 rectangles we're going to add them all to main group we are going to then put line group also inside main group we haven't added anything to line group yet but I'm guessing that's what comes next we have four 150 times make a line so make 150 lines we are positioning the line based on eye so each one should be moved a little bit we're saying for half of them every other line we want to set it to hidden we are appending all the lines into the line group we are showing the main group we are setting variable color to be white we are saying for display objects displayed with an ed I think maybe would be a better term for that variable but that's a right displayed object in main group if has adder fill then set fill equal to color this is effectively going to change the color of the rectangles these 70 rectangles here they do have a fill property so this is going to set the fill property equal to color which is white the first time it will then print the time it will do display dot refresh it will change the color to be a different one it will wrap back around so if it goes off the end of the color wheel then it will wrap back around as though it were a wheel instead of a line so we've got our 70 rectangles we've got our 150 lines right now half the lines are hidden we might have had release displays honestly you know what it probably was I think we just did not I think I have auto auto run off auto reload I mean off so I think when I modified code it did not just automatically execute I'd actually control C control D so I'm pretty sure this is inside the core see here with this with this particular version right now which does have the proposed fix in it the first chunk there when we rendered this the first time it was very very slow we were counting up to that 150 or whatever once we got done with that though once we finished it with the first time then now it has sped up so we're able to get these like relatively fast changes in the color of these rectangles here which looks like one big rectangle but obviously we know it's actually 70 small rectangles and just to verify so basically the fact that this is running fast now this is showing that we do have the fix in place I believe so if we were to go back and comment out our fix so run it like this then I believe essentially every other render will be slow again so it'll do one fast one and then it will do one that's really slow like the first one was and then it will do one fast one and then it will do one really slow and then one fast and then one slow and one fast and one slow I think that's how it will behave this way which is essentially what is currently released in display IO so let's validate that hypothesis color plus 1600 I've never seen that neat makes me think I've been using color wheel wrong I to be honest I had never seen this before either I would say personally I think like rainbow IO in color wheel I actually think of as a little bit easier to use than this this one is interesting like it doesn't require an extra input import I should say and therefore I assume that it's like a little bit less RAM intensive and maybe CPU intensive since you ultimately just have a number and it's just incrementing it or whatever so it's like really bare bones I assume it's really really efficient but I will say as far as the API I do kind of like the color wheel being able to just give it that number 0 to 255 or whatever and it does its own bounds checking and all that stuff so if you give it a big number it just auto wraps around which is all super nice but this is an interesting way to do it that I had not seen either we got our build let's get to bootloader mode made some code to translate it to 0 to 0.5 in separate RGB values with PWM it was a 169 versus 170 issue if I remember this one yeah so you had to go deep to figure that one out yeah it was a bit of digging it was a bit of digging but we made it in the end and I mean the truth is like I get enjoyment out of it it's kind of a thing I enjoy digging through some of this stuff so it works out but when you said color wheel you were using rainbow IO color wheel this one is using this one is using the idea of a color wheel not the actual module itself just the concept yeah totally agreed as far as the efficiency on that one I assume it's probably through the roof compared to the other one so to speak so right now we are in the midst of one of the slow refreshes so the hypothesis I have is that after this slow refresh which is going to be about when we get to 150 I think or maybe 220 actually I'm not sure but anyway after we do get done with the slow one then I think we'll get a fast one but then I think we'll be back to another slow one so there's our fast one ah okay I was wrong actually we got two fast ones but we are back to another slow one now that's kind of interesting it's every third one that's kind of interesting let's watch it again though let's see what it does this time too and I guess the thing is technically the color change is part of this actually technically the color change is part of the refresh that we are already inside of the first one and then the time that we see it change again that's actually the next refresh so it is back and forth but it feels yeah it feels like thirds because that changed twice in rapid succession but the thing is one of those one of those in rapid succession there was actually the start of the current refresh like one of them was a complete full refresh and then the other one was the beginning of the refresh that we are starting here which is going slow yeah we totally do get we totally do have this kind of back and forth thing going on though one thing that would probably be good to do eventually would be to shoot a quick video of this um but we'll probably do it on a stock I think let's do it on the stock image instead of my custom image here which has got prints and all kinds of other stuff going on just to make sure that we are actually seeing the changes we think um and we got an interesting little vertical line artifact in there which does happen sometimes I do see stuff like that from time to time okay so I definitely think we have like this version is definitely worse than the version we just had a moment ago where we had one slow render and then every time after that we got a fast render oops that is just with the fix here has gone aspect or W okay um I'll be right back I'm gonna throw some water or I mean some ice in my water okay let's get it to bootloader got some homemade cold brew this evening whipped out the old french press made some cold brew last night started it last night threw it in the fridge tasty here we go so this is with the fix in place and we do have our slow render here this part still still slow but once we finish it once then we never have a slow one again as long as we don't change anything again like if we were to change the position of our diagonal lines or if we were to swap you know swap the ones that are visible versus invisible I should say hidden and unhidden instead of invisible but if we were to swap those then potentially slow down again but here we just get boom boom boom boom all the way one after another right quick there this does fix the original this does kind of solve the original reproducer which is nice because like I said I had not actually tested it specifically back with this with this reproducer here with the 70 and the 150 so as far as submitting it I think for now I'm just gonna leave it I'm gonna leave it as its own thing this rendered hidden I'm gonna leave it as its own thing and I'm gonna leave first draw alone and then we'll see we'll see what the feedback on the PR is if Scott or anybody has got ideas and thinks it would be better to to change the first draw logic instead then we can do that but for now I'll take it this way where it's like completely sort of just doesn't touch first draw at all leaves first draw alone so if there is some case where first draw is behaving in a way that I'm not fully understanding that is desirable then I won't break it since I'm not messing with it so one thing I do need to do to get this cleaned up though is to move these approximately one bajillion print statements um what I'm gonna do is actually take a new branch right here I'm gonna say I want to make a new branch where I'm at now I'm gonna call it hidden refresh areas with prints should I commit first I probably want to commit first yeah yeah I totally want to commit first because I want this I think we don't want this we can undo that and this one doesn't matter so much but I'll leave it this is revert revert un elated EPD change this had actually nothing to do with the this now I want to make a new branch called new branches here we go new branch called hidden refresh areas with prints with prints and just push this well I hope I pushed that to foamy guy last time you know I'm pretty sure it would have been I don't not look and I probably should have okay now we're gonna go back to just hidden refresh area and I'm gonna take out the prints this way I have a snapshot of it with all those prints in it and by snapshot I mean like I have a branch that I can easily go back to obviously the version control has a snapshot of everything so like we could have gotten back to it without doing that but just makes it nice and easy pretty much every one of these MP prints we're gonna be wanting to get rid of because I think pretty much all of these are ones that I added throughout the process of working on this oh inside get refresh areas if hidden is true which would be the case whether it is self hidden or hidden by parent if that's true then we come inside of here and we say rendered hidden true and we basically do this stuff which will cause us to be rendered so I guess this was an old attempt with this if statement here almost all of them right there save that let's go check in display this was oh and actually we could take out this as well this whole eye was just debugging essentially okay let's look in group looks like we have none in that one so that should be pretty good already okay so let's commit here wow let's run pre-commit and I did not change the strings at all so I'm not entirely sure okay this has a change but I'm going to elect not to put that in my commit yeah I think we should I wonder if we should add a comment here like set rendered hidden true to indicate that we have that's so wordy though it needs to be more succinct if we're gonna do that so this was basically remove this foamy guy it is definitely foamy guy how's it going to shippu you take a big you take a big I probably I'm sure I probably messed up the pronunciation but it's nice to see you on this Friday evening for me but during this deep dive let's say thanks for popping in say hi let's see we should okay let's get a look at where we're at here because this will be the current state France okay so we'll see if this passes I think it takes a while though honestly look at this and make sure the diff is actually so if it's weird on this one shows it makes it look like I changed more than I did I think I mean I guess the mention here changed did it need to let me put it back so really the only part that I changed is this let's try to put that back so everything inside get refresh areas inside group why I guess it didn't change it I did a format inside my editor I pre-commit is it going to leave it or is it going to change it back it left it good evening in swiss nice a big good evening oh this looks like it oh you know these icons you can't trust them sometimes I feel like these icons they sometimes jump the gun like just a second ago this had an axe I swear and then it changed see there now it's back to an axe but did it actually fail or did it just stop because I committed again oh this failed formatting because of the mention why didn't it do that for me locally have different have different version of code for matter is good how to get that that was the last one hopefully we're good on the next one like right here this is the actual bit that I really changed this was the only bit and basically what we did is this was the statement that used to be there and we just wrapped it inside that if statement we do we still have these two messed up so self self.members.items used to be indented one more that is inside this and why this is different so I could quick stroll through here as well and look for other stuff so yeah we probably want that extra line there so we're going to get rid of this change and this change because that's not really part of it though the actual changes we have then we implemented get rendered hidden it just returned self get rendered hidden we said set hidden if we were hidden then turn this to false that way we can get rendered once and then turn it to true after that let's get rid of this this is first line inside finish refresh inside of we're in tile grid here first line empty there it is okay let's also look we should probably do inside of set hidden by parent I'm pretty sure we should also be setting this to false we also added this I guess we could take that out that's after the if hidden trial grid display trial grid finish refresh numbers are different if hidden there it is but not it's not finish refresh it's actually get refresh areas okay this line right here I think created it on the struct or whatever we declared a get rendered hidden function I got all the unwanted space removed let's also look in set hidden by parent see here I think we need to do the exact same thing that would be when hidden is true when we become hidden in this case because our parent got hidden when that happens we set this to false and then get refresh areas if we are hidden we set that to true and then we return this stuff which causes us to actually get rendered the first time so then now this is true which means the next time through we have this to be true which means back out here no longer false therefore it does not call get refresh areas this was remove un remove just say revert white space white space change and we did the hidden by parent a lot of commits but I think we're finally caught up I had not failed yet on this one the last one so I assume that means that at least the code formatting passed hopefully okay yeah now this change is looking a lot better this only has the bit that I changed now that's way better yeah we got this basically minimized to only the actual code I touched perfect okay we would basically to see if that's going to pass or not and maybe give it some more testing so another thing that we should probably be testing well see here should maybe run this on a different device for one thing what was the original test was on Pico W you know I think I do have a teensy but I don't know where it's at it doesn't have circuit python on it code W I do have but not one that is readily connected to a screen already need to wire up a display with one thing that I should definitely test is making them visible again right now we set half of the lines to be hidden but we never turn them back to be visible maybe we'll say every ten times every ten times put that back to zero and then let's loop over all the lines which are all in the line group if it's hidden actually we don't need to use any statement let's just say line hidden equals not line hidden okay so every ten iterations through the main loop we will basically flip-flop all the lines that were hidden become unhidden all the lines that were visible become hidden seems like anytime I make a change I have to re-run black I thought black was supposed to do stuff automatically if I had previously been run pre-commit is the one that will do stuff automatically and it only does it if you pre-commit install if you always do pre-commit run which is how I do it so if you do the same thing that I do then you are manually triggering pre-commit but if you do a pre-commit install instead of manually triggering it then what happens is pre-commit runs every time that you're about to commit so like in your IDE or whatever you do when you go to say like get commit at that time that's when it will then run I tend to not use it that way though I just do it manually yep take it easy Dave Odessa thanks for hanging out with us for a bit have a good night yourself as well depending on your IDE you have a format on every save that doesn't work in a way with get test up yeah if you do and that's pre-commit specifically and then of course like inside of our pre-commit config we have it launch black but it is pre-commit that's doing the automatic bit and it does only if you do install it pre-commit install which is the name of the command that doesn't mean like install it as in make it available as software on your system that's a different kind of install you have to have it installed as well in order to use but this is like installing it into a repo is the act of essentially enabling that automatic hook for whenever you are about to commit to have it go to its thing I think I did not run this latest version I would assume this basically means every 10th iteration will be slow let's make a new build and run as well actually though because then we won't have the prints although now let's run it once like this because we'll see the prints give us something other than the color that allows us to tell how fast it's going and it does look like this failed so while that's doing its slow thing there let's see oh see I told you you cannot trust these icons sometimes see this is X up here but it's just it's not done also I now I have a bunch of them running can I cancel them these ones show X see it's all changing these things all the time anyway we're still in the first one there we go so we got fast fast fast fast going there we must be on the 10th one so we get another slow here because we basically changed which lines and in our case it's every other line either way so like it like shifted or whatever but it's not really that visible doesn't look like that much of a change pre-commit not black never mind thank you yep for sure alright then so then we'll get fast fast fast we get 10 of these then we get to the slow one perfect exactly play as I yes so let's do make another build that loaded and then it will it won't have the extra prints so it won't be printing these which actually this I think is slowing it down a bit as well although not that much right it's still going to be slow when it's in the slow refresh so really this is a matter of how many objects just 220 objects is a lot of objects to render for display oh my recommendation would be if you are trying to draw something with that many objects in it most certainly going to be better off instead of making that many different objects be they shapes or tile grids or whatever instead of making that many objects you're probably going to be better off making one bitmap and one tile grid and then just drawing whatever pixels you want inside of your bitmap if you want two different lines instead of making two bitmaps one with each line you could make one bitmap and then you could draw the two lines fill in the pixels that represent both lines inside that way you have fewer actual objects which means that when refresh gets called it's iterating over fewer objects each time so that's definitely the way to go if you are really interested in getting the most speed that you can out of the rendering but in our case here this was like purposefully kind of a torture test in a word right this was intentionally this was meant to be quote-unquote difficult to render because we wanted it to make the bug easy to spot sometimes I'll click on commit details on the git website just to watch it scroll through thousands of things it is I will agree with that sitting there and watching the log flyby is kind of interesting sometimes is mesmerizing in a way the way it kind of pops in and then we'll give you like a burst of a bunch of them all at once or whatever definitely agree with that so we're back to having way fewer prints now we print times but this is in code pie we print the monotonic time right here but this is in our code pie file not inside the core as far as the actual core is concerned we no longer have any more prints inside there all right so we got fast and this will be the same deal we'll get 10 fast and then a slow or you know maybe I'm off by one or something it might be 9 or 11 but the slow one will be when we change the lines um let's do one where we don't let's do one where we hide every line instead of half of the lines does that make any difference it should go way faster when they are hidden let's just say line hidden true and then that means now when we get into here it'll be flipping every single one instead of every other one I did control C but it's one of the things is while you're inside that slow refresh you can't like keyboard interrupt it will eventually take over but it takes it a while you could actually fix that probably I figured out how to solve that sometime recently you need to add a call to add a call to like interrupt something inside the code that's ticking too long it's but I think it's probably best left as a separate thing okay it'll be all hidden first we should get super fast and then we we'll still have our slow render like when they become what was interesting about that is we didn't have any down here aren't they longer than the big one aren't they longer than the big one so it is way slower while they are visible but that makes sense because while they are visible it has to spend time rendering them and then when it changes from visible to not visible or vice versa those are the longest frames those are the ones when it takes the longest amount of time it does go longer how come we weren't going all the way so our line goes from I 0 to 100 plus I and then 100 and that is definitely longer than the colored rectangle it's actually multiple rectangles it would be pretty nice if we could bail it goes definitely go way longer than the rectangle now the colored ones and then the 10th iteration it will at least hit into false if you do one that's very slow and this is like while it's still processing yeah we're having some kind of problem here we're not drawing the full thing that's no good hmm was that bug that I introduced with my fix here how come when we draw hang on where I got some crazy hearts going on in the youtube there I don't know how that's happening or what is happening there but that was pretty cool that come through I don't know if that actually would have made it to the stream or not but there was just some hearts flying there on the youtube chat before I actually loved this let me try one more thing if we were visible to start with but hidden later on does it behave the same I have this theory that oh hearts are going again they are making it to the stream nice I don't know who if that's a person doing that or how I don't know if that's the thing somebody's doing but thank you for that whoever you are if so try the opposite basically we had them visible first and then we changed them to hidden after I think in this way it would actually work correctly totally must be somebody doing it we're getting all sorts of different ones yeah see this one does work oh you know what this starts to hint at why it's taking so long as well kind of because it shows you how fast it will work there I feel like our focus has a little mess up huh let me click like on the stream ah nice thank you up on the youtube streaming okay so I am going to oh interesting it does break again after you change it once we're going to go back fuck is this a thing that I introduced so then we got it back into the core and figure out what's going on it's super weird though that it's like only certain pixels for other reactions in the youtube chat head reaction oh okay it does feel like we should be back to faster but this is the stock build though right this one will have some slow frames no matter what in fact we're like every other frame is going to be slow which means getting to 10 is actually going to take kind of a while isn't it put this down to 3 and we go back to it's interesting too like it's sped through that first bit but then eventually slows down it's like there's some buffer something of how fast it can basically I need to wait 3 iterations now interesting yeah like the pixels that are outside of the other parts here like very slow interesting and it definitely feels like it can render faster than that so I wonder if there's some other bit in there that we could maybe inch to speed it up although what I said earlier still stands as well like if you are trying to draw this kind of thing you could get it a lot faster with a single bitmap it is looking like I have broken something though because on the version that had my fix in it it wasn't rendering the white outside of like beneath the colored I might need to try to make a more minimal reproducer of that particular bug and then start trying to tackle it this is like way too long to wait for each iteration if I was going to try to like make changes here and then see the results yeah the slowest one was about a hundred seconds okay unfortunately I do think kind of have a bug in terms of what's getting drawn and what's not let's say too failing hearts appear when we click the heart button and chat nice yeah I think it's amazing Nick over on what's that this could be just big or something I don't want you to say say sixty eight but I'm guessing that one of them somewhere is failed no, failed metro M0 DE enough bytes we are back to here I'm gonna make this reproducer much smaller try to isolate this particular issue so I think we want to draw one colored rectangle and then maybe one line so let's get a main group let's get a line group let's make one rectangle say 00 we'll say width of 100 height of 20 sure yeah let's make one line instead of 50 150 x so we'll say 0000 and then 100 100 let's say width of the for loop don't really need to loop over these things with this version let's just say rect didn't actually add it anymore main group append rect fill equals color durations just say this one doesn't need the for loop so I could just line this L push color so obviously this one can go way faster it's only drawing one shape but we do still have the same problem of the line does not get rendered down here let's turn this off and run it like that so when we just draw the line once and we never make it hidden everything's fine if we set that to true we put it right back to false I think it will still work correctly because we didn't refresh yeah but if we set it to true and then call refresh well let's add it to the line group let's do set this to true display let's do that our first refresh I guess our first refresh actually is not until here on a refresh before that actually refresh here and then after we refresh let's say line dot hidden for one second then make it hidden false so show it and then refresh again so we go back to showing it but I think when we go back to showing it it's not going to have the bottom part that wasn't true it's hidden for one second and then we show it and then in our case it seems to just be fine but if we flip it back and forth it gets messed up what if we do it one more time refresh while it's not hidden the number of times we're doing it or is it somehow like because it's in the loop specifically not hidden not visible visible not visible right then it stays not visible let me add one more actually not visible as it goes to visible the last time it does get broken let's make the rectangle a different color okay so that's interesting because it's not visible at all there at the end but it should be hidden false could be doesn't come back different than before invisible hidden visible but it doesn't become visible go into group and say else print f entered hidden was true I would assume that we are here and therefore we're not rendering we have a bug where if we change back and forth every frame we get locked out could be the case basically could like because of that boolean logic maybe it just gets stuck if you switch back and forth in in some case I guess whether it's back and forth or whatever but in some case it feels like we logic yeah so we don't have our logic handled correctly so what has happened I wish I could get the value of this easily in python code but I cannot this is a thing that is kind of like internal we could like add it to shared bindings and then we able to but it would be a fair amount of extra boilerplate code that we need step through the logic here so this is group refresh areas that is our main group so when refresh gets called on the main group it's going to loop over all the members inside the main group if the member is a tile grid and if the member is not null or technically that's the same thing basically the way this is worded is but makes it seem like those are too different but it's not actually this is saying need to get a new battery for my mouse this is saying set the thing to be a tile grid like assume it is a tile grid and then this is saying if it was null that means essentially that it was not actually a tile grid so we can't do it but basically this should not have been tripped the next time but it did why did it get tripped so the first time we call refresh hidden is true which means that rendered hidden should be true because it starts as true but then once we set hidden to true that should have said rendered hidden to false setting rendered hidden false add it down here even though that's not unfortunately the only real way I know how to debug this kind of stuff is like add prints basically everywhere and just keep figuring out where the code is getting to and what the value of the variables is at that state and then it's not not the fastest process but how are we going to do it and I know how to do it I should say feels like our it's changed at some point and got blurrier but setting rendered hidden to false so when we initially turned our thing to hidden we then set rendered hidden to false which makes sense because we haven't rendered yet that's what we're trying to say is like we became hidden but we haven't been rendered yet that got set to false we then would have been rendered when the next time it called refresh the first time in fact that it called refresh right here we then printed hidden right after the refresh so the refresh took place essentially right before this got printed when that refresh took place it should have flopped this variable to become true so let's make sure that that happened whenever we set this to true let's say setting rendered hidden common print yeah keep common add print statements common add more prints they should make that that's probably somebody probably has that I guess but no we should make that it's definitely a good mantra oh interesting did not change when I thought it would so that's probably part of our problem so we got setting rendered hidden to false so when we set hidden to true we then set rendered hidden to false that part went fine I wish this would stop closing my file I was then expecting inside of the refresh here specifically at finish refresh which I assume is nearing one of the last parts I was expecting that to get set back to true but that's not actually happening but we are doing that in get refresh areas maybe it should be in finish refresh so we set rendered hidden to false we refreshed we print the word hidden rendered hidden did not get set back to true until we made this visible when we made this visible it looks to have set this to false again visible is here is after the refresh even so no this is actually coming from here so I mean the crux of our logic is the fact that rendered hidden is not getting put to true inside of here for some reason and it should be it's written to assume that it will so sprinkle a few more in and where we can get inside finish refresh so what I want to see is does this happen before does this happen before it basically trades it back over to true because if it does then I'm thinking maybe we should be setting maybe we should be setting rendered hidden to true inside of here instead maybe here or something I looked and didn't see one okay I'll have to hit up the FA press or something like that maybe one of these days that would be pretty funny I would get maybe a mug or something like that I could do I wonder if they can get custom metal mugs I started using a metal mug recently I got tired of the glass ones I accidentally broke a glass cup and then decided that I wanted unbreakable stuff our focus totally is just getting really jacked I have no idea why it's even changing or whatever but maybe it's just as I move this thing I don't know as we go right now okay so unfortunately when auto refresh is true this is going to spam a lot which obviously auto refresh defaults to true so at the beginning here this spams a whole lot we did set that to false of course this has closed again okay so here's hidden that's after the first interestingly we had two refreshes there which is or at least we had two finish refreshes but we have two tile grids yay okay so one of them is the line and one of them is the rectangle that's the two that we got here okay fair enough so then we printed hidden then we got two more two more refreshes from this no from I guess from the next one from this one I suppose printed hidden we set hidden to false we printed visible these two came from this do we need to change rendered hidden when we become visible I think we need to set rendered hidden back to false if we become visible so when we get set hidden then rendered hidden goes to false the idea then is that we get rendered and then we set rendered hidden to true and then at that point everything stays the same and just waits for the user the user could come along and change hidden to be false again so that our thing becomes visible but it doesn't actually matter because the place where we have the logic I thought was inside already another place where yeah this is only if it's hidden do we no wait wait this is where we change rendered hidden where do we check rendered hidden that's in group don't have any check for hidden here and our first visible does actually work so when we get to here our visible does actually succeed visible and we can see it at that point when we go hidden and when we come back around a visible again that is when we don't get to see it and that is because rendered hidden was true but we should be visible we should be visible so we should be ignoring rendered hidden but we aren't so yeah I think when we become visible we need to put rendered hidden back to false that way it will render next time so that's going to be where it's going to be inside set hidden and set hidden from parent once here so basically we're setting it to false when we become visible what we're setting it to false no no no okay yeah we're setting it to false when we become hidden when hidden becomes true hidden backwards with I keep swapping between saying hidden and saying visible and obviously those are opposites so when we become hidden we set rendered hidden to false when we become visible we should also set rendered hidden to false basically should always like anytime our visibility changes at all we should set that to false because only when that is false does it render so this should almost be called not even called rendered hidden this should be called like rendered verbalizing or whiteboarding things slows me down and helps me think through it yeah oh I'm curious if the camera wasn't on would you still think out loud I do that a lot sometimes sometimes I talk through stuff more or less the exact same as I do on the stream and other times I sit silently and just kind of ponder inside my own head only yeah I will say when the camera is not on and I do elect to speak out loud I tend to use a lot more different accents weirdly enough I find myself making different accents as I'm explaining stuff out loud when I'm alone not when I have the camera on is really an odd thing but does amuse me sometimes we got our build we're in bootloader mode we're going to copy that one finished off the coffee yeah good question I do think it helps like the times when I don't do it are not necessarily because it's like I'm not choosing not to do it because it isn't helpful like it definitely is helpful I couldn't even tell you like from any given time if I do do it and I don't like why but it's the truth that sometimes I do and sometimes I don't and I do think it is very effective tool kind of rubber ducking so to speak I don't actually have a rubber duck although I do have a little Lego robot I do have a couple of little fidgety things on my desk that I can talk to this one worked I think maybe that did fix us up let's go back to the we want to go yeah let's go back to the big one so we have a couple of minutes left I'll probably be ending here in around ten minutes or so what I'll do is let me copy this code this is one one line one rect one rect I'm never going to remember what that means different personalities know about different parts of the code right you got you know your front end developer cap your back end developer cap your database cap your UI, your CSS style cap you got different personalities know about different parts of the code is how it is for me or like in my quote unquote in my day job my Tuesday through Friday day job different parts of the code might be like client code versus server code one line one rect let's go back to the one that does a lot server finish failing to skip not bad it would be worse I do want to do the one where every line is hidden instead of every other line because what we saw is when it was every other line it actually didn't wasn't as messed up and actually have that iteration thing I notice doubled up on the comments here how does that work put it back to a lower one because I think this is going to be slow again oh we need to we actually need to loop back for line in line group it is not defined I'm glad this left error there I don't know why this is like still going under is the screen disconnected that's weird I do have a lot of the prints are back in there now so render all this there we go see last time we didn't get this bit down here it's slow as heck we've done nothing we can't I don't know if we can't do anything but certainly in the current attempt I have done nothing to try to make that faster but we did want it to render at all I will say it, it's awfully slow it feels like it's doing pixels that it doesn't need to it's going all the way up here that's what it feels like it's doing and if that's the case then all these pixels that are in this top parallelogram thing here those are essentially wasted iterations because it's already the correct color I don't know if there's some way it could be skipping somehow I don't know if it it's like it doesn't know that those are already the correct color somehow even though this is slow I do believe that it is still showing the fact that our logic is fixed because with the broken logic we were not drawing the part of the diagonal lines that falls below the colored rectangle and with our new logic that was setting that thing to false it now is correctly drawing the parts that are below the colored rectangle I'm going to have to finish that one now we get a couple of fast ones but then after three we'll get another slow one see it's like it zips all the way but then it's like much slower down here I wonder what that's about I will say it was way faster to hide it than it was to show it as well which is interesting that's so weird because it does it does the entire colored rectangle section first including drawing the white though right like not like it's drawing colored rectangle and then white on top like conceptually that's what it's doing but when it renders it's actually the white is included in the first render and it seems to be really snappy but then this bit down here for whatever reason you know that's honestly that gives me hope the fact that it does this part contained within the colored rectangle it does the white bit of that so fast gives me hope maybe that there is still somewhere where we could change the logic in the core and make this go quite a bit faster because I would assume we should be like the speed at which it was able to render this bit up here I mean I have to think it should theoretically be able to render the lower part at that same speed but we're obviously like way way way slower right now for whatever reason so like that's something maybe to look into at some point is see if we can figure out like what is the difference between when it paints it the first time versus like whatever it's doing here in the slow time let's take our prints back out and then let's pre-commit and commit this one again so take the prints out yeah rendered hidden it might be a good idea to change that honestly at this point I'll go ahead and commit it as is here and then we can always take feedback on the PR as well but it's no longer like in its original conception the idea was when you became hidden you would set that thing to false and then when you got rendered you would set it to true but in the current one it actually does need to get set to false when you become visible in addition to when you become hidden so our change here should be oh we can get rid of that our change here was adding the rendered hidden false oh we don't need to do this inside of the if statement though right we could take this out yeah yeah yeah let's do that I mean honestly set hidden by parent calling this I don't know maybe it doesn't matter let's leave it this way but we can do this outside now right because now we have this on both sides of a if statement we need else we really need else in fact we just always set that to false right that should be the same logic that should be the same logic check this one out I'll run pre-commit and commit and then I'll be wrapping it up for the night so if anybody's got questions comments anything like that now's the time to drop those in the chat I will say thank you to everybody for watching it's been a pleasure hanging out with you all on this Friday evening afternoon I will be back around tomorrow morning over on my own channel so if anybody is interested in some more oops we actually copying if anybody's interested in some more circuit python live stream tomorrow you can catch me over there on twitch and I also am in the same live broadcast channel on discord and I'll drop links in there when I get started also so you don't have to follow me or anything like that I can drop the links in and you can always just get it from there tomorrow I do think I'll be working on something different so if you are interested in that you can join me then I'll say as another quick sneak peek so to speak it will be a project which is related to Enigma machine which was an encryption machine from World War II era so I'm working on some code that implements one of those and I think that's what we'll be working on tomorrow I may still change my mind but that's as of right now what I think I would like to be working on tomorrow I'm going to let this one render a couple of times just to make sure that it's actually doing it correctly since we made that change while that's doing it let's run a pre-commit and then a commit we'll check here so our difference was we moved this outside the if statement we got rid of the else because we were doing it on both halves of if statement that's our only change here perfect so that is now set rendered hidden false when hidden changes always to signify that we're doing it always out of only half the time make sure the stream always interesting thank you for hanging out as well as Patrick as well as everybody else ok hit there so we'll see if that one passes I mean I assume we'll still be failing on those two look like we found two that are a little bit too big so those ones I don't have to either turn something else off or figure out some way to save a few bytes in order to get those two working which probably all of the above options are going to be fairly tricky so I may need help from somebody else on that one I'll let this get to the end here and then look good there we go yeah and the fact that we're still drawing the bits below the colored rectangle is telling me that this version does those behave correctly at least with regards to that it might be interesting to do some profiling like this version versus the stock version I wonder if the timing is any different although the stock version doesn't what does it should render it correctly yeah that might be interesting to compare anyway I will take off for now thanks again to everybody for watching hope everyone has a good evening all of that I'll be back in the morning until next time