 All right. Hello to everyone. Happy Friday. Let us just get a These windows sorted out over here get the chats Where I can see them We'll put the page this one Let's go and be autograph Okay, I think we are There you go So yeah, hello to everyone This is the deep dive program. My name is Tim. I go by phoma guy on github and discord This is weekly live stream program where we are Diving deep into circuit Python Sometimes it is into the circuit Python core the actual C code that is the circuit Python interpreter Other times it's libraries or infrastructure or projects It kind of varies from week to week just depending on what is going on I will take a quick step back though for anybody that might be new if you don't really know what I'm talking about You can learn more at the main website for the project, which is here circuit Python org Basically circuit Python is an implementation of Python that runs on tiny computers called microcontrollers There's a bunch of pictures of those devices over here on the downloads page All these devices essentially have a little computer on board and when you run circuit Python on that computer You can connect this device to your actual computer like your laptop or your desktop computer and it will show up like a thumb drive on that thumb drive There will be a Python code file. You can edit the Python code that is inside that file And the onboard computer on the microcontroller here will execute that Python code for you This allows you to interface with all sorts of electronical Widgets, you know essentially, right? You have a lot of input output pins Which you can use to connect to other hardware and different ones of these microcontrollers have different types of hardware built in So like this macro pad right here has got some cherry MX keys as well as a little display and a knob Whereas this one down here has got a TFT display and a touch overlay I think it has maybe like an accelerometer a couple other things so these things come in all different shapes and sizes and the circuit Python code that We are working on it runs on all these devices and allows us to build projects and interface with other electronics Goodest can be expected nice glad to hear it How's it going over on the discord to Paul Cutler and the shippu good evening to both of you our evening in my time It's really not even evening yet. I suppose it's afternoon, but good day nonetheless But that's a little bit about circuit Python itself Circuit Python is an open-source project. Anybody's allowed to use it. Anybody is allowed to put it on your own microcontrollers build your projects with it. You're also allowed to Port it to your own devices So if you're a hardware manufacturer and you manufacture the device you are allowed to you know port circuit Python to that device If you would like it's free of charge The thing that keeps the lights on so to speak in the circuit Python project is Adafruit This is their website Adafruit.com as you can see I got a little distracted by this one which I want to look into in a minute Adafruit is a hardware and software company based out of New York And they are the ones who are primarily funding the circuit Python project So they pay the team of folks who works on the project full-time There are also folks like me who work part-time on the project and are paid to do so by Adafruit so of course big thanks to them and then also Thanks to anybody who wants to purchase hardware from them by doing that by heading over to Adafruit.com and purchasing some hardware there You are also helping support the circuit Python project So this one caught my eye, which I want to look at for just a second This is completely unrelated to the rest of the stream, but this is kind of a cool thing. It's like a safe Almost looks like a safe wheel or something this feels like you could use it for like a Kind of like you know pretend like Hollywood safe cracking scene or something right like if you wanted to pretend like you were Cracking a safe. It's kind of a fun little Maybe like in an escape room or something like that The mount rotary encoders I guess it doesn't know absolute position though Which would probably make it difficult to use as a safe although, but it has numbers on it, right? Maybe it does know absolute position I'm sure if it actually knows the absolute position or not. Anyway, that's a neat little thing. So What we will be working on today is in Library land. I am gonna be taking a look at this PR that Dan worked on a while back About a year ago now at this point back on the ESP 32 spy library This is the library that is used for the devices that have a built-in ESP 32 as their co-processor Such as for instance the PI portal devices See we could maybe pull this up here and get a look at the back of it as well where we can see the actual ESP 32 on there has a going skir. Good evening to you So The ESP 32 is here and then this device has an at mal samd 51 as its main Main brain that runs circuit python and then that communicates back and forth with the ESP 32 the ESP 32 provides it Wi-Fi access essentially so this library is the library that handles the communication between these two chips the main samd 51 and the Co-processor the ESP 32 Dan made this PR to try to change the API for the socket inside ESP 32 spy library to match the API for C Python which also I believe matches the API for like built-in Wi-Fi stuff like an ESP 32 s2 for instance Or s3 or pico w But it turned out that With this change we actually had this problem where certain code when it was trying to download files The chunks would get out of order Or whatever is going on here, right like it would download the file and it has all the right data, but it's not It's not stitched together correctly, right? It's getting messed up. So We are gonna try to do two things today I noticed this has merged conflicts now. So we're gonna try to Merge main and straighten out the merge conflicts and then if we can get that done we are gonna try to Resolve this issue and the way that we're gonna do that is basically what was suggested in here, which is like These so this is all from memory will dive into it as we get going and so maybe I'm wrong here But from memory what I recall is that These downloads are occurring through the pi portal library which uses like portal base as the main library And I think the change that we will need to make is inside either pi portal or portal base We will need to make a change to handle When the buffer size is zero like when this thing returns zero or not There's a behavior that was supposed to do one thing, you know supposed to being like in the CPython API it does one thing and the the built-in Wi-Fi it does one thing But then on the ESP 32 spy it actually used to have different behavior in this particular case That different behavior. I think is what's causing this so if we can modify portal base To expect the correct behavior, you know quote-unquote correct behavior the behavior that it has in this PR Then it should be able to go and download the files correctly When we use this updated version of the socket if we can get all of that done Then we can get this to the point where it's actually ready to merge which will be really nice because having the socket API match The other devices is really nice that way more code is portable between them and I have my fingers crossed I don't know if it's gonna work out for certain, but I have my fingers crossed I'm hoping maybe that if this has a compatible socket it could also potentially be used on Or I should say with the HTTP server library, which I think would be a nice thing as well Has gone Paul SK was in Kentucky until last Sunday Break off my stay. Oh, no, that's terrible. I'm sorry to hear that had a return immediately So I was presented at the cremation. I have time to be present on the stream. Wow. Yeah I mean, that's my heart goes out to you. That's terrible news, but It's good to see you and hang out on the stream here. Nonetheless And a lot of that lately. Hope you're doing well. Yeah, really agreed. Actually, I just learned about a person I used to work with that passed away this morning as well. Yeah, it seems Seems like that's coming up a lot these days unfortunately So first thing we need to do is just try to get it up to date Which is gonna involve merging main. I don't I probably I guess have this locally, right? Opened up here. We'll go this one. I am currently on main. So we'll update main Origin main. Let's fix that get remote dash V origin is a to fruit get remote rename origin Hey to fruit get main updated Got back from visiting my mom in the hospital. So yeah, yeah, it comes for all inevitably one way or another Is the truth? but uh And appreciate what you got while you can for sure so we'll need to add this as a Mote we're gonna want to have this as a remote. I usually try to match the name of the branch with the get hub Although in this case, I'm doing a shortened one for Dan H because I kind of I think of his name I think is how it is in the Scored maybe Let's go now. We should be able to go I think if we check it out. Okay, we want to go To fruit. I think since I have the remote Now I can check it out this way and then if I go and push I think it will actually push the right place so let's go Gotta start merging main into this got our conflicts, which we expected because that's what get hub reported and so we're gonna try to Solve these as best as we can Take a quick peek through just before I start going Okay, so inside main There's an extra line here, I don't know that that matters One way or another truthfully I guess we could pull it because it is in main and main it looks like these were changed to runtime errors Gonna take those this one was actually changed to a connection error Here we start having some major differences. So So from main the change was OS error right here But from our compatible socket over here. The change is much more substantial. It is essentially I mean it seemed Like right is gone have received Must be further down. I guess So I think what we're gonna do is on this one We're actually so it's kind of nonsensical, but what I'm gonna do is I'm gonna pull in from main but then I'm just gonna Go right away and also pull in That didn't do what I wanted. I undo in that case. I'm gonna pull in from here Then I'm gonna X In a X from main, but we're gonna keep in mind that OS error inside of read line we're gonna keep that in mind if we find a read line over here eventually We will try to swap that back so that we keep the change from main, but for now I'll go X We'll grab from left Okay, here we have Changes for both. So I don't know about this one. So Strangely, this is a zero here. I'm not entirely sure. Oh, well, there's a variable right above it. I don't know. That's num read invites Num to read gets done down here Guessing that we will want this I suppose and then probably also want the underscore available My assumption being that available was probably removed as Not being part of the API Mention available though Yeah, this is the difference in behavior right there receive reading as much as possible when buff size is zero That I think is the crux of our pictures here Make other non-standard methods be internal named with preceding underscore So I think available is falling into this category Has gone X will Magnus Compatible socket sound interesting is ESP 32 spy just for airlift. Yes. It is Airlift stuff. Yep. Pyportal, airlift feather wing All of those kinds of devices Go magical on there so that we get the zero and the underscore available So read has changed actually here Be gone. I mean, I'm guessing we'll find read somewhere else or maybe we already passed it I don't know underscore available Sock num gets the reverse action actually sock num becomes non-private really Nected is back the other way Available we saw already Sock num. Sock num changes, okay Little confused why we just don't have like read for instance was read not Was read not part of the API You'd remove dip. Okay. Yeah, just got a Just got to do what the name of the function is and read a little bit remove not deprecated non-standard methods Read write read line. So those are all gone and actually read line That was the one that had the change to OS error. So if that's gone, then we know we're not actually looking For that change anymore Usgi server. So this one WSGI server Everything works out, right? We would just not need this code anymore because we could use HTTP server library Sure. Why I guess it got deleted just because Yeah, okay, enough that does that count as deleted file or is it gonna leave an empty copy of that file? This is another just WSGI server. Okay. That was it for our merge So We want to do some sanity testing to see if I broke anything with my merging Very possible. I'm gonna go ahead and update my pi portal, which is a pi portal A tenno and is upside down Very hard to see I guess the lights just too much for right now for some reason. Oh, that did not help actually It's probably about as good as we get Copy that. Oh, is this built in? Is this built in? We need to make sure this is not here we go We can do that by euro buff size Okay, yeah, I think it is gonna be built in Because I think if it was not built in I would not have these in the root of my circuit pi drive And in fact, we don't have it over here In the lid folder So as of right now, we should if it was not frozen in We will know right here is where that okay, so what I'm gonna do is actually delete this one I have no idea You want to keep that? I have no idea what it is. It's the thing. That's probably No WSGI would be better for web applications like Django Let me buy be better for HTTP server I don't know. I mean I look at them as essentially the same They both essentially give you the same kind of thing that WSGI example as well as HTTP server library I don't know if there's any differences. I'm not sure if WSGI is like somehow different in some way, but The examples essentially end up working roughly the same although HTTP server because of all the new features and it has a lot of More interesting stuff now That the WSGI server would not have made as easy especially things like serving the files and Doing post requests with get requests and doing Indication and kinds of things truthfully lots of good stuff over there This still does show here though, right? Did it not update? Okay, there it is Okay, so the fact that this import Succeeded I think tells us that ESP 32 spy is Frozen in through this build Yeah, and we want to make sure to definitely use the root of CircuitPy not the libs folder also gonna go ahead and update the Circuit Python on my device Loader this is a way of avoiding copy pasting running into issues. Oh, let's see. I missed a couple of the chats here Push to your branch and check out Your commit let's push your branch check out your commit. I'm not sure what that was to Bsgi is meant to be cross-platform compatible, but I mean HTTP Yeah, I mean as far as I like I think they implement the same things The way of avoiding copy pasting Have you used WSGI? Yeah, well requests is the other way around right requests is the client whereas WSGI and HTTPServer both of those are like the server you would send requests to those servers So request is like the client side whereas those other ones are like the server side If you're wrong, but isn't WSGI working on some microcontrollers as far as I know I mean it first of all, what's that Michael Pooke used to? Thanks for all your Updates to the HTTP server library As far as I know WSGI this one I mean it only works on things with the airlift right so things like the pi portal Well all three of the pi portals the matrix portal the Mag tag About it that oh no no mag tag is is s2 actually excuse me not mag tag. Yeah Couple of those though the pipe portals are the main ones truthfully I to the best of my knowledge This only works on those Whether or not within that realm of those airlift capable devices like are there some of those that it does not work on that? I do not know the answer to if that's your actual question It could be I would not be surprised if it wouldn't work on like Samdi 21's I would not be surprised if there's not enough RAM But I also don't actually know Saying push the merge changes up to your branch Go to USB sub module directory do a get pull check out your branch to you push to you Then you should be able to build with your changes. I think I can build with my changes. Yes, sir I'm not sure I follow. I think I can still I'm I can build with my changes for anything that I need to do I mean right now we don't necessarily need to build. Oh You're saying like to remove it from frozen. I think maybe potentially Like make a new build of circuit Python without it frozen in potentially that's what you're meaning Which we could do in this case. I will just like if you paste it in the root of circuit pi it will override the It will override the frozen in one so like and we could double check by saying in here like print This print this That shows that we're using this one definitely Remember having issues WS GI Yeah, I that would also not surprise me like anything that uses ESP 32 s2 would be the built-in Wi-Fi rather than the airlift ESP 32 spy. So yeah, I mean like I Would not necessarily expect them to work. I don't know for sure Maybe they do maybe there's a way to make it work that part I don't know but I would not be surprised if it didn't Also, I was doing something wrong and ran into this use when I was because I expecting to work more like it should be server That's how many server stuff. Oh one line F string. You should be server. It's coming away Yeah, it should be it should be a server There now we can do a let's grab a simple test action here, which Is there I mean, is there a simple test in the examples here? Or do we want to be going to requests library? Maybe This is For a simple test that should be fine. All right code that's here is what? This select example. I'm pretty sure this is in the repo. I don't know you need to keep it I guess using airlift only stuff in our f 52 840 chips last year Last time this year but last year this time Still did it wrong. Okay. I'm giving up after that, but I think we got the right just at least on the last one This is looking in secrets pie for Wi-Fi credentials. I don't know if those are there. So I'm gonna check on that real fast It'd be fine. Okay. I'm gonna delete some of this, right? We don't Stuff yeah, the firmware version comes into this a bit as well. I wonder if there's a new firmware version Nice, okay. Yeah, here we go. We got basic tests working fine this one includes a get request or File that has a .html extension that actually contains raw text and then a JSON File does not have the post Maybe let's see if we can do the one with the post as well because so far those would be only gets AIO post RTC try that let's try that Road time API It's 2 and s3 have only recently gotten sort out to be stable. Yeah Yeah, they were brand new this time last year or closer to brand new definitely Rocky for a little while definitely so Looks like this just updates once a second. Is it correct six six nine and then 1632 Which is local time. Yeah, 1632 nice All right, let's grab the one from requests. Whoops. So far. We're all good Well, I think the simple test in here is got the post actually Yeah, it posts data to like paste bin or something, right? It's 2dp been running HTTP server on a pi Pico being particularly challenging Well, I can tell you for the last like We can I have to two weeks for my Quote-unquote day job the day job that I do on the days that are not Mondays. I have been working on a Device that's based on raspberry pi Pico that does include a web server as part of it Although I did not really start the web server portion until very recently For what it's worth thus far, I have not really had any trouble running the web server on the Pico Using the HTTP server library. It's a very basic web server But I have been running it on that Pico device Without any issues knock on wood. So hopefully we keep going though. We were able to post the 31f Up to HTTP been and then we were able to get the data back out and it contained the data that we posted perfect same thing here we posted some JSON data and then Got it back out afterwards and it was still the same data. So awesome. So our posting is working our getting is working So far so good. So really what we will need now is the Broken example, which is I suspect the broken ones are basically going to be anything that tries to download a file that is Quote-unquote too big whatever too big is I don't know the exact cutoff size, but if it is Quote-unquote too big Then it becomes a chunked response or something. I think this is kind of my guess and When we do the chunked response that is when we are having the Trouble that's leading to the Scrambling here. We need the code for this You have been to know whether there are some differences between Pico and S2 that might impact SSL Well, one of the issues that somehow adapted HTTP server on Pico to use HTTPS, but I was unable to reproduce I have not done any HTTPS stuff I've been I've been sending requests out of the Pico W2 servers over HTTPS I have not been doing any requests going into that Pico server over HTTPS. It's always been HTTP for me I Don't know how you would work out the cert In order to be able to do it HTTPS. I'm sure I would imagine it's possible somehow, but I don't know Wisnet yeah, Wisnet's a little bit separate as well. Wisnet's over there on the Ethernet side of things Project will use your quest with us. So yeah server side. So you got same in thing going as I do so far at least Yeah, that would be interesting to be able to do the HTTPS into the server I don't know how I don't know how you could keep your cert up to date though. It's like Be tricky. I wouldn't know how to load the cert in there and then I also wouldn't really know how to To keep it safe Okay, I think we only need this if we don't have an SD card I'm pretty sure if we have an SD card, we don't have to do that thing right and save boot safeguard mechsism Rename boot pie Maybe you do have to do it. This doesn't say anything about SD card. I used that before when I tested this I don't remember having to do that, but maybe Let's take a look in the code a little bit Once it was none circle fill Retrieving URL response equals fetch response is response Loop count plus one This is all try accept if we catch any of these errors just say an error occurred Increment the error count if the error counts less than 20 We keep going if it's greater than 20 we quit It does time dot sleep for a full minute in between retries, which we might lower if we start getting errors error count time Okay, if you touch the screen then it fetches immediately So what is doing I guess fetch I guess I Based on path Based on path title Default background image JSON path. Okay, let's trace this code through a little bit here I'm not sure what is Where the actual download is occurring and we're gonna need to find that ultimately so we may as well Even before we run it. We may as well start looking for that Actually have fetch so image Based on path is Variable it used It found Though I think we need to go to network base the network base looking about the Yeah, the service spots from Pico may not have any room left Yeah, the SD card maybe could help if you have an SD card, but the Pico itself is pretty small Want to run better encryption you had 2048 You probably choke it out Honestly, I was doing MD5 hashes on the Pico W and it was taken like almost a full minute to MD5 hashe Maybe like 20 or 30k file or something. I got to the point where it's slow enough I didn't even bother anymore And the issue is talking about 70s every content of our HTTPS Yeah, that would not surprise me If it is way slower I don't I should be told like it's all lower like I don't really understand the nitty-gritty of any of it But like I was saying the just the MD5 hash was like really slow So I would not I would assume that any sort of modern encryption algorithm is gonna be probably at least as CPU intensive. You know what you really need it is a You almost just need an ASIC for that right you need a chip that's designed to handle the crypto for networking And then if that thing does all the hard work you free up the main processor the Pico W to actually be able to Exist and not get choked out pretty much able to use it on this to author claims it work on the Pico Yeah, I mean the thing about that is the S2 Its entire Wi-Fi stack is different like to the best of my knowledge It's supposed to work the same in circuit Python, but underneath the hood, of course the ESP based device has the ESP IDF whereas the Pico has No, I guess Raspberry Pi SDK. I don't even really know what's under the hood on the Pico So they're supposed to work the same at the circuit Python level to the best of my knowledge But it is completely different under the hood so I Would also not be shocked if there were cases where it doesn't work. Yeah, it is interesting though That person was saying they got it to work with HTTPS That is definitely more than I have even attempted with it It's pretty cool though if they got it working This doesn't really so use local response fake request if not response connect Select response Wi-Fi requests dot get Where does it where does it use the JSON image path What was it called? It's Jason path So I have this at all in here data Did we did it call fetch data? fetch Just fetch We have nothing about image json in here We I guess need to look on self dot underscore Wi-Fi, but that I mean surely this is just gonna be requests object, right? But where does our processing of like finding the image json? underscore Wi-Fi Wi-Fi dot connect Wi-Fi module Network base whatever got passed in I need to clone these and you stop looking through here I mean, I guess we probably have them up. I'm gonna go ahead and run it actually here in a minute though Before I keep chasing the code when I get them opened up, but then before I just keep going through the code I'm gonna Go ahead and run it like maybe we get lucky and it just doesn't do that anymore. That'd be pretty nice. I doubt it the case but If by some miracle it is it'll be nice not to need to sit down some more It's an official See here or practical way to be use external Yeah, definitely that's what I would do like if I were trying to actually connect Https stuff is put an nginx in front with another server Meaning late just wrapped up two-week carpentry project so we can get back sir Python and PCB stuff nice and nice to have you back I think there's official documentation that says you can't do it Get it working you're gonna be slow and hang a lot. Do we have pipe portal library here? well, it does definitely seem like Wanting to be downloading directly to Built-in storage. I don't where does that get to find though, right? Like that behavior is not in here anywhere It's like inside fetch. It's like the only place it could be for downloading, right? I said out of brightness Probably we have a very old pipe portal library. Oh, but that's I installed a new I ever installed the new circuit Python a point two beta zero. So we downloaded So but pipe portal library is built in as well. I would Have assumed incorrectly. It turns out. Okay, so maybe we have an old pipe portal library Delete that if I was honestly a good idea There's basic basic off But still travel is plain text. No SD card found Maybe if we plug in an SD card, maybe it'll use that. Alright, so we grab We grabbed an image path out of the original JSON we got We then converted it to an AIO URL Which is basically gonna make us be able to convert the JPEG image into a BMP image But we then did fail no writable So now it's gonna probably just keep retrying so we should probably just That actually and what I'm gonna do is put in an SD card and hope that it will download to the SD card If that's plugged in because I would rather not Change the built-in storage to writable I Can get away with it. I can get away without it All right, cool this time we didn't get that message So I assume that means that it found the SD card and it's gonna use it there be well, that's living There we go, and we did Get a messed up image. It's got that same sort of diagonal kind of blocky-ness thing going on It also seems that the colors are jacked up on the top half. I'm not sure what I'm sure what's going on with the the messed up colors But I kind of assume that that's all part of the same thing like fingers crossed. I'm hoping when we fix Whatever we fix in here that it will Fix that too, hopefully I'm not secure. Yeah, I mean definitely not secure if you can see the network traffic Yeah, for sure because like you say it's all plain text Right, so if you have wire shark on the network or made in the middle like if you control the router or anything like that Like you'll definitely be able to see everything They were out of stock Chargers. Yeah, chargers. No itsy-bitsies Gilly is coalescing about trying to make a mesh candle system You create a billy hub with an RPI to control multiple devices. It's a question. I don't know In Raspberry Pi connected multiple through Bluetooth. I'm not sure now we do unfortunately go back to digging here So I've fetched it all. I guess because it's just in base And click through which is kind of annoying Right nice if I wonder what I would need to do to get these loaded to where they're like Maybe they're both in the same one or something Using local so this prints this and we did not we did not see that print We do not see that so we did not come in here If not response which would be true because remote response is none here. We did not go into here Therefore response has to still be none here Necked retrieving data. Did we see that or retrieving you retrieving data? There we go And then it says reply is okay Is that from codepie or okay? There's our retrieving URL Which then connected inside We don't it does not look like reply is okay is in here reply is okay from Interesting, where's this check response? How did we get into check response? That's pretty weird, right? So fetch It's the response from here and then returns it It doesn't ever call Doesn't ever call a check response. How where does this get called from? First data, where does this get called from? fetch data But then fetch data just calls fetch But our code is not calling fetch data is Just calling fetch Get into fetch data Wait, what where is this in it? Little base in it. Oh, this file is big. What class is this portal base? so portal base class Has fetch but network base also has fetch Perform that should be perform Which data from the URL we initialized with perform any parsing So it would be self that it would be portal that network dot fetch Would be the network fetch, but we're calling just portal dot fetch Which means we're calling the one on portal base. Okay, so I'm caught up here. So Inside fetch here we say network fetch data Which inside network fetch data we do Call parse data Which calls check response which prints this thing? Okay. Okay. Okay Where did we download from because something is like downloading that image? If content type equals JSON Content type equals But what if content type equals equals JSON Content type equals equals content JSON and then we have the same thing here interesting in our case Process JSON is what it will be reverse See this is gonna be getting tricky in here because somewhere in here is where it's Pulling out the image JSON path checking out WLAD a little bit As well, but I ended up making a circuit Python version of something kind of similar use a pin JSON traverse JSON transform Optional JSON post-processing apply any transformations. I think we don't have do we have transformations? I don't think we have transformations. I don't think so I think there would be like transform something one of these would say transform We have image dim JSON path We have image JSON path I don't know what the difference between those are either does pipe portal docks have a This on traversal path for a background image to display defaults to none this on traversal path for the original dimensions of Image tuple used with fetch defaults to none Dimensions I Guess dimensions. Why would we need the dimensions? I guess we need the dimensions When we send it to Adafruit IO to Invert it to bit just got it. Yeah, it's got a height than a width there. It must be we're doing those It's kind of weird though, right because the server could find the height than what they think Without us telling that's all right It's on traverse values It's on data first down the specified JSON path and return the value or values those values here have The image image path on it This is not going to do me any good to change this here. Yeah, we're inside process JSON inside network I'm going to print values out so I can see if it has the image path in it So I can RF it to a 40 has BLE 5 Possible not going to go down in our fsdk revival past Don't know hardly anything about Bluetooth. Oh How to reload in there I might turn out a reload off. You never saw values. Oh, there it is. Oh Hey, that was Not at all what I expected to find inside values It's basically what got put here Hmm Okay, well that does not have our image in it at all. So Now we need to look in the image things. What was the if image JSON path image URL equals reverse If image URL original image which that we saw printed I think Just being on this side now apparently original This is original URL Yeah, original URL Inverted URL this last night. I was streaming a little bit. It's one little moth thing. We get these like Any little black mods that look like hearts they show up usually like late spring and they last for a couple of weeks and Whatever reason this one has been interested in me while I am trying to stream Uh Vert image Vert URL Okay, here. We're finally getting somewhere. Okay chunk where hold on. Where are we in here? This is inside process image So we have chunk size 4096 if SD card file name SD card file name Also use a smaller trunk size. I don't know if that's still relevant. This might not be the case anymore I don't know I've never Never seen any kind of issues downloading stuff. What it's I guess chunk size. I mean, it's not the full file size I guess but Self that W get so W get then is probably so can we skip all of this? Can I make a Can I skip like all of this? Can I say? Vert URL fetching stream from can I just grab this URL to file Fetch test for L is this I don't know if I need to give it all the rest of the stuff or not. Is W get Look like it returns a response oops Seem required arguments File name chunk size. Oh, which I need to also initialize SD card though, right? staying in around Closer to have you extra tuning in Not quite ours. Do not They don't don't blend in quite that well If this thing had just these black little parts that kind of look like the humps on a heart The ones we have are super tiny and they basically just look like tiny black hearts Their silhouette looks almost exactly like a heart Got it lantern fly. You know, it's pretty crazy looking. I think I tried to look up what they were at one time I may have found something but I do not recall what it was We need to initialize the SD card I can I do that actually will it do it for me? Maybe we'll do it for me Maybe I'll just give it the file And if we don't in it, right, let's give it a try So we say URL is that let's what is it file name equals? slash SD slash Downloaded dot BMP chunk size I don't know we'll do 512. I guess that's what it was doing and I want to run it again. So I'm gonna go like this So I don't think it will actually display Best of my knowledge. I think it will just download If it works it will download none type has no attribute get and type has no attribute get So we probably need to initialize more than just pipe portal by itself How we do that little base network To 78 it's an invasive species from China. Oh, wow, it's hurting some of the crops like soybeans ours are a bit different that little guy looks like it's almost Got like a pretend ladybug on it like it's trying to mimic a ladybug with its back there So this is saying self dot underscore Wi-Fi dot request dot get so none type has no get so Requests is none. It sounds like I'm also gonna make a file just called f Inside f it's just gonna say Import fetch test just say import f Gonna get milled in robotics team created a robot to clean trees of their eggs. Wow Sounds like a very difficult robotics task. There we go. We did so Wi-Fi is not none But requests is none. So we need to figure out why requests would be none Anyways, that's tricky. I don't even know what is self. Where does self Wi-Fi even come from? mine self Wi-Fi equals Wi-Fi module which got passed to us for network base, but now where that happened, right? Say we create a pi portal object. That's what we did. So pi portal extends portal base It's passed a super init network Network is network Extends network base Are we not getting network? We have network? I don't know if this will actually print anything Seems like we have a network object Network gets passed into super init, which is portal base So when network is pi portal network and it gets created passes along Wi-Fi to super Anything with requests base problem, self.network equals network and with requests Network base this gets Wi-Fi module comes in Secret's data request equals none. Okay, we're getting close request equals none Where does it get not equals to none? Wi-Fi.request Next, makes a lot of sense It does Like kind of red moth I could find these one ours are black. Oh, it looks pretty sweet though. Our portal has no portal.network Wi-Fi request it's none false. Okay Nice we are downloading it looks like So yeah, and then in this case like it will save it will say after wget and then nothing else will happen Why don't we go ahead and make the old on-disk bitmap though and show this on the display so Import display IO. Let's make a main group equals group. Let's make a on-disk ODB equals on-disk bitmap. What's this one? It's a bitmap Well, it's needs this yeah, is it is it file file okay slash SD slash Downloaded dot BMP. Yeah, I think that's it tile grid we need tile grid equals display I O dot tile grid that's gonna be bitmap equals ODB pixel shader is ODB dot pixel shader Main group and the tile grid display Not show main group file true Display what's the built-in this? Hopefully this will download the file and then put it on the display and we'll see the one with the messed up blocks or whatever Because this is a much smaller reproducer, which we can then start modifying inside of portal base slash pi portal To try to get it fixed without having the entire whole Cleveland art Program in this case will have much less so 19 ODB file does not take any keyword arguments All right, there we go perfect. We've got our Big Zaggy weird rectangular stair steppy thing going on So now we just need to fix it Where did we last leave off in here? So inside of W get side of W get fetching stream from url Reply is okay Debug maybe we should turn debug on each chunks That's not is that part of requests An iterator that will stream data by only reading chunk size bytes and yielding them when we cannot buffer the whole data stream sounds like Wishing that we are in stream bool false And we'll determine if we buffer everything or whether only to Buffer everything or whether to only read Only read only when requested your request or just all it just says is that stuff So is it inside of requests library inside of request library, but the thing is Request library works with your devices I'm just gonna open this locally. I don't really like browsing github anymore Honestly, I think this change where they have changed this it's more functional But for some reason I don't like it as much. I can't quite like put my finger on it, but I just don't like looking at code in the github front end anymore It's like it's weird too because it's pretty similar to this and like it's more functional than it used to be but You know for whatever reason I just don't I can't get behind it. I don't like the new way for some reason West stream Not socket. It's called self that request Wait, what? What is this? Okay Throw away To read receive into buff It equals zero looks potentially relevant Throw away though. That's kind of odd Uh Yes, this is such a big thing. Um Well, so session dot request Just calls self that request Would that be though self that request I thought this was self that request F request It's inside Oh I've send request session Okay session does not extend anything. Okay session is Just itself Session session has a method called request If it requests is not session Inside of this method called request It calls self that request. Is that not itself? 25 is that not where I just was? Oh, this is redirects What does stream get used? This is the only place If allow redirects If location in response headers and 300 status code What if we don't What if we don't redirect though Last response equals response Retry count Try count while retry count less than socket get socket Send request but this But this does not take stream If okay It has adder receive Okay This is going to be relevant. I'm pretty sure this part of the code But says if it has receive Then we say result that receive Socket receive one else if it doesn't have receive then what it did was result equals a byte array one And then socket read into results If result h Things seem to be okay. So break with the socket set. That's kind of interesting. We're only reading one byte here though, right? Self that closed socket If not socket out of retries If we did the retries We got a socket to be set We read the first byte the first byte was an h We created a response object We don't consider redirex then we just uh Returned our response object It seems like stream would get ignored if you didn't follow a redirect But but also if i'm honest, it seems like Like nothing happens if you don't follow a redirect. So let's assume We were sending this request and we were not going to have any redirect So let let's say, uh, you know, it doesn't matter if this is true or false Let's say the server that's answering us Is is definitely not going to answer with a redirect, right? So we'll make that assumption and then we step through the code. So we're inside here Uh, let's say we don't have any headers Uh, proto dummy host path. Sure. Yeah, we have a valid url. Let's say no problem Uh, we'll say we're on port 80 for now Yeah, we're on port 80 Uh, I don't know last response. Let's just say it's the first time we've done it So I assume that last response will be none. So I assume we won't do this We try count a zero last exact none while so we're yep zero is less than two Plus one on there get the socket Again, we haven't touched stream yet. So we get the socket. We say Okay, it's true. We say try to send the request And at this point we still we have not done anything with stream We say we're going to try to send the request If that fails, we're going to set last exception to be the exception We're going to set okay to be false. So if that succeeds Last exec will stay none Okay, we'll stay true That succeeds If okay was true We check if our socket has received if it does we say result equals receive one It does not have receive we say result equals byte array one And we say socket receive in two No matter which one of those we do next thing we do is say If result equals h If that's true, then we break After having set our socket I assume that just means like this socket here We break Okay, we break Break would break would break out of the while loop actually here Ah, okay, that would break out of the while loop, which means we would not close the socket importantly So we would break. Okay. So now we're outside the while loop. We came out here. We did not close If not socket. So this part will not be true for us. Let's say that we got a socket Say that we do It says response equals response a new response from socket passing in self I mean does one of these somehow have stream on it. I don't think so, right Dream was here. We haven't done anything with it yet We say response equals that and then Allow redirects like I don't think it matters, right? Let's say it's true, but We said we're taking the assumption that our request is not going to return a redirect So this would be false We say last response equals response and then we just return it like we never used stream We are in requests so is stream responsible for like is stream related to The Thing getting broken apart like does that maybe only happen when stream is true That's kind of like my thought is I kind of guessing that that's probably the case All right, where is this? This is at 640 ish Okay, so now we're gonna print out whether stream is true or false and then we're gonna print out if we are following a redirect sorry Greed I don't need the left menu at least make it an option that can be disabled to be can be just haven't looked Our goal to collapse it. Okay, maybe I'll need to find that Makes it look like to get a bold we did not print any of that at all is requests frozen in Okay requests is frozen in so we actually want to take that modified one Put it not inside libs, but instead at the root of circuit pi That's not ideal attribute set socket Can't use this for fuss requests for some reason Pitch test line nine pi portal weird the portal base Weird that portal base is baked in but not pi portal I don't like it because I end up with so many At the pi portal I have another one out here Installs portal base Even though we already have one in the root, which is the one that will get This big 19 so network 90. Well, okay, and it 151 which is just going to be network network network 95 Which is wi-fi Comes from portal base wi-fi coprocessor 71 Why is it 71 and not 70? I don't know I know requests has no set socket. Is that old? Session socket equals socket pool And 2tps equals requests dot session. We pass it the socket and ssl And then later on we would say this object dot get or post or whatever So is set socket Like an old way to do that or Requests used to have set socket. Let's look at portal base Still call set socket How did it work with the Cleveland like we downloaded it several times already? We had old requests. I guess right we must have had old requests Say something about set socket That's a legacy api Use session instead But if it does exist, why does this thing? It doesn't exist What is requests? That imported import eight of fruit requests as requests Stand why it wouldn't have that then doesn't have anything weird Have anything Well does not look like we are going to be solving this evening I have learned a lot about it. We've peeled back a bunch of layers of the onion at least so I do not understand why This does not have any things like my import name response About session so so like sessions should be a thing that exists. We have import requests as requests We have requests session This has no session What is this do? I don't import eight of fruit requests as requests der request only this Import eight of fruit requests as requests Der requests all of these I do not understand why we just got two different things stream is true But we have yet to see That like location thing Never saw it Okay, so I think that stream argument gets ignored unless You follow redirects And There actually are redirects In the response to your request In our case, it doesn't seem like his case You pass in stream here It's to session back up here This one comes in as stream Just like we stepped through before I mean, I also still don't understand how this is even finishing right because We've only read one byte like where when did we read the rest? Didn't do that at all. We just returned response Back to W get Or was that that was in portal based. I'm so lost now W get So isn't this I'm lost now. I don't know where we were trying to find Fetch the fetch. I mean it was Which calls fetch data data calls Different fetch and then parse data that calls This is the one that that calls the portal base fetch No, this calls fetch data And there's so many layers in this thing. It's like Very lost Fetch data. So fetch data calls a thing called fetch, but fetch data is also called by a thing called fetch But I don't know which what is self that fetch Apparently this okay Which says request dot get where did we where's the w get? I swear we've passed a w get at some point. This one does not have stream Where did stream now? I'm like so lost again If I request get Yeah, I'm lost I don't understand where we read the rest of the thing like right here. We read one byte As far as I can tell we did not read anything else and then we just returned this back out of request, but Which then got returned back out of here As I guess response which then got returned from the fetch here, which this is The fetch that's inside network Base Came from the fetch data. Maybe I should be writing these down. I guess I'm sorry. I just bashed the microphone again. Yeah, there's no stream here either Okay We're gonna write down Why does it do that step? Oh, this is amazing. Yeah The let's not worry about cleveland art project So in fetch test we called Pyportal dot w get first Pyportal dot w get called what? Where did pyportal dot w get come from? We're just gonna list out the whole stack here And then I'll probably wrap it up pretty quick after this and we'll have to continue on another time, but I'm gonna try to write these out. So w get is Network dot w get so w get gets overwritten So when we call network, uh, when we call pyportal Pyportal Yeah, when we call pyportal dot w get We are actually calling Network Dot W get I'm gonna say self dot network It was dot get I'm gonna start writing down line numbers actually too Well, that gets tricky though This doesn't really have a line number so to speak. I mean, I guess it's In it pyportal init py 190, I guess let's call it Oh, that's The best way to do this feels so lost self dot network dot get that would be so network gets instantiated here So network is a pyportal network And we're calling get on it pyportal network dot get But pyportal network itself does not have Get by itself. It does not have a get which means it's gonna be inside network base. I'm pretty sure It would be network base Get doesn't exist. All right. Well, that didn't last very long We're like already just I don't know I don't know Am I like missing something? How can we be calling get when we don't have a get W get oh my gosh So so pyportal network again though does not have w get so we are still going back to portal base It does have a w get this is at 261 inside portal base uh, okay, so inside here it calls underscore a wi-fi dot requests dot get so that is inside of requests Dot get It's ultimately gonna be calling to session dot get I'm pretty sure Yeah get calls session Dot request session You know what's weird though is in here it's get but this just calls default session request With get as this Which means I if I understand right it means we're not calling this Instead we're actually just calling directly to this Feels like we skipped the layer When it's at 273 that it calls this one This is inside request dot py No slash just line number 829 829 So this calls So it was get it calls into requests It calls into request Excuse me from 829 which is back up onto session I'm pretty sure is going to be back up here. This is the session Dot request session dot request is at line 625 And inside of here what happens? Okay, that's this is this one. So we have our retries We call send request I'm like getting messed up with the lines here, right because this is where this is called But then this is where it's Plared I'm not being consistent Just gonna make myself even more confused, which is kind of hard because I'm already really confused And so and this is where we do receive stream here, but we don't Do anything with it as far as I can tell Unless we follow a redirect We do call this thing send request underscore send request And we also create a response object and then we return that So I guess it must be in here where the next like meaningful thing happens But where's that? This is declared at requests 577 That's this is just sending through the socket self dot underscore send Send but nothing in here reads data back, right? Not everything in here is always sending Nothing is reading So I don't know if that's I don't think that's where we need to be I'm very confused by this bit So we because we read one byte we receive one byte we either do it this way or we do it this way But either way we get one byte we check it if it's h then we break out of the while loop We have a socket so we don't do that we create a response I mean is this when we create a response are we reading in here? Received length okay, maybe yeah, I guess so, huh? parse headers raw session status code read into read to read to receive into The response object the response from request contains all the headers and content But I mean we must be reading inside here somewhere, right? We get the sock we get the session Socket equals sock encoding cached headers Receive length is zero receive buffer is 32 bytes Remaining none chunked false backwards compatible Not equals not has adder receive into hdp equals underscore read to and then a single space with a byte string. So that's interesting if not hdp if session Session dot close socket Else if there was no session then socket dot close And raise unable to read response so Yeah, I don't think we're having that status code equals int bytes self read to this thing self that reason equals byte array read to Oh read to like read until read until you find this character the space Okay, read to And then this one is saying read to the new line slash r slash in So reason is that? And then parse headers Uh, but then we just have raw none and session session 577 That was send request. Okay. Yeah, that was right. Oh, yeah, but that I think is maybe I don't think there's anything important actually in here. So we need to basically go back to 676 Is where we called this and we go on from there and the next one is the response This is inside requests 700 and It goes to 178 but 7676 What do we do with it once we get it back response equals response? We just return it It's from request, which means we're just returning it like all the way up the chain basically But where does it get read like nobody has read it yet? Okay. I think I'm gonna pass Not pass anything really I think I'm gonna head out for now So thanks to everybody for hanging out. DJ Devon says I'm heading out Gonna get some more dinner. Yep. Take it easy DJ Devon Thank you everybody for hanging out and watching along. I hope everyone has a good evening For the rest of your night. I hope you have a good weekend and all that stuff as well I'll be back tomorrow morning over on my own channel tomorrow morning at 10 a.m. Central time. I'll be back streaming some more circuit python stuff Yeah, I don't know for sure. Maybe we'll keep going on this maybe we'll work on some other stuff I'm not entirely sure so in the morning. I'll figure that out Yeah, be happy to have anybody who's watching along now if you like this kind of content You can join me on that Saturday morning stream as well and I'm working on similar stuff typically So if that sounds like something you'd be interested in then I will see you tomorrow potentially For everybody else. Thanks again for watching and Yeah, just have a good night and all that stuff and I will catch you all next time