 Hello hello if you're watching this video not live make sure and check the description for time codes they tend these deep dives tend to go about two hours so the time codes below will let you find the spots you wanted to watch thank you in advance to the folks that help get the time codes going I'm gonna get going and we'll say some hello's and then we'll get into housekeeping I gotta pause all the videos coming back in at me hey Gary first first chat person oh I have been working my brain is just like pretty toast pretty toast from this week my hands been bothering me a little bit too happy to graph am I saying your name right I hope I am but I wouldn't put it past me to say that I'm saying it wrong sorry for the noise behind me there's a the windows open so I can get some air in here I have long pants on which means it might get a little hot hi Muhammad seems quiet so far hi David I have my first vaccine dose scheduled for next Monday and I'm super excited just got pinged by New York Times saying they're gonna start Johnson Johnson and Johnson again almost spilled my water hi Patrick you're you're two minutes early how awesome is that hey K match so look I've been doing all this merging Gareth says which one I don't know actually either Moderna or Frizer I can let you know next week after I get it I'm very excited been excited to get access I spent this week merging micro Python and that's what we're gonna talk about today the more was like are you gonna do it on stream and I was like I can't I use too much of the screen ah Mark didn't schedule themselves for a late Friday call so he got here on time Minnesota Mentats in the discord as well more like Miata biotograph that closer I'm sorry I appreciate you hanging out every week you could like email me how you pronounce it if you want hi unexpected maker you'll be excited to hear that I'm working on merging micro Python in I think some definitely some folks are thanks Gareth I'm usually I usually don't have too bad of a reaction to vaccine so I'm not too worried I usually get the flu shot so when I get the flu shot it's usually like you know you're just kind of recovering from something but I'll take it take it over COVID any day hi Andrew hi David all right should we do housekeeping and get this show on the road I've got stuff to fix if if we run out of questions and stuff I'm so glad biotograph says yeah I like hanging out here hello tiny LED matrix hi Johnny hi Mahabad Hussain hi generation generation glad everybody could join I'm gonna get started on housekeeping and then if you have questions Bruce S is almost on time I'll do housekeeping and then we can answer any questions folks have and then Patrick says I had zero reaction but my wife had a fever of 101 for an evening after the second shot yeah I think my uncle was feeling kind of unwell and like basically slept it off after the second shot but I'm only getting my first next week okay let's do housekeeping so hello everyone my name is Scott and I go by tanute online I work for Adafruit on circuit Python circuit Python is a very easy to use version of Python designed for really tiny inexpensive computers called microcontrollers they live on tiny boards kind of like this one this is a stem a QT with the samd 21 microcontroller on it and it runs circuit Python so what you do is you plug it in it shows up as a drive with the code.py file on it you edit that file to make it do whatever you'd like to do on the code on a little tiny device like this you can connect it to LEDs you can connect it to sensors like temperature sensors or movement sensors all that sort of stuff so that's what we're here for Adafruit is an open source hardware and software company based out of New York City I work remotely so I'm actually in Seattle which is why my times are usually in Pacific and if you want to support me or in or them please go to Adafruit.com and purchase something from them they pay me to do these streams and all the development that I do throughout the week as well if you want to chat with me and a lot of others throughout the week not just during the stream I'd encourage you to join the Adafruit Discord server by going to the URL adafru.it slash discord that's where I work am during the work week and a lot of folks are there all throughout the week talking about all sort of all sorts of things relating to electronics and things like that we do moderate moderate it to so we make sure it's a welcoming environment for everyone this is a deep dive it happens every week typically this time Fridays at 2 p.m. 2 p.m. Pacific here on the Adafruit Discord streams it typically goes for about two hours or more depending on what we're doing questions are always welcome so if you have any questions please feel free to ask them I'll do my best to answer them and that's including if I'm in the middle of something although if I'm in the middle of answering a question I'm trying to be better about not not interrupting one question for another question so that's most of it next week looks like it will be on Friday as well and just a heads up because it has happened in the past is that this cat here is spook and he is epileptic which means he does have seizures from time to time he's been doing really well so I don't expect it to happen but if I'm like not actually talking to the stream that's probably why I'll just watch him to make sure he's make sure he's okay and then we'll go back to the stream so I'll do some more hello's mark says I had virtually none from the AstraZeneca shop but most people I know had chills and soreness for 12 to 24 hours hi Ham's lab hope I'm doing well I am doing well I've I accomplished a lot I feel this week and I'm kind of spent on it so we'll see how well my brain sticks with me during this dream here and as Patrick says it says it's deep tangents oh no Bruce lost their internet but it has their cell hello Han good afternoon and hello Minnesota mentat as well so um what do I want to talk about um before I get into well I don't know I don't expect to make or I don't think watch this whole thing so maybe I'll start with the micro python-y stuff and then we can work our way outside of my work stuff if we want to so um circuit python is a version of python for micro controllers that is um has been around a number of years now uh I started working on it in 2016 for Adafruit um oh DCD you want a clap I can do a clap that was at 9-11 that is the clock here for me um Rick hello from Redmond uh Gareth would welcome a weekly update yeah so um let's see uh so background is circuit python is I've been working on it since 2016 it is based on micro python so micro python had a kind of a two-year start or so um before we started looking at it Adafruit hired me uh first there's a project just on a project by project basis and then full time to work on what became circuit python um at first the first few times we we managed to update um while you're here unexpected maker so I'll cover the highlights right now um so uh a couple times we've updated the version of micro python that we're based on um but it's been like two years so if you look at let me go to my desktop if you look at the release notes you'll see at the bottom of the release release notes um bottom of our release notes it will tell you what uh what we're based on so that it says this release is based on micro python 1.9.4 and this commit so it's not actually on a proper um not a proper uh like I will say chicken because people are talking about chicken in the in the chat um proper commit so this is actually it's between 1.94 and then 110 was released after that um and that was in like let's see this commit was June of 2018 so we're like um circuit python's version of all of the stuff that we use from micro python is quite old now um and we're in this phase right now with circuit python where we've done 6.2 and we want to do a 7.0 release which is like a major version number that's when um you can change the apis of things um the the highest the leftmost number and a version number is usually the like you must pay attention to go to this version sort of thing um so now is a really good time to uh to take a look at what we're using and um one of the things that we do inherit from micro python is the npy versions um and we want to make sure and we only ever update npy versions on those major version numbers so that if you're downloading like a bundle of some sort like you're going to download a 7x bundle or a 6x bundle or a 5x bundle and those of you who've been doing circuit python for a while know we actually still have per major version bundles even if the npy version doesn't change um so uh because we're in this phase of like we've started 7 but we haven't released any beta or alphas of 7 yet um micro dev had started merging newer micro python and we've inherited a few new things from micro python into circuit python um it just felt like a good time for us to get caught up um and micro dev had started working on merging 1.14 which is about like two years worth of changes from micro python into circuit python um the last two times we had done merges i think dan did them and they took a few weeks like they're really like quite tricky things um merges are quite tricky so uh it's not something that we do that often um but it's kind of enough stuff at this point that uh it's kind of a good time to do it so david has a couple questions first will circup know what npy to take based on the version of the board and both patrick and narodak replied yes um this is an this is does have an asterisk right because we haven't done any formally we haven't done any um formal 7x releases but circup does support 7.x like the this the like the perk of it builds right now will be marked as 7x so um there is a going to be a warning that dan has started telling people of like if you are using those s3 builds that mark mark themselves as 7x the npy version that works with it is going to change um yeah and it is unreleased narodak points out that the 7x support is unreleased so my goal is to get us hopefully to 115 before we do any sort of like pre-release versions of 7 so what we'll do is we'll we'll shift the like there's the the bundle the thing that builds the bundle has a like a version number that it uses to build the py cross that it uses to build the bundle so what we'll do is we'll make sure and set that kind of like past where all of the mergers come in so 7x when it's actually released should be kind of like switched over um as long as things don't get too too nasty um and then david g says how will the learn guide new zip thing behave that's a good question i think what it will do is it will download just the the current stable version so um but i could be wrong i'm not i'm not entirely sure that's something that we may have to change uh as we go forwards um and so microdiv had started this 114 merge and i took a look at it after i was kind of like i think it's time for us to do this i took a look at it i was just like i don't know where to start um it's just like thousands of files that are changing um so it's a very daunting task um and then i tried to do it and i was like oh i'll try to do it i'll just do like it's it's literally like get merge microbyte down 115 and we can go into details about how that works um and then i i spent like two hours on it i was like i can't do this i was like i need to i would need to have a i need to come up with a separate plan and the separate plan i came up with was to do it in stages so the stages is i'm basically going to follow every release um so instead of just doing one big merge of 115 which is the latest there's actually six releases between the 194 that that 6 2 is based on and current micro python they just released 115 so it's 110 111 12 113 114 115 um are all of the releases that are kind of like between where we were and where we'd like to be um and so when i did that it was a bit more tractable one thing that happened is that in 113 it turns out micro python um standardized code formatting which is awesome it's something that i'm very excited about and micro dev did it on circuit python as well so new micro python and new circuit python have the same formatter um but it turns out that 110 111 12 do not so um the first step i've had for merging those is to uh i format at first and then i do a merge um so where am i at well 110 has been merged in this is why my brain's kind of like gone um so 110 is merged in so if you do s3 builds right now it it's based on 110 and you can see here like it truncates the commits um and so it's from like june 2018 through whatever this one was which i think is like january 2019 so that's like how far behind we are um so that's out uh and that's merged in dancin did some basic tests so so basically my process is format the release version of micro python merge it into circuit python and what will happen is you'll get merge conflicts it's typically about i think it was like i was doing it this morning it was like 300 or three or 400 files uh that had merge conflicts um some of those things that end up in that bucket are things that i like i just know we don't want in circuit python so like micro python has a java script port and a power pc port and their own nrf port um so there's like an own stm port so i started doing this file do i have it up um i i have this python file that i'll check in kind of at the end which is just like if you have a standing merge this goes through and like does a bunch of git stuff to basically kind of just set things like broadly set things in particular ways hi paul um so what i do is i have sublime merge yeah unexpected mirror says damien's doing faster release cycles now as well since 113 so cp is really is going to really want to stay updated more often go forwards yeah that's a good point and i think that um i'm actually going to email damien and and chat with him because it's not all that often that we're that once we're at 115 we're going to be actually quite close and so it's worth um chatting with him at that point um but also like one of the reason one of the benefits also of doing this like 110 through 115 merges is that it's kind of like getting over that hump of like it's not just something that we do once in a blue moon it's like something that we do regularly so ideally we'll not only ideally will we be caught up but we'll also have a process for how we do it so like this script for example will be is that second step of like okay like a lot changes let's whittle it down to the stuff that needs our attention and then do that um but also like when we're caught up to 115 it'll be a good chance for us to reevaluate reevaluate the differences of circuit python stuff versus micro python and maybe there's points that we actually want to pull them upstream so that we don't have to worry about some of the stuff because what happens is that like if we have a slightly different version of something and they and micro pythons does it slightly differently then we have to like every time we have to kind of like take the changes that we both made to each side and like kind of overlap them and make them work together um which is what a lot of that that merging step is and so that merging step i'm using i use sublime merge now i don't actually have a merge right now um but if you look so this is the latest commit and you can see it was so i was working on 112 this morning so 110 is checked in 111 is out for review but it's the ci is broken so on the one thing we can do on the stream is fix that and then 112 i started this morning and i've gotten through that phase of getting everything merged in but i haven't actually tried to compile anything so the next step after doing that merge is just like go through the all the files and there's some similarities between the first two as well so like some of the decisions you have to make for the merge are the same um which is why it's like i do have all this context and people have been pinging me for help and i'm just like i'm sorry i'm like totally in these weeds like and and i'm making progress hacking through these weeds but like my brain has has it's weird i i i don't have i i say i don't have that much space for context switching but at the same time i have a hobby project that i've been working on as well um so let me answer some questions hams lab asks do you have plans to support the arduino portenta um i'm not sure that portenta is an h stm 32 h7 i believe which we have basic support for but it's not something that we've like actively been working on um mark says just wanted to pop in and say thank you for all the hard work on this it's amazing how fast you can go from concept to execution with this thank you oh you're welcome that's kind of the whole goal at circuit python is to really get people going really quickly um david asks on the discord are each of the one x going to be tested or or we only care about the last one so that's another good point in there reason that i'm really happy with this approach uh of doing each individual release is that um i'm actually doing like kind of minimal testing um i'm so what i do is i i merge and then i try to make np y cross compile and then once epi y cross compiles i can try to compile the unix version of micro python and then once that compiles i can run all the tests and micro python has really good test coverage which is one of the things i think that makes micro python really good so they have really good test coverage for that core like vm stuff of micro python um so i make sure all the tests work and then once all the tests work then i then it's a question of building all of the um like the actual boards so i've been building like the the raspberry pi pico version because i wasn't using that so i've been using that for testing make sure it comes up the drive works uh the serial connection works and then that's when i start pushing the github uh as just like a way to bulk build um bulk build all the boards uh because some may have like particular settings and i think we'll see that if we get to fixing 111 um i suspect that there's like one like in 110 like there's three boards that have like native emitter on um which is something that we don't typically use but apparently a few folks had turned on and there was some merge conflicts there that i hadn't resolved um so that's kind of the processes that like and that's that's where 111 is 111 is the like i built one of every port sports and then i got all the all the test pass and then um then it was a matter of like just building everything and making sure it all works and like the docks broke so i had to fix the docks that sort of thing um so 111 is close um and then 112 is kind of at the earlier stages where i've done this like i did it's probably like three straight hours so i don't know i don't have a great way to show it on here but basically when you do a merge and maybe i could start 113 but when you get a merge you get a list of files and it says um unlike it's not changes here it's it's unmerged things um and here you can see some diffs but what you it has a resolve workflow that is like three columns it's like the circuit python version the merged version it thinks works and then the micro python version and this is for like 150 files that i go through that i went through it this morning and that's the third time i've done it this week of you get three panes and what you can do is in sublime merge like it tries to automatically do it but there's parts that it can't and for the parts that it can't you can click like little arrows on each side to like select which side you want but you can also edit the middle pane to merge them all together so basically i do that as a first pass and then then i try to like get everything working and then i have to go in and change some stuff because i i certainly don't do it all correctly that the first time in that that merge phase um okay i'm behind on comments marx asked uh is there anything particularly you're looking for in the merge reviews i did look through the 111 pr and built it for the rp 2040 none else not sure what else i can do um yeah so i think kind of the way that i'm taking is like it's a lot of stuff um there will be instability because there's just places i may have merged the code wrong um i think it it's really gonna the the place people can help is use the latest version so use the latest s3 version um right now that's 110 but once we once i fix the ci we can get 111 so use the latest version it's going to be changing a lot soon or like as i work through these um but we will probably find bugs and and the help that folks can help with is really when we find these bugs hunting them down um and what david i never answered david's question so each version of each one of these versions we're making sure all the ci tests pass so all of the the micro python level vm tests will pass all of the boards will will build um but in these cases where we do start to find issues there will be issues there's no there's lots of room for for mistakes um so uh if you find issues or when issues come up that's when it can be really helpful to like we can all try to figure them out um and fix them uh and so that's really where it's going to be helpful because in the past one of the reasons that we've kind of avoided doing these merges is that it does cause a lot of instability and again that's not because circuit python or micro python individually are unstable um it's just because like there's so much room for error and all of this merging stuff um so the nice thing about um the nice thing about doing this in six steps instead of one big step is that we can actually we'll be able to figure out when the errors were introduced potentially so that should allow for us to narrow down where we need to find bugs as as they come up um which will be good um david g says the changelog will be crazy yes it will it's going to be it's we're going to ingest a lot of stuff from micro python and i think this number um if you look at uh meterfruit circuit python main there's this uh 10 000 commits ahead and 2 000 commits behind like this 2 000 commits behind is going to dwindle down to like whatever the commits after 115 are um so bruce says if i build with the current main is 110 merge there yes 110 is merged into in the main 111 is um 111 is in the works so david asks can you briefly explain what the merge does so this is git land and maybe i need the maybe i need like a whiteboard but basically like there's two like git is uh has a tree structure to how things are related and so you know here's a here's a release a version of circuit python and a version of micro python and we've kind of built along separately on those two things now it emerges if i if i move it over what emerges of what we're doing by merging micro python is we're taking a version of micro python and what we're doing is we're merging that together with a version of circuit python so in git terms you'll get a commit and that commit has two parents it means that it has the history the the state of that code at that point has all the history of circuit python but all the history of the micro python stuff that went into that version as well so what that merge commits function is is to um it is to rectify any of the stuff that conflicts between each other so the merge commit itself is just like one big thing that encompasses all the changes but it also brings in that that entire separate branch of history um so in practice maybe i should talk about this was like what this does for circuit python is that it means that like all of this all of the code that we share that we still use with circuit python or with micro python is just being updated so all of the vm code so how the python code actually executes is being updated um the uh some of the internal modules get updated as well although we've kind of split some of those off um and like the mpy and q-string mechanics all get updated as well so the version of mpy's changes the new mpy stuff um will be able to do like the native code things as well um so in practice what it does is it it means that circuit python's kind of python engine will be newer and and better match the current version of like the python engine of of micro python um we don't you as as folks know who know the difference between like we don't use the we don't use the hardware apis of micro python so like we'll we'll be up to date on those things but um we don't use them and in fact i've been deleting them um and so like that that side of circuit python will not change like the the circuit python apis won't change well it will change as all the stuff that it was like that the more like the python the python machine uh virtual machine uh will be updated um as patrick dubby says maybe we can get community to run on automation on different boards um yeah that would be cool that something somersoft has looked at um and i was trying to get this like uh start up to do for us as well yeah the it's a the the portenta is a dual core m7 m4 just kind of neat i think one thing that we're going to get is like in like we could just go over um we could go over like the the release notes of all the different micro python versions um if that would help people to kind of understand how things are changing and i think lemur wanted to look at that too so maybe i'll just do that um david g points out an issue we find in 112 could be fixed in 114 that's totally true um that's totally true yeah which it and i'm trying my darndest to do this as fast as i can like i'm i'm pretty happy with how fast it's going of course i'm taking the weekend to reset but um i would hope i think these first few are going to be harder than the last few so i would hope i'm more than 50 percent of the way through um but we'll see it depends on it depends on what kind of changes were done in micro python and i won't know that for sure until i look at them love the factory asks does the toolchain allow for a code builder front end i'm not sure what you mean by that so if you could elaborate maybe that could help me could help me out david g says so maybe we just make a list of things to verify later yeah i think um yeah i'm that's why i'm just like i'd love people to just like jerry's really good about this like he has like a bunch of projects and he just updates to the latest version of circuit python and then make sure his projects work um so that's that's really what would be helpful is just use it and and if you're in the code then and you find bugs uh help us find where they were introduced um mark says merging code like this is so thankless so thank you and good luck staying sane yeah i'll show you i i have another project i've been doing a lot of coding this week and part of it's because i have this other project uh that's pretty different and uh yeah but the nice thing about doing this merge in six steps is like i i get those hits of accomplishment early on um like like getting uh getting 110 going what getting 110 merged in is really good ionic breezy asks how often is this done well not as often as we probably should have the last time we did it was about 2018 when the the 1.94 was merged in so it's been two years um or three years yeah almost three years i guess because it was june of 2018 because what the last time we did it we updated it to whatever the latest thing was at that point or or close to it so it's been a little while like we had done it this is probably the third time we've done it third or fourth time we've done it since so in 2016 when i started working on micro piped on at the time like that's when we first started for when we kind of forked and then i think i did one merge and then dan did two others and then we had this two-year gap and now here we are um so i'm happy i'm happy with the approach um the merge takes all the updates to micro python and sticks them into circuit python and then someone has to go and make sure that all of those updates work yeah thank you andrew that's a good way to good way to put it um so i think it might be good to just like let we could just go over the release notes of micro python 110 to 115 and i can just talk through it is that interesting to folks um instead of fixing stuff it is really helpful to read that i've been reading them because it's really it's really helpful to know when i'm looking at all the changes all right it got some yeses let's do it okay so let's go to micro python that'll take hours well this is i have one critique of the release notes that damien does for the for micro python because um he puts all the commits in the release notes and that's actually you'll see it makes it harder to look through uh i would like it if he does the summary at the top which is great um but it would be nice if he didn't do the like complete listing because you can get that from githi history um keithie says yes i'm trying to wrap my head around the scale and approach of this and i feel like that'd be insightful to me um okay so this is the latest release i want to do it in reverse order david g says is there a difference in merging the change from micro python into circuit circuit python and redoing the change to circuit python in the last micro python who um it does make some of a difference it i just recently learned that get actually get you can have multiple parents of a commit and it prefers uh the whether you have the the first parent or the non first parent actually changes the way things get rendered we ran into this with web late actually because we set web late up to merge changes in um and it kind of it it kind of like tells get which one is the main parent and which one is like the known as the feature bridge um fundamentally i don't think that the merge changes you would have to rectify would be the same um but the way that it's like compared would be slightly different like get when you're merging has this notion of like hours and theirs right like this is kind of the key part of get being a distributed version control system where like a bunch of people all have different versions and what you're doing is you're like merging someone else's into your copy sort of thing um okay so let's scroll down and you'll see why i have that critique about so this is all the like very detailed change log stuff um that is really what you would get if you looked at the get commits so uh i should give Damia that feedback okay so here's um here's the top of 110 so this is the first one we did um david asked so well we have a lot of new strings to web late uh is it obvious that this is very interesting so there is a change in 110 that damien did where he made some contractions so like the errors had is not and he changed it to isn't or or somebody in micro python changed it isn't just to save a single character and so one of the that's a good example of something like that was a change that happened in a number of places and what i had to do is actually when i merged it i actually chose to not change that so because in circuit python micro python doesn't have this but in circuit python we have translations and so if we change the the error code there as david's talking about we would have to have the translators retranslate it then so um that is an example of a change and maybe we'll see it here in the release notes but that's an example of a change where i am when i'm doing a merge i'm making a decision about which version i want to keep and so there will be new there will definitely be new strings but my my intention when i'm doing merging is to keep keep the existing versions that we have in circuit python if they're close enough okay so in this release there are a wide range of improvements and additions was the main improvement was to the native emitter to have a much more comprehensive support for general python features for generators and exception handling the generated machine code is smaller and is efficient um so i don't know a whole lot about that but basically whereas you could take a python file and convert it into python bytecode but uh micro python has this notion of emitter so it takes in a python file and could actually generate like machine code and i think one thing is here in in 110 he's referring to it as native but i think later and maybe we'll see this the notion of native was switched to calling it machine code which i think is actually in 112 that i'm gonna have to fix um so that's something fuzzy fuzz testing was used to eliminate corner cases um underscores and numeric literals are supported so that's that's a good example of something that like circuit python wouldn't support but will now support uh which is the uh like underscores and numeric literals um new crypto live i i kept that this elwep stuff here i removed because like our policy for network stuff is like we're really going to just do all the networking stuff outside of the core um core code size reduced uh a bit uh which is kind of neat the stm port has something uh that's relevant to us the esp8266 is changed in that again we don't use that we don't use the 32 new nrf port this caused some merge problems because we actually have an nrf port they they named the folder the same so one of the things that this merge micro python script does is for all the changes that happened in ports slash nrf it just automatically takes the circuit python copy um and it just ignores the it ignores all the changes that come in for micro python um they have some dock changes we don't use that as well we have their own way of doing docks um dupe term we don't really use uh and neopixel stuff we don't use um i thought circuit python already supported underscores and numeric literals it's possible so these 110 notes are actually remember circuit python was starting from somewhere between 1.94 and 110 so it's possible that we are like on the on the far side of of that feature so we if we already had it like it's just because we were like between between 1.94 and this 110 these 110 notes um good question though okay so let's go to 111 so that's one of the reasons like because we don't use a lot of the port stuff like sometimes it wasn't really like worth it for us to go through this work but it just means we got really far behind so i i do hope that probably we'll we'll try to stay more up to date it is a it is a huge effort okay so these are 111 so in this release the npy format has moved to version four and has some significant improvements file size reduced is by is reduced on average by 35 percent load time is reduced by 40 percent which is great news right like that's good for circuit python and they have support to save native viper and inline assembler code or machine code generated from any other source size reduction of npy files was achieved by adding a queue string window to reuse past queue strings when encoding them by packing queue strings directly in the bytecode and by defining static queue string sets so i think one place we are going to have to test a lot is all the npy changes um but as we'll see there's more npy changes so maybe like when 111's merged and it's probably not worth it getting too worried about it yet um some vm opcodes were changed to fix a bug um this one here caused a lot of changes and i actually have a micro lab change out for this just most uppercase macros have been converted to lowercase to make a more consistent capi and the default python variables from uh python to python three npy cross compiler supports the new npy version four version and the npy yeah so npy crosses what takes a python file and produces the npy file um and then there's also npy tool which takes an npy file and produces c code structures that you would freeze in freeze into your build um it also allows you to select the architecture of the native emitter and you can print the version hi here you go um a module system for external user contributed c modules has been implemented and documented i'm actually i deleted the docs develop section but the nice thing is is like anything i delete in circuit python is still we can still reference it in micro python uh new javascript port which i just deleted um bare metal ports have this stuff again we don't use that stm we don't use it all this stuff we don't use esp 32 we don't use and then detail changes so i'm not going to go into so that gives you an idea of like usually the stuff at the top is the most relevant to us david g asks so will testing with dot py possibly give different results from testing with npy uh yeah yep ideally no but potentially yes like the way npy works is generally like there's this process to take in a python file and you have in-men or in-memory data structures right like the different sets of byte codes for different functions and then if you want to npy file you take those in-memory data structures and kind of spit those back out into npy files which then can get loaded up and hopefully you end up with similar data structures afterwards or the same um but you skip that like round trip to an npy file if you're just executing it um that web late gang is all on discord thank you web late gang um okay so that's one 11 so what do we test first just i just use it is what i would say test the things that you're working on um you'll find it you'll find the bugs if they're there i know that lady it is gonna start testing it so of course like once we find issues we could get help fixing them okay so here's one 12 this is what i started this morning and we'll probably be pr'd next week early next week ideally um it says this release sees a reduction in overall byte code size due to compression of the byte code prelude saving seven bytes of byte code per function for roughly 80 percent of functions uh so that's a good memory benefit byte code opcode values are also redefined to group them based on the argument size and format and we also support is also added for the matrix multiplication operator at which is interesting and uh as i was doing this merge is the byte code renumbering was actually pretty nice there's some there's some places where uh like some code had a big long list of like conditions and it was just now it's just like oh is it in this range which is good for code size um the npy file format moved to version five to support the new byte coding coding and it added support for relocation of native machine code along with those separate read only data and bss sections this allows a micro python system to import dynamic native modules npy files that are generated from c code examples of this feature are provided in examples which i deleted in documentation in this directory that i also deleted but again it's still in micro python um and so this is why we've been warning people like the npy version is going to change it's like one eleven will get in and then it'll be a few days or a week until one twelve is there um some code saving optimizations were implemented leading to a reduction in firmware size new mechanism to free scripts into firmware is provided which we don't use because we dan redid how we manage freezing anyway um so like that's when i see those changes i wouldn't do those uh when weak module links are enabled an import will now automatically search the built-ins for the the u variant if the non-u variant works this again we wouldn't um we wouldn't do i generally really don't like weak links weak links are basically you can import a module under two different names so like microtime like u time and time um performance benchmarking test suite uh i have but i'm i haven't run it bealy we don't use support little fs file systems i wanted to leave this code in there and it changes some of the vss stuff vfs stuff it's possible we may actually want to use the little fs if we ever have um non-usb bealy capable boards um that's that's a case where uh little fs would work just fine but generally we use fat fs so that it allows a it allows the mass usb mass storage you can't do usb mass storage with little fs which is why there's this weird choice on on rp2040 micropiphone little fs is an alternative file system so a file system's job is to like keep track of what your file names are their permissions the directories and like how that data from the files is laid out in just sort of like raw blocks um little fs is um better by all counts it's a better file system um because it can do more advanced things like ware leveling um and i haven't looked at it too much that much more um by all accounts like little fs is better than fat fs but there's one big reason that fat fs is better and that's simply because it's much more broadly supported by host operating systems and through the way that the mass storage stuff works in on usb is it's it talks over blocks so it's just reading and writing data it's um kind of under the underneath the file system so in order for it to support uh for for you to be able to plug something in and read it on your computer your your host operating system needs to have um file system level support um and it doesn't they don't as far as i know have like little fs support so you wouldn't be able to have a little fs drive it's kind of like um like mac has hfs or apfs like or windows has ntfs right like um fat is kind of like the least common denominator it's it's not well it's it's very old it's like fat 12 but it's it's universally supported um as hugo points out like all os's know how to handle fat because it's just so old um and so it's uh it's where we want to be in terms of um having that really good user experience um and while people are concerned about wear leveling and stuff like the reality is is we've been doing this for five years now almost and we have i've never heard of a case where somebody identifies it like their flash is actually broken um so like the benefits of using fat vastly outweigh the the the problems with it another problem with with uh the way that usb mass storage works in general is that because the host os is manipulating the file system data structures itself it means that you can't have two kind of host os is doing it that at the same time so that's why we're pretty strict in circuit python about like if you can write it over usb you can't write it from your circuit python code yourself because that if you allowed that and there is a flag that you can turn on so that that is allowed but if you allow that there's a chance that like you'll conflict and then mess up the data structures that that define your file system um so yeah um little fs i think it could be interesting in the long run so i kind of like squirreled that away and left those files in there um machine we don't use scm 32 port we don't use esp ports we don't use um new minimal port for samd they needed something different um but we don't use it it's it's very minimal bare metal power pc i just delete um we don't need it uh new git sub modules again like they they just store stuff differently than we do um and the code of conduct which i don't remember seeing i wonder if i accidentally added it um circuit python needs fat but little fs support for randis is awesome in micro python what's the benefit of a randisk eric says is there a safe for where y'all have been thinking about for the future to allow both embedded and host access to files yeah it's called ble um ble the ble stuff won't have this problem because the protocol that i've been working on is done at the files at the file level so like the protocol at that level is like i want to write this file or read this file and that allows like the file system manipulation to be centralized in circuit python um there is a usb protocol that you may have heard of called mtp media transfer protocol that also works at that level but the problem is is that um it's not well supported on mac i think um like it's it's not as like it's just not as universal as fat as and and like that cannot be underestimated as how good that makes usb fat it's like it works like and circuit python's gone really far just because we we use the usb tech that is like the lowest common denominator that works across everything like that's how we can be a keyboard and a mouse and midi and all that stuff is like we do that love the factory says what on chip peripherals are used for a neopixel one channel of a timer uh it depends on the port uh some ports don't use a peripheral at all they'll turn off interrupts and bit bang it like samd is an example uh the rp2040 uses a pio because it's really good for that um we don't tend to use a timer the other trick people can play is um you can use a spy device but the problem is that you have to recode all of the data to transmit into larger data sections which takes a bunch of ram the esp 30 or 32 s2 has a an rmt peripheral which is a timer but you tell it how long the pulses are and so that that is i think what we use for neopixel neopixel there so it depends and that's exactly why it's good to have a an api like we do that's above all that is like it doesn't matter to you how we do it it we know how to do it we do it okay so that's one 12 so we've got three more and this is why i would like to see damien remove all this stuff it's like it's not helpful like you can get that detailed list by looking at the commit messages okay so 113 um this release of micro python includes a new implementation of the micro async i o module which aims to be more compatible with c python's async i o module and we have async i o enabled in circuit python already and it's also been changed to be more compatible so like fully expect to be uh to have some stuff there uh the main changes is use a task object for each core routine and allowing more flexibility to queue tasks in various places example the main root main run loop tasks waiting on events locks or other tasks it no longer requires preallocating a fixed queue size for the main run loop a pairing heap is used to queue tasks in the core is optionally implemented in c uh most code in this repository is now auto-formatted using uncrustify for c and black for python so that's what i've been doing before we do the merge already so once we get 113 merged and i won't have to do that step um different bluetooth stack which we don't use we don't use their bluetooth stuff here pep526 has been implemented which adds the um new syntax for variable annotations such as x colon in equals which is a type annotation thing and then 572 has also been implemented which is adds the colon equals operator also known as known as the walrus operator so i think would be nice it's nice to get those new stuff whole code base now supports compressing error messages this is an idea they got from us um they have a different scheme for doing the compression so i will not inherit their scheme but i think i was looking at this when i was doing the 115 merge like we say translate and i we might i might change that they they say like py error text or something um yeah it is the walrus operator you didn't mishear me there's a there was a lot of controversy the controversy about that was the one of the main drivers of guido stepping down as the the leader of python i'll answer questions after let me finish reading this new features support for non-bullying results for equality and inequality tests the addition of heap lock function binary operations for viper unit operands and time support for little fsv2 file systems important bug fix when importing arm machine code from an mpy file which we don't use but maybe we will going forwards um running synchronized tests are good so their targets yeah their testing game is so good unix port different builds are now organized into build variants analogous to boards on bare metal um sdm32 port we don't care about esp8266 we don't care about esp32 we don't care about also a new m mi r m i mxrt port which we already have a copy of and we won't use as well um so is there a difference between fu equals bar and fu colon equals bar i think the walrus operator is used like within other expressions um so it allows you to say like i don't know i've never actually used it um generally i think one thing that's really good about python is that it allows you to ignore the stuff you don't want to use which is true for any modern c as well c and c plus pluses like when i joined google they're like oh yeah there's all this stuff in in the language that we don't use uh unexpected maker for about rana ram fs is um can use ram like a file system for runtime stuff for loading and saving of volatile data it's just a nice alternative for stuff like streaming text etc or for building data before dumping to the file system um garith asks would you pull all features from each version of micro python in or are there any circumstances and samples where you wouldn't um i mean i think i talked a lot about the stuff that i wouldn't pull in um generally i'm open to stuff garith says you shouldn't use quite a lot of javascript exactly um and that that's the same with like usb stuff um the things that i don't want to pull in are the things that um kind of go against what we're going for right like the there is this large picture of like what's important to us as circuit python which is like very easy access um very beginner friendly um and like the language stuff as well so like if even if something is advanced and this is something that i've been pushing people hard on that i've been pushing back on me about like more advanced features like interrupts and threads and multicore like i'm open to having those things and so i'm willing to get all of this async i o scheduling stuff from micro python um but it's not going to be like i'm going to leave it in there but it's not something that i'm going to be really worried about making work uh same is true for all the like native npy stuff it's like i'm pretty hesitant about it because you know we not only do we have different bundles of npy's for different versions but we would potentially have different architecture bundles as well so like if you're on esp use this one if you're on cortex a m or m0 or m7 you use these ones like i don't really want to get to that world i think there's a benefit to if yeah it's just it's going to be a cross section of stuff um but generally like the micro python people do really good work like they're really smart folks um and the core python stuff of everything is like we would like i think i was on i think it was on the simple electronics podcast or something and they asked like where would circuit python be if you'd started from scratch rather than from micro python and i just said like it wouldn't exist like i'm not about to like implement python on micro controllers like that's not my thing um that's something that damien is really really really good at so um you know i'll pull stuff in as long as it doesn't as long as it matches the model that we have for circuit python which is also why like you're not seeing me pull these ports like the ports are just implemented in a different way they're great is a reference for us if we ever want to take a look at something um but like we have different approaches for some things so um i'm not going to go out of my way to to switch us back to the the micro python approach for for some of that stuff um there was like some weird things like we had marked we had i i had changed like the mp y tool to like spit out a bunch of size information in comments um and like i didn't need to keep that davis says this is great scott however i really have to turn in now good night good night davis thanks for hanging out um this will be up later if you want to pick up where you left off when you're awake tomorrow david says was there a time where people were saying that using foo and bar make it less accessible and is not good for stemming being open is that silly opinion i don't know i think to some degree um but again it like without context variables don't make much sense dylan says with threading and interrupts couldn't we just write them in ccode block to use that inside circuit python um maybe with threading but interrupts the way that interrupts work is you tend to have uh like on arm at least there is an interrupt table so it's like it's just a listing of like where functions are for specific interrupts and i guess from c you like there's no sandbox i i assume there's no sandbox around the ccode so like yeah you could write to that table and override it um but that's well out of the side of what we would support in the sense of like make sure it works like that's one thing like circuit python's not secure at all so like if you want to do nasty stuff like that you can um it's just like your mileage may vary and it's not don't come to don't come to us asking why it's not working we're definitely not going to do that um but yeah i mean some people are like i know thea in particular like she added some of the native stuff and is using some is very excited was hoping for us to update uh because of this newer native stuff so i think i think there will be people it won't be me but it will be other people that may want to push those boundaries and i'm totally open to it like um just like the async stuff is like the async stuff is not that interesting to me but like warrior of wire who turned it on like um obviously has a much better idea of where it fits in and like i love i love having people like that who who have an idea of want to make it happen um like i'm always open to that as long as it doesn't um as long as it doesn't conflict with any of these other principles that we've established um okay two more do do do i don't need the details okay so for 114 this was in February so damien has as unexpected maker pointed out damien who's the lead for micro python and the and the original guy he's really smart um he started doing more regular releases so like i this is why i said i think i'm like kind of in the thick of it right now is because i think in the longer like 14 and 15 i think would be pretty easy because the amount of changes in there will be fewer hopefully um our next item makers is async is super important for newer modern lives like async web servers like micro dot yeah i think it's interesting i think networking really drives home the need for async networking is like so unfun for me to work on like doing all of that esp work for networking stuff it's just like networking is always unreliable and you always have to code against that and that's just like not what i enjoy doing it's not my kind of coding but yeah i think i mean the async stuff is there and i'm not going to be the person to push it but i'm also not the only person working on this i am more than happy to do reviews for people that are uh doing it okay so let's go over 114 unexpected maker says so unfun haha so sorry mate yeah billy has some of that but um networking is is worse because billy is like not very like the tech stack is not very deep whereas the networking tech stack is quite deep um okay 114 the release of micropathons sees the addition of the rp2 port again this is a different port from what we have so we're not going to inherit any of the the rp2 stuff from micropython and we knew it was going to be called rp2 so that's why it's raspberry pi directory instead um the micro bluetooth stuff we don't use um many new sections have been added to the docs about developing and building again that's a reference so like i don't one thing that we've struggled with is like early on in circuit python i kept more of the micropython docs in our copy of the docs and generally that's just confused people um so i think my bias now is my bias generally is like we always have the reference for micropython we could always look back at that um but generally we don't use something i should just delete it as patrick w says could you rename the cp ports folder to to eliminate all of these naming conflicts uh yeah we could change it from ports to something else but you know that that is an example of something i wanted to do from the start of coming into micropython and i got uh i didn't i got to know from some people but i gotta like yeah we should do that but never did it and then two years later they did it in micropython so like we had a ports directory before micropython did um and yeah we could rename it to remove those things but it's really not that bad like the the script like the the pain of merging is not from that um it's the pain of merging is just the the like stuff you have to have so much background for um okay uh the native emitter and in the line assembler now fully support arm for cortex m cpus also support for cpost plus user modules just kind of cool reproducible builds can be achieved by setting source date epic in the shell build environment keystring preprocessing is now done in parallel to speed up building and there's a new ci.sh script um the keystring preprocessing like i've been changing some of it but we also do the um we do translations using the same mechanics as the keystring stuff so we have to that takes some time to to do um here's an here's an example the machine iscritsi and spy classes have been modified so they no longer construct both software and hardware based peripheral interfaces this is something we we did in circuit python from the get-go as well um they have soft iscritsi and soft spy classes but those are those are the equivalent of our bit-buying i o that we've had forever um some other random module stuff um i i don't know this quite as well because i haven't done the merge yet that'll be next week hopefully uh new raw paste mode added to the REPL it includes flow controls that'll be interesting that's like pretty similar to what i've been doing on the vealy side so it'll be interesting for me to take a deeper look at that makes pasting code faster more reliable and uses less memory but one of my critiques with this ideas um it's still all over the cdc stuff the cdc interface so i'd much rather have it on like a dedicated serial port i know damien was thinking about this it's something him and i talked about at one point about having like a standard a standard serial protocol for doing like all sorts of stuff over a serial link um i mean that's one one thing that's really nice about having this relationship between circuit python and micro python is like we can all we can both kind of experiment with things that we think are interesting and like borrow from each other for the stuff that works um that's about it and then if we just look at 115 here so this is released five days ago so this is kind of the motivating factor like hey now's a great time to do it like we just got a new release we're in this mode of like we're between major releases now it's like all right let's just do it um as general support the core using cmake which we don't use um we've redone the make files a lot um and i'll be curious to see what this is but i i've not been impressed with what i've looked at for cmake so i wouldn't expect us to change um multiple precision integer library which would be fine fixed for regular expressions i think that's from jeff actually um that might be something we haven't oh yeah it is from jeff like that's something we may already have in circuit python so like when the diff comes in it'll just show that it was like edited on both sides but it will merge just fine um in the micro async io library a new micro python extension has been added threads a flag which i'll have to take a look at this is something that i've drawn the line between micro python and circuit python is that like damien's okay adding extensions to modules um and i'm not because i want to make sure that when you import a module it is a subset of c python but by extension what he means here is like an api that is not on uh not available in c python so if it's threads a flag if we just pull up c or python async io right if we just look at the docs here if there's this async io module and oh that's a whole it's a whole tasks queues streams locks right so these are the things that are on this module but this um threads a flag is not so like that's one that's one difference between the way that we treat it is like maybe i want to provide threads a flag but i would not do it on a module called async io um but like micro python prefixes all of their modules with the u in order to differentiate that but that's not that's not what i want with circuit python because i also want people to be able to inherit their knowledge as well so like we've had problems with like i can't import time because we had it as u time it's like no let's just put it as time and it'll be a subset of proper time um so yeah that that is something that i would look i would look for in the merge of 115 to like and that's one that was my only constraint when warrior of wire wanted to add async support i said well i want you i want the async support to be upwards compatible to c python um even if it means that it's not compatible with micro pythons version like my preference is to to match c python over micro python and in cases like this where micro python chooses not to match c python like that's where we we decide to not match micro python um it allows preemptive code like i accused to signal async io tasks which are by their nature cooperative but this is good async io current task has been added with the same semantics as c python like that's good that's something we would keep um build system stuff interesting that they network dot land support has been removed to be added back in the future basic support for nonvolatile storage is added um yeah it looks like that's about it the rest of this stuff is port specific zephyr port has been updated to 25 um minnesota mentat says love this input i'm glad you like it um i did kind of glance like i have been glancing over this because it impacts doing all the merge stuff is dcd says i'd like to hear what your outside project is okay we can talk about are there any other micro python circuit by the questions i'll show off my other project and then we can go back one thing i would like to do before i call it the weekend is it would be good to look at what's failing in the 111 pull request because that phase of like making sure the ci works is um it takes a while because the ci is slow so um that's something worth doing before before the weekend so i think i did mention this other project that is totally unrelated to this stuff although it is python some people are typing so sounds like an agenda fixed the ci and while that works and builds talks about the outside project oh yeah give yourself a break i know it's been i told when i made the pull request for 111 last night lady and i was like go rest your eyes like because i i use rescue time to track what i was doing and on tuesday or wednesday i i had like six hours on the computer where 99 of that time was spent in sublime merge sublime text in the terminal like i can't do more than like a solid six hours and today i'm probably at like four already um because i was doing the one one 12 merge earlier all right let's say i like hugo's idea i'll talk about this other project at the end but i do want to just take a look at this so here's um eric says yes so here's the the 111 merge and um we had 44 failing checks so this is something new i'll show i talked about this last week but um if you click this here it said 427 annotations and so it's it's showing 427 errors so this is actually splitting out the errors per board and language combination that's why it's so large uh so so long of a list uh which is great so let's figure out let's take a look at one of these and figure out why it's breaking i'm hoping it's not uh it is size i was worried about that i may not be able to fix this today we're actually yeah yeah we were worried about this i don't have the brain power to fix this right now um this is something we were worried about uh like code size changes and some of our builds are just pretty small um and so this is gonna take i mean it's good it would be a good topic but my brain i don't have enough left in the tank to to handle this oh here we go here's a here's a here's a bug that's not code size so let's fix that we'll at least we can do one run so oh one thing i did want to show off is um let me ban this person inject caffeine i did have chocolate earlier but i definitely don't do caffeine um okay so oh one thing i wanted to give a tip out um there's a command called git work tree which i have been using um this is the start of it so what this allows you to do is it allows you to use a git repository locally in multiple folders on different branches so what i'm actually doing here is this portion of my prompt here is the is the directory so i have a merged micropython directory i have a pending merge directory and i think i have a i have a circuit python one as well so i'm actually able to have like different checkouts on my on my uh drive of like micropython circuit python and the merge that i'm doing and also the merge that um the merge that uh is the previous one i did that's in this like ci state um which actually is where i want to be so this merge micropython is the 112 one but i actually want this one not the 110 one reference that's right i thought that's what i called it i have too many branches oh it's it's not mp this is the one i want um daniel asks is there a specific c python version that you target compatibility with so originally micropython was python 3.4 it's never been python 2 so it's always been python 3 it was originally python 3.4 i think that the ci runs against so the ci does some tests where it runs in c python and prints the output and compares it to the output of micropython um i think it's running on 3.7 right now um i'm actually i have 3.9 on this computer so there's actually two tests that uh fail on my computer but don't fail on the ci because the ci is behind um so i go by what the ci is doing and so the ci is using like three seven or three eight um for its test right now okay so this is it's uh let's just fix this first uh okay sub module update ports what what what is failing here open book m4 so that's going to be Admiral Sandy so i'm what i'm gonna do is i'm gonna fix this error and then we'll get the ci going again currently three six plus random additions damien has brought in yeah i've we've always offered to pay damien to do to work on the core that he likes to do is something so this is probably a case like i deleted something that i shouldn't have uh i square c peripheral is the problem let me just clean and rebuild again so it's this mp define const prop get is new but we are getting it did you mean f strings unexpected maker because we have those in circuit python already my birds those in early okay so i think this is the issue is that i deleted this define um if i remember right define const prop get what does that what is that from so here i am in circuit python so here is where i can just snag it again but i don't know i don't remember the history i don't know if that's a micro python thing or not so let me just find my window and i'm going to do i'm going to check the history of this hi is it h or yeah h which phil points out is not a great name for that okay so it was added by notro oh and it's just this for the property thing right let me just snag that i'll put that back and it goes above to find const map we'll just i have to be very careful in here because i have three versions open i have circuit python merge micro python penning merge and micro python it's going to mess me up they're all get every or i'll get repos though so i can move the changes if i need to so anything with i square c peripheral would have failed okay so it goes here i don't know why i deleted it i didn't like i hadn't looked up the history git status is my friend it's true i i type git status all the time okay so git status dip and we have a spare trailing space now if we make again yeah autocrack f strings yeah circuit python does have s strings the guy who made the micro python um poll request also did it in circuit python there's a couple cases where we merge stuff before upstream does because it's always kind of a crapshoot upstream whether it gets merged in quickly or not um okay so this is a separate issue let's just clean and make sure that that's the case still this one i haven't seen before expected expression before colon token so this might not be quite correct so i'm going to ignore the further changes this is an important skill i think from getting uh yeah i get blame i just use github api for for the blame stuff um when you're doing errors it's not always the best you sometimes you want to look at the the top level thing um start with the first one first especially because these are like pretty close pretty closely related um like do the first one first usually so here prop get and let's just pull up the looks that's all that's he oh no we're missing a curly brace that's what it does that's why that's where that spare spare space came from can you extract what the c preprocessor is actually generating yeah it worked i fixed it it's just a copy and paste there um okay so it it built um let's call that good so what i have been doing and marked it as draft is that i'm being a little pedantic about having just one merge commit so i'm git adding but then if we look here i have git log and i just have this one that is the one that i pushed as well i cherry picked a merge commit today actually as well which is kind of wild but uh you can do git commit amend and what that will do is it just changes it re it overwrites the previous version so that's what i've been doing which is kind of bad but i'm gonna have to cherry pick the next version too it's a little little bad git foo but it's fine it fine it worked out yeah right because i'm i'm cherry picking i i'm working ahead right like in the ideal world i get like 110 merged completely in and then i start working on 111 and that merge commit will be against 110 um but in practice i'm getting ahead of myself and so that 110 merge commit is changing a bit between the time that i started the merge and when i am finishing it so what i did is i cherry pick the merge commit onto the new onto the newer version of circuit pipe on just just yeah right it says i don't like when people cherry pick yeah generally it's a bad idea but in this case i'm cherry picking a commit i'm replacing a commit that's only on a draft pr so nobody should have a copy of it nobody should be basing anything off of it yet um which is why i did this force with lease when i pushed it said if that if that branch is still what i thought it was then it's okay to overwrite it and so now if we look at the pr so this is the 110 one that's in here's the 111 and you can see there's a red x here but if we refresh it should have a new commit here yep new commit and it's got uh an orange dot Hugo says are you getting the diff between micro pythons 1x and x plus 1 and then cherry picking that no i'm cherry picking i'm cherry picking the merge that i already did i was looking at this because i i want the merge to be kind of like against updated circuit python and micro python like that's not changing but what's changing is like the portion of circuit python that i'm up against like there's this one tiny change now in 111 that isn't in my 112 branch um so i'll have to when when 111 is in after i think of it as a rebase but you can't rebase merges or if you do you have to redo all of the conflict resolution stuff which is exactly what i don't want to have to redo um so the cherry pick i'm doing is basically i'm using cherry pick to rebase a single commit that is a merge commit it's very advanced skit for those of you who are not following it's very advanced skit i've been doing this for years and this is the first time i've i've hit it okay so that's going that will definitely not pass because there are the builds that are too too large now and i'm going to deal with that tomorrow or like next week when i'm fresh um so let me just take the last 25 minutes or so and and answer any questions but also talk about the other project that has also been keeping me very um occupied actually here's a question first so how are you bringing in the pre-release changes from micro python per release changes for micro python so what i do is i do i have a somewhere maybe i closed it accidentally a pending merge circuit python if we move to micro python so here's my micro python copy and you can see it it's on a branch format 112 so this is what i'm using to like if i wanted to do if i wanted to start 113 i wouldn't actually do it this way because 113 is formatted but 10 11 12 were not formatted so what i would do is i would check it out um so now i'm on the version of micro python 113 i would run the code format stuff which is not this um and then i do git merge so from from the latest version of circuit python i do git merge but if i'm working on if i'm doing 112 i'm git merging from 111 even though 111 is not checked in yet because i'm working ahead like see i take time i want to i want to kind of like overlap my work a little bit yeah it's a deep git got deep git dive so yeah so because i'm working ahead and that like that 111 commit might change a little bit i have to re i have to cherry pick on top of like the the final version but like there was no merge commits i just cherry picked it it was fine you just have to give like cherry pick like dash and tell it which one of the two is the one that you want to change or something i don't know it worked i did it this morning for the first time yeah with 111 i did it with 111 this morning on on the 110 after it been merged in so as you know i'm a git power user i really like it i think git is really good for version control so i had this idea um to use git to store laws it's not not my idea some other people have thought of it too but i like the the state legislation state legislative session is in its last week here so there's been a lot of news around different bills and stuff in in washington state in particular there's a so there's a website you can go to legislation legislation.wa.gov so this is the state in the u.s that i live in so there's there's laws that like numbers of levels right there's like city laws county laws state laws federal laws and so um this is actually only the laws themselves are only pertinent folks who live in washington um but you can see there's this portal here that the state of washington has and you can do things like bill information so like there's this bill 1336 which is really interesting to all of us hopefully it is created well it only matters to people in washington but it's creating an expanded unrestricted authority for public entities to provide telecommunication communication services to end users um which means uh if i had to rephrase it it's it's related to broadband in washington state so it allows um yes i'm trying to convince patrick to work on this project let's be real patrick also helped me with my election calendar stuff which we did a deep dive on uh like six months ago now um i'm interested in politics particularly i think um everybody should have access to high-speed internet for a reasonable price um which i think i hope everybody watching could agree with and i think allowing public utilities which is what this bill does is one way for that to happen so the people that may provide um like water or sewer or electricity to folks can now sell internet directly to those folks as well although the bill is not actually fully passed so this is kind of the this is the process in washington so like there's two legislatures legislative bodies there's the house in the senate um bills are essentially diffs so this is kind of where it is bills are essentially diffs to the existing law and it can a bill can start in house or senate it's sponsored by a single person with a bunch of co-sponsors yeah david's getting exactly to where i'm going he says put law and get to better track legislation and then new laws are pr so that's exactly what i'm doing um that's like exactly what i'm trying to do because like that's the language that i speak um which is like seeing get changes uh as like how the law will look in the future if a bill is passed and things like that um code is law or was it the opposite so interesting um there's actually a history to this and i was there's a an article i could find again um yeah connectivity is sparse in eastern washington um joe says we have a monopoly putting everyone else out of business and ripping off its users it's called bell canada i feel yeah so it's interesting like local telecom monopolies are really common in the us at least i don't know outside of it but like um there's this always this tension of like incentivizing businesses to lay infrastructure like like wires like literal wires and like station like boxes and things um and monopoly government bodies give like monopolies in order to incentivize businesses to do that um and so there's the legislation kind of like allows for the framework for those monopolies and one thing that send this bill is like defining uh what like there's a notion of whether an area is underserved by a particular person there um oh yeah and that ars technica thing is is a great so washington dc which is not a state but is a region of the us has their laws in get already um but yeah it's not quite not quite what i would expect um ooh i haven't seen this second one i have to read this second one david says i'm an expert in serving elected red representative that vote law and make amendments but not in the us neat yeah so so here's this broadband bill the process is it gets introduced in one of the the legislative bodies in the in washington state the house is like 98 people and the senate is 49 or something and they like they have different term lengths and stuff um so it goes through one there's like committee stage where it has to get out of committee and like a lot of bills never get out of committee um and then um once they get through like once they pass the chamber then they go to the other chamber and once they pass about chambers they go to the governor and the governor has they like is a singularly elected official who has the right to veto uh but the legislature can override the veto with blah blah blah basically it's polar best it's the way that i see it so um what i've been working on is as as the so the bill itself in many cases and that's not exclusively true but a lot of bills are essentially diffs on the existing active law and it turns out there's like two different ways so when the legislature meets and in washington that's only for like the first three months of the year and it varies every other year um but uh if you look back there's actually history to how the laws were codified so the legislature passes a bunch of bills but then like the question was like how how do you know what's still active and so early on in the history of washington state um there was an 1881 code which is like it's a it's an aggregation of existing law that's currently active and then there's like a couple others and then there's like actually like third party publishers that do these aggregations in in what are called codes and what they do is they take the they take the laws as their past um and uh they aggregate them together and give them reference numbers so that like lawyers and law enforcement can reference particular parts of the the laws that are currently active that's the the process of like codification um and in 1950 50 1951 uh the state of washington which is where i live and what i've been looking at they they adopted a new system called the revised code of washington they established an office in olympia which is where our laws are made and they uh there's an office there that is the code reviser so it's a person that is tasked with maintaining a currently list a current uh like legal listing of what is currently active um and there's been some rules set up that basically mean that many many of the bills are essentially diffs so they say like delete this section add this section rewrite this section uh of the the rcw the revised code of washington um so what i was wanting to do is um what i wanted to do was basically put a get and get hub get lab model on top of that um so that uh you could go and like a bill is essentially a pull request where it's like somebody proposes changing the law in this way and then you get feedback from a bunch of people and they change it they might propose other changes to it and then it goes through a series of approvals and then at some point it gets merged into the into the code um so that's yeah and hugo said pull request with a law maintainer edits um but then you have get conflicts with multiple amendment proposals that want to change the same portion there are rules to know what amendment will be voted first yeah yeah just tell me we don't have to hand enter 50 states legal code so i'm not planning on doing all 50 states um yeah i am very i'm interested in politics but i have lots of ideas but this is generally just a like i want to show stuff gary says i worked as a teenager law library and updating law books by removing page 112 and inserting pages 112 a through j hundreds of times per book yeah it's interesting because like all of all of the states do it different ways there was i i was trying to think of a better domain for this project than what i have now um but there is a uh a post that somebody linked to about how like all the different states do it differently um bit of graph says yeah the graph says i i recall one state georgia tried to copyright take down their laws maybe there's some debate in the discord about that as well um luckily in washington we have uh open laws there is actually like a a law that people are advocating for for like uniform electronic access law uniform electronic access law law access legal material act yeah so there is this thing that says um 20 states in the district of columbia have adopted it and washington is one of them i haven't read it exactly so this is one thing that's really interesting is that um one thing that's really interesting is that there's this whole history to laws which i am have kind of nerded out about of like i read the the really origin of the laws of washington and like washington state itself was a territory before it was a state and like when it became a territory it inherited some laws from like Idaho or somewhere else and like part of me is going to nerd out and i want to like backfill history is better as best i can um oh there you go supreme court rules that georgia can't copyright its annotated code yeah so annotated code is kind of the term that's used for for the laws that i'm talking about um it's like it's codified there's a link in in the discord chat again um so what we're dealing with in washington is so here's bill information and you could see like when everything happens so this is really going to be interesting to convert to like pull request stuff um and then there's also amendments so these are all the things these are like pull requests to the pull request basically um and you can see like whether they were adopted or not adopted and then like there's like an engrossing process or a substitution process which is where like they made some revisions um garis says legal codes and computer codes have some interesting commonalities they do um yeah i'm very interested in this so this is one thing that we have and then the other thing that we have in washington and again this is like going to vary state to state um but there's the washington rcw which is the the revised code of washington and there's a law that says you can refer to it as the rcw um so this is the the current state as of december 16th 2019 of um all of the all of the laws of the state of washington um which is there's some pretty it's been fascinating for me to just i've been working with this data and there's some really interesting things that i just ran across like i don't remember where it is um but like is this where there's like state birds that are designated in this there's like state holidays and like sunday is considered a state holiday actually let me show you what i've done because there's a search in here and maybe it'll help me find these things so uh i i got this domain wa-law.org and i'm running a github instance and so if a bunch of people go there it'll be pretty slow all right it's a git lab instance so i'm not doing this on github and the reason i'm not is because i want to be able to create a bunch of fake accounts um for all of the different people that are involved right so uh the person who is doing the poll request is going to be the the primary sponsor of the legislation um and i'll be so i'll be able to make all these fake accounts and that's probably one of the next things i'm going to do um yeah voila i love that it's spelled like people pronounce voila voila yeah alice try says gotta love open data yeah it has been really nice like all of this data i'm using what basically what i've been doing is i've and all the tools are actually here too which i should probably put on proper git lab as well um but it's a it's just a couple python scripts that scrape um the different parts of the website and their apis that they've got um basically i imported all of the code and by code i mean legal code so if we go back and it is definitely slow but that's okay if we pull up the revised code of washington it's like on like one of the smallest versions of the uh digital ocean droplets because i'm cheap and i do a lot of ideas and they all don't always go anywhere so if we look here i want to see like sunday it's probably too vague um so i'm using adoc oh this is what i was talking about it worked so i'm using so i was using markdown because i like markdown a lot um but uh ascii doc which is dot adoc is actually oh municode oh i don't know look at that but see i'm not only interested in like the law itself but the process that made it i mean it's cool that it's open access like that's super cool municode yeah so i would like scrape this stuff not yet codified right so there's like adopted but not like merged um one thing to remember is that most elected representatives are not heavy get users and get is challenging for many people in it 100 right this is mainly for me and this is a debate i've had with my partner becca which is like people don't know how to use get and get hub but one thing that's interesting is that there is there are people that are tasked with this right like there is a code revisors office and they have technology that help manage revising the code um which is where i'm getting a lot of this data and maybe it's that office alone that needs to adopt stuff like this instead um patrick also points out that they don't write the bills they sponsor their staff their staff does it as well right so there's a lot of people involved like it's interesting there's also if you just go to the legislature page page there's um you can actually find like an hour long video on making a like how to make a a bill i forget where it is but it's somewhere here um so like there are people whose job it is to help them make the bills um and make it in a way that matches like there's there's laws about how you make bills um so it's it is there um so yeah so what i'm what i've done so far is i've um converted the it is also interesting that these um the the codes themselves like the section so it's title which is folder in my world title chapter and chapters are an are a single file and then chapters have sections um and so each section actually has a history which is fascinating so so let me pull up so this is title one chapter 16 so let me just find that here so title one chapter 16 and then what you get is you just get the sections but you can hit complete chapter um and so this is what i scraped and you can see i'm missing the note section i haven't imported that yet but it's actually interesting this is your blame this is your equivalent equivalent of git blame if you want to see where like this particular section came from and when it was modified you can actually click explicitly to it and it will take you to pdfs of individual things so here is the session laws from 1953 and it's all you know there are some laws about public access so they've ocr'd all of us so you can actually like copy it out of the pdf and get the text of the pdf as well so yeah this is 1953 chapter 184 um so you here you can see chapter 184 and so it says section 43 86 140 is amended to read as follows um but that's not what we're looking at but here it's section this it as there's as derived from this other thing is amended to read as follows the fiscal biennium of the state shall commence on the first day of july and then if we look here it says the fiscal biennium of the state shall commence on the first day of july so like there is actually like a huge at least in washington there there's a lot of like research that you can do theoretically programmatically um to figure out um the history of all of these things so basically i want to be at some point i want to try to like backfill the history of all the all of the laws so you can do just like i did earlier and get blame but get blame on the laws which is kind of that's exactly what this is doing here um where it's telling you like after a bill is passed it becomes session law and basically in the order that the bills are passed by in a session they get a chapter number um david says there is bill and there's amendment just remember that you're only touching at the text version but there's also the semantic the meaning and the how pieces of the same law interact and how various laws interact 100% and that's why i think this the git view is pretty interesting because it's giving you a i think it's giving you a better view of the context than like if we look at this 1953 thing it's it's going from this particular section of like title 43 and then the next section is about title one um like there's no bigger picture like it talks about what the bill is doing but it doesn't give you the context of like the individual sections right like it's this is your poll request description and act relating to the state budgetary and fiscal affairs right like this is your poll request description this is your poll request title like it makes so much sense to me as a poll request um that gets merged in and the person who merges it is the code model the code um reviser is the person that's actually finally putting it into this final form um so yeah my project is this while law stuff um i've got i've got the form of this i've i've scraped the active bills um i actually i i formatted this better um this formatting here is actually much nicer than what you would get if we look at 50 here like the current system actually doesn't format that well um and i found some other cases where the law actually defines like the physical structure of forms like eviction notices and stuff and they use like html tables to do it and they render terribly in this um and so i think i could probably do a better job with ask you doc doing that as well um so yeah this has been the other thing that has my attention i just thought those funny the following are state legal holidays and the first entry is sunday like it really surprised me that like every sunday is like a state legal holiday um in the same way that like new year's day is a state holiday the text chains are usually attached with the recital that is the explanation when the judge needs to understand the law he can look at the documentation of why that piece was put there and don't forget about dead code you have a piece of law that are not acceptable not used anymore 100% and one thing i found is that like for law for bills themselves like you can have sections that say this this other thing like goes into effect on this date or expires at this date that sort of stuff and i my plan with that stuff is like there will also be stuff other stuff that can end up in the poll poll request description so yeah that's my other that's been my like nights and weekends sort of thing that i've been really curious about like after after following more closely along with that broadband bill about like here's the bill and then there's this amendment and like what amendments are they what do they actually do like i was like i had i know a way that i can understand what this is doing but it's not in that form so um this is my my working towards being able to expose that that process and that evolution of bills uh in what i think is a better way but it's definitely not a universally better way um so yeah that's uh that's that and the reason i'm i'm actually going with gitlab i think i said this but gitlab allows me to mess with all the accounts and stuff too not just if i was on github or gitlab proper then i like i would either have to pay per user or um or i would just have like one account that did everything but this will allow me to like you'll actually be able to go like walla.org slash like you're a representative slash rcw and you could see like all their branches for for stuff well the team that says i've used subversion a long time ago for similar documents version control git is so much better suited and dcd says very cool tainate patrick says this is great thank you yeah so the next steps on this for me is um in my let's see i actually have it open here in tabs although i restarted my computer so this is this is the my local copy of it i mean it's always gonna ask patrick says when are you when you're ready to share more widely let us know um alice tries says it's an interesting one from a visualization slash artist sort of thing this is pretty cool so you know those those animations they make of git evolution where they have this like tree with branches for the different structures and like people people representing blobs that come along and like change them i think that would be pretty cool to like if we're if i'm able to backfill like you know 100 plus years of of evolution of stuff like to the varying degrees um yeah i don't want to be a legislature but legislator not yet at least uh but yeah i think that i i you know there's a huge benefit to having stuff in a machine readable form so like when my partner was like oh you should really do it as like some other form i'm like well once it's in this form that's machine readable like we can evolve it to other forms as well um but yeah so if we look at wallah i think if i just source that i can do the import bills oh one one tip that i'm doing on these is um there's a pipeline package called requests cash which acts like requests except it caches all your fetches which is really nice so i just have like a sequel light database here that allows me to like only hit i am i'm only hitting their servers once uh which is really cool so here you can see that i like for every legislator i'm like printing out their different bills whether they have amendments um and i crashed for some reason so i i would have to fix that but i was like trying to get to the point where i could spit out like here's the modifications that every bill is making and then i can start to actually like load the file up make the modification and make a commit um and then i have to figure out like i've got to make all the accounts in the get lab instance for all the different people and push the different branches for the different things and like laws like bills that don't get passed indirect i see your question i'll get to you before um before we get there if folks have any more final questions now is the time to ask them because i'm already over time um but yeah so i i need to make all all of the the legislator accounts and then i'll be able to have them each have their own fork and then um and then i will have uh different branches for every person for their for their bills that they're working on and bills can actually have a history right so like bills may not have passed in a particular session and they have to start over every two years in this case um so it might be interesting to try to like there's some data in washington state back into the 90s so there might be able to might be able to like tie like a particular get branch to something that crosses multiple um multiple sessions so like multiple prs basically um dcd says it would be nice to track it back to who contributed funding to the sponsors yeah i think there's a lot of work on there as well um on the like transparency is that david says are they already using ai for lawmaking there is an article about using ai to understand what bills were how bills would be codified when scott runs for office all those snake puppet photos are going to need to be explained yeah and you can do votes so so for votes for recorded votes i was thinking i would do each each individual's account would do thumbs up or thumbs down on the poll request or like on part of the part of the thing keithie says i wanted to see something like this for a while this is fantastic bills in terms of service agreements are things i wish i could track history a better hundred percent yeah terms of service is really interesting um that would yeah that would be really interesting i think this is like this is just the beginning i'm i'm not the first person to talk about this and it's just early days okay so indra i had a question let's answer that question then we'll we'll wrap up um unrelated to this project so if you're interested in this project pick me on discord um it is public um so yeah let me let me know um david says you are not the first i know i'm not the first i'm just trying to do it locally um you need to track abstain another nine non-binary states gara says great to see your open government work thanks i mean if you're in a if you're in a place that doesn't have like like i could only get this far because i could scrape the data so like there is a prerequisite there and i'm luck luckily in a state that has really good open openness already um indra says hi scott completely different question from the topic totally okay you're getting us back closer to what it what where we would be um why is adafruit the nrf 52 840 still using soft device 611 stack instead of 72 um because of the different memory layout i haven't looked at that in a while but that's what i would suspect is there's like because the memory late a memory layout would make it hard to update um especially if we're shipping devices that like have a bootloader with the layout of six one um if you haven't made an issue yet about that question i would recommend it um because tac is also the person ask i'm not i'm not the best person to ask about which soft device version we we use i would love to it's been a while it would be good to update especially because when i'm not doing these micro python merges i'm doing um when i'm not doing these micro python virgin burges i'm i i am working on billy stuff uh that i didn't talk about today but yeah all right well with that thank you all for hanging out um definitely that yeah this is my project and once i regain my energy maybe i'll do more more of this this weekend and it's like perfect timing for me to miss this legislative session but um at least in the washington state but that's okay um that is interesting to all leave that open let me pull up my notes for for how to say goodbye um oh you know i don't have that uh thank you everyone it's been a pleasure hanging out with you for these two hours in this deep dive uh if you're new to these deep dives they happen every week usually at 2 p.m pacific on fridays uh occasionally they're moved to thursday next week we'll be on friday as well um and uh yeah if you want to chat with me and a bunch of others uh you can ping me about this uh wallah stuff or uh or uh circuit python you you can find us on the eight different discord server by going to the url adafru.it slash discord uh it's in the description usually and uh thank you as always to david for taking time codes um really really appreciate it so thanks thank give dcd a shout out for for thanking for time codes for that um if you want to support me or and or adafruit that adafruit pays me to work on the stuff um please check out adafruit.com purchase some stuff stuff there lots of boards that run circuit python uh which is what we're working on and um i think that's it uh i just really appreciate everyone hanging out and i'll see you uh next week i guess i'll i i usually pet the cat at the end so so i'll wake him up and do that have a great weekend everyone is your nose cold when the window being open he wants more pets i think all right have a good one everyone