 Welcome to the micro of dev sync. It is still November 30th of November and Mike Hansen has joined us again. Hooray He's back. He I'm still here for a second day We Just started chatting about The mark to build process and I hadn't hit record yet. So that's the context of where we're at Chris were you about to start talking about what you've been doing? Yeah, I might as well just finish that up. So a couple of things I've done today One is I created a there's a feature slash a mark to boot Guess what I called it Yeah, mark to boot branch now That has the changes I've been making That are directly related to the boot sequence document that we will share soon our fine community members But I wanted to give my work in progress to Mike so that he would You would see where I how far I've gotten With with that and not you duplicate any work or anything So if you have a chance to look at the Document I there have been some pretty major updates to it. I Added a section about getting the the devices Activity status versus internet connectivity status Because I don't think I think Regardless of whether or not we can connect to the internet if we don't connect it to a network That's when the Wi-Fi setup should be launched Because that's all Wi-Fi said it does is it connects you to your way on our land network Our land network and so and after that once you connect it to your network, then you can say okay It does that network have an internet connection. So that's probably the biggest change I made in that document today And something I talked with Mike about a little bit on that at most And that is You know, I was basically fleshing out what I spoke we talked about yesterday, which was you know, how are we getting from You know all the skills initialized to the point where we can Launch Wi-Fi or pairing or something. So that's kind of what I put in there Um, please take a look No, I didn't see in the document or I saw the no internet event that was in there But I didn't maybe I missed it. Are there events related to the Wi-Fi being connected or not? Um, yeah, those would be inside the Wi-Fi skill. Okay Um, I should probably at least mention the one that we're looking forward is to know that Network is up And I mean, I don't know if there's any trigger to if it goes down at some point if we I mean it's somebody should probably be on the bus somewhere so we could change the GUI or something right if in a later stage Yeah, well, we're not really concentrating on Um, you know, what happens if it goes down after boot right now. We're just concentrating on the boot. So, yeah, yeah So, yeah, um Please look through that provide, you know, put comments in the in the document or whatever you should all have access to and let me know and if it If we like it then I'll we'll move forward with that I'm trying to get a document into a state where we can share it with the community Do much like we did the activity one so that's kind of what I've been working on And what I will continue to work on and I did Take a look at can I don't look at your script a little bit trying to understand? The network manager and how all that works I I've never touched that so I just been looking at the genome Website trying to figure out all these different states and what they mean and all this other stuff but um, so but what it looks like to me and can correct me if I'm wrong Is that you're going to be able to the the code you wrote will tell me if If we're connected to the network It's not really checking for an internet connection. It's like I'm not going out, you know looking for you Google.com or whatever, right? It's just it's looking to see if it's got a network connection. Is that right? Yeah, I mean it was specifically designed to deal with the aw connect bring up sequence So it's detecting right now and I haven't had a chance to run it on a fresh build yet. I don't know if anybody else has Uh, but once I get a chance to look at the wi-fi status log I'll be able to tell what I think I can detect can detect right now Is that the access point is in a active or configuring state So I believe that's what config means So if you look at it's putting out a state for each of the devices if you look at the wlan zero state There's three things I saw I saw active Disconnected and config So I'm pretty sure it config is the access point Going through its bring up sequence Disconnected is obviously the ssid's out and that network connection is no longer active And active is as you might think we have an active connection to the wi-fi network connected to the wlan zero Now that does not mean we're connected to the internet that is established by the wi-fi skill Who goes through that thing I told you yesterday where he pings the Google dns server and he tries the analytics for microsoft and if he's successful He says I've got an internet connection and if he fails He just basically Keeps trying that now There's also the concept of the clock being set But I suspect But that's a less of an issue and we can ignore it for now and like all good problems will go away So the wi-fi skill Will fail if the clock's not set if we get internet connectivity And he gets kicked off before ntp gets set But that should be okay because if the wi-fi skill fails he'll Time out and retry again and at some point in time the assumption is the clock will be set and the ssl search will be accepted So yeah But that's that's what that script does it looks for those three states basically and Then I wasn't sure what to do So the only thing I actually did is if it's disconnected more than like five or six minutes it reboots I'm assuming that's still in there But that's it Okay, so yeah, I'm off. So it sounds like I can use what you've done so far for No network connectivity status that I would document. I'd put in the document and that's good. Yeah, I mean no Um Well, you can you can use it for network connectivity status. You can't use it for internet connectivity Exactly. That's what I'm saying is I wanted that separate status. Yeah, and I didn't know if you needed that That logic so if you wanted that script To send a message out the message bus or just use the file system To update an indicator. I didn't really care. Yeah, I haven't gotten that far I was just trying to you know figure out what it was doing and if it did what I wanted it it was doing what You know at least from from base camp when I was doing what I needed it to do when it is so Yeah, I mean On the one hand, you can say it doesn't matter. You can fire up the wi-fi skill before you even get a network It'll just keep failing But on the other hand if you look at the boot sequence that we talked about in hawai Probably wouldn't be firing up any skills until you get a network connection I don't understand why the wi-fi skill would ever fail Like based on if the clock's not set Runs and the clock hasn't been set Then it'll fail on fetching the https site because the certificate the ssl cert won't be valid The dates won't won't jive You mean like but by fail you mean it'll throw an error because it's it checks the connection and no, I mean When I say fail, I mean the wi-fi skill will not report that is is successfully connected to the internet Okay, until the network's connected until the time is updated Yeah, one is until the system clock is updated Two is until such time it is it has a network connection, right? Yep connect to the network can't get to the internet So there's several conditions under which it will fail Yes, I think I just wanted to clarify the the meaning of the term fail in that scenario because like By definition the clock won't be set when the like is never set when the wi-fi skill runs and the wi-fi skill currently completes its process of Of off because of order of bring up and that's kind of what we talked about in hawai if we didn't fire up the wi-fi skill Which i'm not saying it's harming anything by doing stuff But the clock the clock will never be set before the wi-fi skill before the wi-fi gets I'm just saying it's not totally changing when it gets For changing no, no, but you need the network connection like didn't get fired up Until you had a network connection Then by the time the wi-fi skill got control The network clock would probably also be set So, you know, that's a possibility, but i'm not And then you bring up sequence the wi-fi skill is not launched until Yeah, i'm saying if you don't launch it until you have a network Then you won't see any errors out of a period, right? And that's the plan Well, but hang on if you don't launch the wi-fi skill until you have a network Then what's the point in launching the wi-fi skill? Well, okay, we don't want the wi-fi skill until we know that we don't have a network Okay No, no, no, no Because I can connect to a network and not be able to reach the internet They are two distinctly different states of being Correct That's why I'm not Iverson clear yet, but that's okay. Let's just move on. Um, Chris, were you Oh, I was let me make you clear So if you have a local subnet in your house and it is not connected to your Internet router I can have the mark to connect to that network and not be able to reach the internet. Correct. Yes That's called edge connectivity, right? Yeah. Yeah, I I fully get all that. Um, Oh Yeah, let's just leave it for now um I was thinking to um, I was going to say it's it would be good to Uh to document um any of these Uh cases that we're not handling right now So I think you were talking about, you know, like um, if Say you were connected to a network say you were plugged into an ethernet cable But you still didn't have an internet connection You know, then what do we do if you're connected to a wi-fi network and we don't have an internet connection Then what do we do and obviously there's you know, the device does something at the moment. Um But they're not cases that we're trying to address with this current piece of work, right? No, I say I don't know if that's true. Uh, Michael said that he wants to Handle the mark two case. So the mark two case Would would go something like this if I am connected to the edge, but I'm not connected to the internet Then the wi-fi skill will fail And I'll be aware of that in the mark two and that is the proper behavior, correct Because at the end of the day the mark two requires internet connectivity, which is a superset of network connectivity Yes, I think but I feel like there's going to be some some Deeper UX work to look at In the future around like, you know How do we guide? You know, imagine your device has already been running for for three weeks and then for some reason your network is alive Like your your your LAN is alive and you're on wi-fi, but it's not connected to the internet, you know What are we doing that circumstance? Do we like just tell people do we like, you know See if they want to connect to another wi-fi network. Do we Yeah, Michael said I've got to worry about that. So I'm not worried. Yeah. Yeah But it'd be good for us to Just just note when we find these sorts of pathways that we're not chasing down right now. I think Well, preferably we would seek out the message contact your internet provider, right Yeah, we'll we'll look at that Yeah, that's multiple things. I've thought about it a little bit, but yeah, we're not talking about it right now That's not too much. That was tongue-in-cheek, but okay Yeah, well, no. Yeah, I mean, well, well, yeah, there's distinction. Is your internet, uh You know, are you saying wanting to like have you moved your micro to another location and the wi-fi is just different In theory in that case your phone is not going to be connected to the wi-fi to the internet either so All right, so or or is the internet just down for a while and and uh, there's yeah, there's multiple different ways We could handle it and but if that's not the scope right now, we yeah Well, the code is in there now. Just so you know if that condition exists for more than five minutes, it'll reboot the system Yeah, I don't know that that's a long-term solution that I gave you that you put in That the build is doing Yes. Yeah okay um, but then yeah, we We also then end up with the Microff screaming in the middle of the night. I've I've created a wi-fi network called my craft Yeah, right. That's what we want to avoid. Yeah. Yeah, so that may not be the desired behavior. I'm just saying it's your behavior right now It is the my two behavior. I know. Yeah, chris. Are you uh Anything else from you? No Cool. Derek Uh, yeah, I did uh do a little work on the screens that uh would go not the Out of scopes that we just talked about but that would go in Um, that would show the progress Um between Well, ideally and I don't know I answer if we can do this Let's say you've uh, you've got this screen that says choose a wi-fi network to connect your my craft device and um At that stage You know, you're doing this on your phone And then you've you've actually fired off your your wi-fi creds to the mark too um At that stage it would be nice to switch from that choose the wi-fi network to connect your my craft device to this um progress page So if we can kick that off all it's real simple It just says it's empty to connect the wi-fi and has a the busy indicator the qt busy indicator in the middle That's what I would suggest for that page and then if success we do the connected uh If not success then um, that's when we prompt the the user That it's going to restart the wi-fi Set up. So I've kind of got the green success check mark that we've currently got and then Kind of a new page for the failure You know red with an x same connection has failed and and the wi-fi setup is going to To restart. Yeah, you'll you'll probably need to get with panic or to affect that change, right? On the gooey, um No, I think we can do that can't we No, we don't we don't handle the um connectivity status monitoring uh We don't handle the connect connection Is it that's done and isn't this what the debuts messages are all about? So the debuts messages are yeah designed to understand what's going on in aw connect Are you meaning trigger trigger it to go back? I mean it'll go back to the Mycroft access point, right if it fails to connect automatically So what I'm what I'm saying is right now in the aw connect containers. There are Sorry, I don't mean by being displayed By panic or correct? Yes Sorry say that again there Are there screens and audio? Dialogs that panic or in the aw connect container are rendering No, no, that's all in the wi-fi connect skill So the wi-fi connect skill is the one that tells you to go to the Access point and enter your credentials. Yes. Yeah. Oh, I thought that was in the panic or aw connect stuff. No So in that case the code that I gave you should be integrated in the wi-fi skill so that If it is indeed the case that we get a config message when the access point is configuring We could trigger that screen Yeah, it might actually make sense for it to live in the wi-fi connect skill, you know and have it very specifically be a pentcore aw connect version of wi-fi connection Well, I could talk a little bit about that cool, uh, so I um When searching with for the deboss messages as well and and figured out eventually That you had to connect to this special, uh deboss socket that it would be connected has Now I found uh, I found a different property. So so ken your script you dig in down into the actual devices um, there's another property That deals with just at a high level check connectivity So you can query it for a numeric state. I don't know if this this is like across all the hardware devices or what? Uh, but anyway, it reports back a value from zero to four And four means it's got full connectivity Um, so I'm not sure how this is different than the the config message you mentioned um, so what I've what I did is I got this into There goes my minecraft Uh, what I got this is uh as into the enclosure or the mark two enclosure skill Um, or just the mark two skill Well, wait a minute. So there's the mark two skill and then there's the hardware or the enclosure Yeah, so I put I put this in the mark two skill, but it's as a test and it should probably go in the enclosure Some somewhere in the howl that's related to mark two specifically since this requires the aw connect system d Or deboss uh, yeah, that's been that's been a bonus contention whether this should live in a mark two skill Or whether it should live in the enclosure Um, I don't care either way Uh, putting it in the enclosure would have the additional benefit that if we did decide to not load skills Until some trigger point Then at least it would have a chance to run whereas if you put it in the skill and you make that decision It will never run. Yeah, but the argument for putting it in the wi-fi skill is that And I wasn't aware of this if it is indeed the wi-fi skill that's rendering all of the Screens and dialogues for the access point like you know go to your phone and do this and Yeah, now and now put up the connected or whatever Screen Then if you're looking to detect failure or if you're looking to detect a reset condition Seems to be best placed in that wi-fi skill Well, so the problem I wouldn't put it in the wi-fi skill at this point since it it's very mark two specific, right? It requires connecting to that special deboss socket that it can't connect to the system deboss Um, so it should said to have a special mark two wi-fi skill Oh, I see. Well in either case Um, so I have I have it set up where there's a thread now That's that's just polling checking that connectivity value and basically reports a state change as a as a bus event So that's that's something I can When I can push somewhere or put that in the enclosure or you put that in the mark two skill I put it in the mark two skill That's the kind of thing that I put in the enclosure, right? And then anything that reacts to it could be in the skill Well, I'm not sure wherever it should go Um, I can can get it in there now. I you use the deboss Python library. I had some difficulty with that so we could Maybe we can talk about that Tomorrow, I just use the qt bus Uh command line to just I suspect you had the same issues gez did with the deboss, uh python module You may want to discuss that with him. There needs pyg objects and all that fun stuff So I I just I just did a sub-process out to qt bus, which is already there and didn't require anything special Um, I think I think that's another benefit of checking it in the enclosure. Um, because you know, I can see skills, you know skills, uh Eventually, you know, shouldn't be able to just run pseudo shell commands on random people's devices Um, but something in the enclosure you would expect to be able to like interact With the hardware. Um, and so that makes a lot of sense And then if it's in emitting the the messages out in the the wi-fi skill response to those that makes a whole lot of sense um So, yeah, that would be my first explanation It goes in, uh, that would probably go into, uh, client Uh, client basis, uh for the enclosure Okay Yeah, so there is a because be careful. There's two enclosures. There's a client enclosure And there's a hardware enclosure Yeah, I think probably be in the client enclosure It could go in the hardware enclosure Since it's panic or specific I mean, I think they they're gonna that's going to change anyway like we we need to Consolidate Yeah, but just to be consistent to take a look at both the client enclosure code and the hardware enclosure code Because the hardware enclosure code actually has a mark 2.pi under it Uh, and figure out where you think it's best place, but I think the uh, the consensus is that in somewhere somewhere in there uh be the guy that reads the uh bus and Sends out notifications on the message bus and then the Responders to those messages could be anywhere and they'd probably be a skill Yeah, so this would be Derek. I guess responding then we can we need to decide on a connected disconnected event For this and then Derek could respond Basically in his gooey wait You know the status gooey wait for the connected event and if you time out then then go to the failure state, right? Yeah, yeah Love it sounds good Um, you know Derek you want to give your update real quick? Yeah, do we finish you but Oh, yeah, yeah other than working on those those gooey things with the wi-fi stuff I just continue to my my cat march, um to the next prototype and We did an update on on where things were at with The pcbea that sj201 and just kind of projected out how long it's going to take to get the First couple of prototypes so Yeah, all good there looks like we're gonna get the the pcbea is by the end of the week They'll be shipped But they're coming from china so they're not going to be in Kevin's hands until probably a week from today At the fastest and then he's got a program and turn them around to send to me so Probably the eighth or ninth. I'll get those in hand and then hopefully I can get them put into a prototype By the end of next week Yeah, so all goes well by mid month we should mid december we should have two prototypes of the the next gen And then we talk about trying to get We're actually getting ten boards on this this spin. So the next eight We might not be able to get done before Christmas just because of Sure, you're gonna get two before christmas for sure, which is really all we kind of need The the follow on eight Don't have a specific purpose right now You know if like an investor or potential Perspective client is interested. That's kind of what they're here marked for So For you guys, you know as much as I'd love to get you all one of the those prototypes They're not really going to be functionally all that much different Then the ones with the ports coming out the sides Eventually Um Like you have devices already, right? I can see a Something flashing in the background Oh, yeah, that's my uh, that's the one I assembled. I'm I was trying to mess around with it and it's it's stuck in a boot Yeah, you've got you got a 3d printer one though, right that one worked okay. Yeah. Yeah, that one they both work fine I'm just messing around with it. We'll get you a third one when we can too. Is that right, Derek? Yeah Yeah, yeah, we should we have another we've got a stock of extra the the laser cut ones Um, so yeah, we can get you a third one of those Okay, cool Um Cool Mike you were already talking say anything else from from your end Um, let's see. So I Yeah, I've done the Attempted this this skill for the wi-fi connectivity Um, I was kind of looking at I think uh Derek had mentioned yesterday uh I know that the there's like there's a failure state where you lose the internet and or you don't have the internet and you You can't really do much more with the mark 2 for setup, right? Uh, at least not via voice um So I I'd wondered uh, I have some stuff that I could do where you could Do speech to text locally with a finite set of commands Um, and I was going to see I make sure it should it should run just fine on the on the pi 4 board here um, so I was I was looking into getting Something small and self-contained for that and seeing if uh if it would run. Okay, and maybe people would be interested in Uh, and having that is like a fallback You know, uh just in case um, and then And then lastly, there's I'm also working on the local text to speech stuff to see if I can get that going too with some decent performance co-co um And ken smith I continue to work on uh unraveling the method as nimic one And uh getting that to work in sane manner I it will interleave which is I guess desirable in this case but um Yeah, anyway, so that's what I'm working on and hope you'll have that buttoned up by tomorrow Just so you know, Michael or what I'm talking about is that right now The mom is taking on speech to text. We do this too if we fail we fall back to local by default I believe on each text we use Uh, mimic one. Oh no for speech to text. We use um pocket spanks, right? is there Just for the wakewood, um, if the if precise fails we fall back to pocket spanks for the wakewood But we don't do it for we don't fall back to local speech to text if we fail, right? No But that would be great. Uh, okay. Yeah I don't know which one I'm using but deep speech, uh, the latest huge score Works really good. It's just slow as hell Yeah, I've been working on getting koki to run on this Yeah, the this you got to do in the streaming mode to have any kind of performance I've got them both running on the mark two or on my machine both the previous one which is 0.9 or whatever That works. Okay. It's relatively fast about three seconds Uh, you know recognition is pretty poor And then the huge score model that they just came out with the recognition is great But on the mark two it takes about 10 or 12 seconds For the stream or not For speech text recognition What I'm working on specifically right now is text speech Where we go to our preferred model which is up in the cloud on our mimic 2 server And that works by if it fails Falls back and renders it using a local version of mimic one, right? That's already built in And I'm trying to get that to work sanely because There's other issues from a session orientation perspective. I thought that are architecturally Foundationally problematic but Just to be able to fall back And play the local and interleave, right? So I said here go render this sentence, right? And it's gonna Or maybe these sentences and it's going to break up into many little files And each one i'm going to go and say hey mimic two give me this You know give me this wave file back when that fails. I'll say locally. Hey mimic one give me that wave file back Now right now what it does does is an a play which it shouldn't It should be going to the tts service and adding it to the queue. Yeah, in worst case, they'll be interleaved You may get a mimic two voice and a couple of mimic ones Until mimic two comes back But that seems like the same way it should work and that's specifically what i'm working on right now So is is the plan then for the tts service to to take wave data you can push at it? In the future or or how are you working this? Sorry this the sound service Oh the stt you're saying I know so for so you you said the problem right now is it runs an a play This does it does the tts service actually run that a play So that's the problem. We have a tts service that the tts plugins like mimic two honor Historically speaking mimic one was the fallback and it didn't honor going through the queue So it just does an a play and so you'll get all kinds of funky behavior So i'm working on fixing that and interspersing them But really the problem is from a session level Um, I don't want to get into that. That's just an architectural issue. Yeah, okay. That makes sense Um Yeah, so maybe when I when I get my text to speech running locally we can talk to and They can make hopefully make that work properly Uh within their the interleaving What I've what I've heard Is larynx uh sounds is that your project larynx? Yeah, larynx is my project that sounds better than mimic one So I'd rather fall back to that Yeah, it should run uh just fine on on the pi four now For large large sentences we can we can break it up and it can actually that's already happening by the tts uh layer Okay, great. So it breaks it up into sentences and then yeah, well Uh It's oh, I had a minimum it breaks it up on sentences. It should be breaking it up on a length as well And if it's not that can be fixed as well. Could I um, could I make a small suggestion? Ken and mike you guys should totally hook up Michael just a button up all I'm doing is getting uh Mimic one and mimic two or any plugin to work properly We can then come back and rally around and revisit improving mimic one. Okay, great Sorry about that guys. Uh, that's all good. Um, and there is a there's an overs larynx plugin already. I believe um, but it sounds like uh, there is potentially the idea of a you know, um I don't know if potentially just any as a config value of like this tts is local Therefore can act as a backup. You know, um, whereas at the moment The only thing that does that is mimic one. I believe Um, but yeah, that would be an interesting feature Yeah, I'm I'm trying to uh train a different model further for this specifically that has, um About a hundred voices available in it And should should run faster than larynx does right now Uh, so It hopefully will be a good fallback Awesome Um, oh that leaves me. Uh, I Um, it's been a lot of yesterday trying to Debug what was going on with the latest mark 2 build So as I mentioned, I think before the recording started Um, I found some interesting things with the way that the New xdg paths were getting Um, defined and created and that there was some Um, some race conditions there Um, so it looks like, uh, okay, and javas have have done some nice work and solve that for me overnight Um But I don't think that that will be the end of it So I'm hoping it is and maybe it was just, you know, that thing that was causing You know and uh Future, you know, all sorts of other weird things based on it, you know, not having access to the config, but Um, there might be some other stuff there. So Uh, I'll keep looking at that um Also Getting back to some of the language stuff. So we've um Found some issues in the azabai jhani um lingua franca pr um So just some small stuff. Uh, and there's still the The the never-ending time zone dramas over there, uh That Yeah, need need some attention Um But yeah, probably I've been sticking to Um community prs. I will jump back and um and just double check that the wolfram skill is is working properly on the on the Production server um against the production server Uh, and then if someone can can review that and we can get that merged in that will be awesome I think that's it. All right We'll another slightly longer one, but we'll keep cutting them down And uh, everyone knows what they're doing for the next 24 hours Yes, beautiful. Yep. All right. Go forth and do great things We'll do so, yeah. All right. Bye guys