 Okay, welcome to October 5th, Microsoft DevSync. So this is our sprint planning meeting for the next couple of weeks. Chris Baer has pulled up our backlog of stuff. Looks like we've still got a couple of things left to do before we're all done with everything. So let's go through here and before we get started assigning things to the next sprint or filtering out things that have already been assigned to the sprint, let's talk a little bit about the high level goals for the sprint. So the last sprint was about trying to get the mark two enclosure up and running and we had two activities around the wakeward tagger. One was getting the database side of things squared away and then starting work on the implementation for the tagger itself. And we got one of those done but we haven't sort of worked on the tagger yet. So I think we should continue with that work. I think this is a good time period while we're waiting for the next spin of the SJ201s to come in. So I think we should get right on that right away. I've got some ideas about how to do that in a minute. And on the hardware side, we should know that there's anything for us to do right now until the hardware comes back. But I do want to make sure that if we get boards back before the end of the sprint that we know exactly what we're going to do with them. So I want to make sure that we've got the priorities with respect to that setup. So with that being said, does anybody else have any opinions or ideas or remembrances that we've promised that we would do? Well, my question is did we actually get everything done in the last sprint that needed to be done? Do we have SJ201s integrated and operational and running the full stack? That's the question. Yeah, so all the SJ201 specific code, the leds, the volume control and the switches are completed. I have not put the pull request up yet because I'm not sure if we want to include the post stuff, the power on self tests in the enclosure code for that pull request. So the code's done and it's working. I'd like to get the new SJ201s in to verify that because the one I have is the one with the external amp and so I can't test the volume. But other than that, yeah, the code's done. I'm actually moving on to the power on diagnostic stuff. So yes, the answer to that is yes. Okay, and so my next question is for Chris Veier and Derek and maybe Gazz, which is there's a big piece of that hardware integration that involves simply the Pi 4, which is getting the Wi-Fi to come online, getting it to all of the stuff that's related to the operating system level or the compute module, let's call it as opposed to the sound card, which is what the SJ201 effectively is. So is that squared away? Can we do a demo today of us taking the Pi 4 and putting an image in and getting it up on Wi-Fi and so on and so forth? Wi-Fi, no. I think what we've been doing and what the current plan is is you can just use the existing Kivi image and everything should work just fine. Except for maybe the screen orientation. So yeah, I think that's... I haven't really taken any time, I haven't worked on the tagger, so I haven't taken any time to try to boot mine up and see if there is anything that's a problem as far as that image running on my device. That's a good point too. The code I have runs on Kivi. I haven't tested it on QT. So there's two things we could do. We could plug in a QT image and see how well it, and this is maybe something we should do all in Hawaii is compare the two, or maybe do before we go. But what can each do? What are the pluses and minuses of each, that kind of thing? Yeah, I did have a ticket to this sprint for benchmarking resources each resource usage of the two of them, because I know that's been a big question from everyone is like how much resources do they both take, and is that going to be a problem? So that's one of the things that I wanted to do in this sprint, so that we could help move that ball forward, so that we are closer to making a decision on that, and when we do get the ball, as we can move forward more quickly. I had a conversation this morning with Aditya actually, and we spent about half our 45 minutes going over that. So I've got some information to report there, but I think that's actually something that we could ask them for. He relayed some very encouraging information to me regarding the resource utilization of QT versus QB, and also about their commitment to the project as well. So it made me feel a lot better about taking on QT as the official tech GUI that we support going forward, but I would like to get some of that stuff in writing. We got into some of the details, but not necessarily all of the details that we identified in the matrix of things to evaluate. So maybe we could just ask him for some help on that. I know he watches and listens to these. I was actually going to suggest that we get with Aditya and have them do the work. I agree with Michael there. They've made a big investment in it. They're between the plasma image and everything else. There's a lot that's happened over there. So if I'm in the shoes, I would want to make my commitment to be the default. So since we don't really have a lot of personnel resources to chase that, and they've already made a big investment, then let's ask them to go ahead and benchmark the stuff so that we can have a little bit more visibility. My next question is, does this kill the company? So in other words, does the decision we make between Kibbe and Qt, does it kill the company if we go the wrong way? I don't think so. It may jeopardize our community involvement, which is if we kill the community, that kills the company. So that would be a deal killer. But is beyond that, is there a practical reason why, oh my God, if we select, you know, Betamax, we're done? No. Okay, then I would argue that we should make this decision quickly and move on. We just got so much time. We've only got so much time, and this is slowing us down. But we can have a separate meeting about that. We can crank it out here. I mean, I know that these meetings have a tendency to get into this subject like every week. Yeah. Yeah, I do look forward to having this resolved. Let's just take an item to forward all the stuff that we've talked about, all the questions that we've compiled. Let's forward that to a DJ and see if we can get him to help us out on that. Okay, Gez, is that on your plate? Eric and I meet with them every week, so we'll chat to them in about 12 hours or so. And he's in your time zone, yeah? Not at the moment. He's stuck overseas because Australia has a limit on how many people go back in the country every day. Yeah. Okay. So, okay. I've been kind of handing it out, but so a couple things. One, I don't know if Gez may have mentioned this, but we've kind of already planted the seed there. We talked about it last week about setting up some benchmarking stuff. But also, I have ran Qt on the SJ201 with the Pi 4 and did Wi-Fi setup and stuff. And it worked. So, we do know that at least. Michael, is it appropriate for me to check out the Qt image and scroll it and get this stuff working on that and make sure there's no issues or that still down? I think it's, you know, if we need to go that route, I think we can go there later unless you think it's going to affect the power on self-test work. I don't think it's going to affect the power on self-test stuff or hardware. I am planning on putting a module check in to make sure the necessary modules have been installed during power on self-test. And I suspect everything here, we're going over here. I certainly believe we can push it off possibly until the summit. Okay, yeah, let's just not do any work that we don't have to do until we have to do it. Josh, just flagging, I muted you because we were getting a lot of stuff, so you might need to be a computer at some point. I can do it from my headset, I just need to remember to do it. Okay, so just as a reminder, the description for Sprint 16 is going to be get the wake word tiger online. That's the number one new thing. And then the other thing is, what is the other thing? Is there another thing? We need to be ready for the mark two for the next FJ201 revision to come back. Are we settled on the form factor, Derek? Like you're not going to be moving any holes around or anything? We went ahead and did move the holes, I think. Yeah, so for this design we should be good. Hopefully. But are we settled on, can we start working plastics because what I'm seeing, I don't have like a fancy Gantt chart, but what my little internal clock is telling me is that if we don't get a mold maker, we're going to end up having a pile of SJ201s and no plastic enclosures to put them in. Well, right, but we have to do some basic engineering stuff before we can, I mean I can get quotes and stuff, but there could be you know, we might need more ventilation, we might need, we got to pass some engineering hurdles before we pull the trigger. But yeah, we can get quotes moving. Okay. So that means we need to get, I mean we need to get plastics design going. Do we have a, do we have, so Michael this spin of SJ201s is going to have all the holes in it and everything's going to be the way that it's going to be when we go to production? Yes, the physical form factor should not change. So we should, so Derek once we've, once they've frozen that you and I should get going like we should start, I mean we should start printing and cutting and drilling and sanding and get unit one done so that we can do DFM and get the, get the molds cut. Yeah, yeah, yeah, that's interesting. Yeah, we need to start cranking up our printers and making parts, yes. I'm literally cleaning my, my wash station now. And this is your resonating chamber that you sent me. So, yeah, because I don't want that to be I'm concerned that that will be the hold up, right, that we'll have everything else done electronically and software wise and it'll be like, hey, you're, you know, zero tolerance is another eight weeks out. Are we going to use zero tolerance or do you want to use somebody else? Well, I mean, I think there's a lot of things that we've got to decide, you know, we want to decide on how long we want the mold to last. We want it to be a 10,000 or 20,000 shot mold and, you know, aluminum molds. We don't have, we don't have 10,000 dollars to buy steel molds, so there'll be aluminum molds. Right. Yeah, I mean, I've got another possible around the molder in Kentucky that uses China that actually cut the molds, but they do the they do the injection molding in Kentucky. Might be a little cheaper. But there's a couple of different things that I would like to do. For example, right now when we're looking at this could be this could be a whole separate discussion too, by the way, so Michael, I think it is a whole separate discussion. I was actually looking into excuse me. I've been looking into some software along these lines when we get into managing our inventory and manufacturing process and that sort of thing. So if anybody has any favorite software of that sort, but they'd like to have us consider that would be appreciated. Yes, so my ideal situation has always been we find a PCB specialty full product assembler, so like their number one best thing to do is PCBs, but they can handle the full assembly of the product and do all the testing. And it seems like one of the companies that kind of fits in the small production role likes one of the companies that we've been quoting recently. All right, so we'll table that for now. Let's get into the Sprint 16 planning here. I think we don't need to go through this is kind of take forever to go through this line by line and there's a lot of people on the call. So let's maybe everybody can pull up the same backlog and we can skim through it and everybody can individually go through and add in or take out the issues that they think will build along here. So in particular, we want to leave in all of the stuff for the weight weight tagger for sure. And then we should prioritize the things for the Mark II that depend on the ESJ201 bring up. And we probably should add in a few more tasks here now that Kevin's in our system, we can add in some of the dependencies that he's going to be working on in terms of what he's going to do to bring up the ESJ201 and make sure that it works. Test all of the subsystems and all of that good stuff. We don't have to do that all right here in real time though. What I would like to talk about though is how do we approach the weight weight tagger and I think this is maybe an opportunity here or maybe not to try some development stuff that we haven't done recently at least in terms of getting two people working on the same system. So, Ken would you be comfortable working on the web stuff relating to the weight weight tagger? Well I mean I can learn anything but the back end is something called Selene so I'd have to come up with a speed on that but yeah sure. Well I was wondering so my question is would it be useful to consider maybe you and Chris doing some part of the sprint or some part of your days of the sprint doing some pair programming or having one of you doing maybe the design for sorry getting all my acronyms mixed up. So one of you working on the test stuff and the other one working on the actual code you know that sort of thing where it's you know two people working on the same system and you know compliment each other. I wonder if that might be something that we can consider for this particular sprint. I'm flexible. I mean I assume the power on self diagnostics can weight so yeah I can certainly pair up with Chris this sprint on this stuff. Chris any thoughts? I don't have a particular suggestion in terms of like how exactly to do it but it seems like it's work can some done some of this work in the past and so I just thought that maybe he could help make things more efficient for you and maybe you know offload some of the testing code or you know that sort of thing. Yeah we can certainly go to world. I think that makes sense. I'll get with Chris in the morning and I'll start sketching out some of the VK tasks you know under guidance from Chris and we can get the Chris could be working on the development while I'm working on the testing stuff so that certainly can be done in that manner. Okay great. So I was just wondering if I should be pushing some review tasks that can instead of Chris have been defaulting pushing things to Chris he knows the systems the most but this just sort of continues the same problem where we consolidate everything under the same people. Yeah I think it makes sense to push some of those reviews to me specifically since really Chris is the lynchpin this sprint but we want to keep him focused on the bagging stuff. I can certainly do the testing stuff as well as some code reviews and things like that yeah I don't see there being an issue but you know I think we should try to keep Chris free because he's got a lot of work with sprint. Is the tagging stuff going into production this sprint? Well that would be an idea. Wait a second. Well I would be surprised if it would be done before the end of the sprint but that's certainly a noble objective. Okay so Josh you said into production and then Chris you said no. So Josh what is your interpretation of into production first? We have the entire machine learning we've closed so somebody goes to their unit to say my crop it has a data volunteer it just flows into the appropriate folder you know that gets flagged in the Salini system is something that needs to be tagged it flows through first tagging and a validation it gets fed into the data set and we hit a button somewhere and retrain the wakeward tagger and importantly we evaluate the resulting model against the existing model to determine which one's better and whichever one's better we push to the remote devices and doing that for him let me give you an opportunity or alteration to what you just said so nobody puts the button each night possibly at midnight who knows what time a script a cron job kicks off it does a query against the database it reaches a determination on threshold being met and automatically kicks off the machine learning process the only manual intervention that would be in the system would be if the new trained model outperformed the current default model and that's where the process is kind of left at right now because only because there's no send mail running on the training server otherwise I would have emailed out the notification to a distribution so everything you said is done except for the actual and the notification at the end if it's deemed the model needs to be promoted but there's no manual intervention until the determination is made that the model needs to be promoted okay and then the next step of that that same exact process for arbitrary well there's two additional steps so that process for arbitrary so somebody goes and says hey Judy and it automatically sets up that entire flow for a hey Judy wake word so that's item one that is not essential and not mission critical but it would make us it would be a significant differentiator for our company against the other companies that's number one and then number two would be using a transfer learning mechanism on the device as part of the skill so that the end user can contribute 10 or 20 utterances and using transfer learning integrate those utterances into the resulting model so my guess is those two things won't be done immediately but they should be somewhere in the back of yeah I think it's safe to say they won't be done in the next two weeks okay but do you think we can get the first thing done in the next two weeks because that's a I think it's a pretty we spent a lot of time on it and I think I have a great deal of faith and confidence and you guys that you can do it in that timeframe do you think we can push to do it and as if so is there a place that I can be helpful that's a good question I still think there's a lot to do I think we might get this built in this print there are going to be a handful of API calls after the written tests a couple screens that have to be modified I haven't looked at the salinity code in the UI for a while we haven't figured out how we're going to have samples tagged yet for how we're going to select sample to be tagged we haven't figured out how we're going to do the prioritization of how we're going to do the tag depending on if there's still a lot of questions and if they were all answered then maybe that would be a thing of a different tune but I just don't think two weeks is enough time to figure out the stuff and get it coded and get it into production okay well the benefit of using Agile is we can give it a shot if we don't get there then we move it to the next print the first goal would be to get this built in the next two weeks because there's a lot to do so what if the answer to these questions being answered not completely actually it might not be a bad idea for us to enumerate the questions and maybe we just meet first thing in the morning and get them answered that's somewhat significant for you to be able to move forward yeah I mean the two big things I think are how we're going to select samples to be tagged there has to be something written for that I don't know I know there's some ideas I think Michael's probably got some and maybe you have some but how we pop up how do we pick the next one to throw on the screen and the other one the other one's just defining the what tags are there and how do they relate to each other yeah so the answer to the first question I mean if I was going to simplify it would be the oldest first the oldest untagged samples I suspect you have more questions around what exactly the UI is going to require of you regarding what classifications are we actually tagging is that where you were going in other words high low wake word not wake word stuff like that well yeah that's something that I have a pretty decent idea of how the screens look and how they're going to look I've got a pretty decent idea of what the database needs to look like but yeah we need to probably delineate what tags we're looking for because then if we the oldest first doesn't get us the nice spread that we talked about in the past too so if we have 5,000 samples from one person who was using number one we would actually my understanding is that we would prefer to tag a single sample from every user than 5,000 samples from the same user and so there's a few things like that that might help yeah we've outlined a lot of these requirements in the documentation already so I think we need to go through that and design a system just come up with an algorithm that satisfies those requirements yeah I mean I can stub in something that just something from the database and throws it on the screen you know there's a lot of stuff I can do without some of these questions being answered but you need to get them figured out before I can call it done do they have the working UI screens already mocked up regarding the initial process flow or are they TBD Derrick has got me something close enough that I can get what the tags are I can get them pretty well defined basically the idea is the screen stays the same for every different type of tag and just the information changes on them so determine the initial tags or what other than wake word or not wake word that I don't know I don't know what we want to what tag I thought we'd been over this ad nauseam we haven't been yeah I think we have but it depends on how you see it but I actually Chris I don't actually think it's a great idea to stub out the algorithm that we're talking about here just use a dumb algorithm get the whole system working with a you know whatever obvious easy to implement algorithm there is choosing which wake words to tag which get the system working and then we can make the algorithm better you know as the next pass right that's the way to get everything up and running quickly so let's not worry let's not let the the exact algorithm hold us up on making progress on the system while you know we can make progress on getting the whole system up and running while we're you know we have some design meetings around and taking a look at all the requirements that we that we have as Josh points out talked about that now see him I can do that cool I just there's one of my expectations is I am getting this introduction we've talked about it a lot I don't know that we've solidified any answers but certainly getting started with maybe I don't know a maximum of starting and just bagging on wake word not wake words now and then I don't know maybe Derek had it designed as a series of binary is it's beginning to sound like so the first binary would be wake word not wake word maybe the next binary it might be higher low pitch I don't know we got rid of the binary idea actually it's it's going to be you know like one crane will be is this a wake word and you'll actually have like four no it's not there's multiple yes it is or it's something I mean the other thing that I think is is kind of a monkey wrench into it whether we're going to start dealing with larger samples that have many words in them and trying to break them up and read pass them back through I I don't know if I'd be for that first iteration I try to keep it simple and yeah I think we did for now we just tag them as having multiple and reiterate on that later yeah okay I mean we take that as a first pass but I think that one point two is going to be oh hey we have to actually solve this problem because every single sample has multiple that's just my thought a prediction if you look yeah let's let's blow through this quickly we we've gotten kind of jammed up a little bit on the dev side and you know the we've got a limited runway but you know with with Michael's fundraising effort and some of the other things that have happened we have a really clear path towards towards actually shipping this thing and being successful and like being relevant but only if we speed things up we have to move faster from a calendar perspective we just have to and you know it's it's I know everybody's been working hard and there's a limited amount of time that people can kind of work at this start up pace but in my view we need to go back to this kind of insane start up pace for a couple of just a couple of months maybe two three months total through the end of the year until these things start shipping at which point you know we we become we go in a maintenance mode and and start incrementing and improving like we're we're really close to having a product shipping it's kind of like the video game companies with the what do they call it the crunch or whatever where we're getting really close we just need to you know apply some gasoline or some 91% pure alcohol would probably work fine remind me not to strike a spark in here yeah and I don't think we necessarily you know need to start a death march just yet we'll get there but the but I would like to you know take this approach of doing things in passes you know especially with this agile development approach which is not my forte but you know the the goal here should be to get a version up and running as fast as possible right and then we'll add features to it as we go right so one of the features is you know having a good algorithm for picking which things to tag right so by starting off we just have an algorithm whatever Chris can code that you know isn't totally brain dead but something that actually functions right you know so that we're not tagging the same way for over again other than that you know just make it work we'll make it better we'll make it better later and you know ideally we can get through a bunch of iterations like that within a sprint right if we can get the whole system up and running within you know the first you know half of the sprint or so then we can spend the next you know next half of the sprint making it better and sure I don't think we'll necessarily get to the point where that won't necessarily make things faster in terms of getting a you know production ready system online any faster but I think it will definitely help us with the perception of of progress getting us used to you know to getting things done and then continuously improving them the the one difference that I guess one thing I do want to keep in mind though is that I don't want to take a hasty approach to doing this in the sense of we're going to hack this and it's going to be you know much bigger to go at the end of it right I want to do it right and you know especially with respect to just testing and with respect to documentation so those those those are the caveats when it comes to you know doing things quickly so one quick thing that I just thought of is that ideally the samples shouldn't be provided chronologically sequential because if someone is doing you know if their device is triggering over and over again and then you start to be able to like just listen into an entire conversation and I know that's something that's happened in the past so I think you know just some kind of random you know just adding adding a random goal to the to the algorithm would be good okay it's a good thought yeah I mean a lot of ways to do that could be randomization it could just be you know tag more than one thing from the same user you know except out of one every ten samples or something like that yeah yeah yeah but that's a good that's a good warning but unintended consequences of sharing data you know along those lines we could add in fact maybe institute some policies if we find that it's actually becoming a problem institute some policies around just leaving out a percentage of samples you know just say hey we will never tag these like we don't tag odd member samples from each user you know something like that okay so do we have enough information to construct this sprint now I'm just glancing through it real quick everything that's tied with purple relates to the wake word collection so we can keep all that in here anything that's not tagged purple we should look at it to see if it's important for the SJ201 bring up and if it's not then we should punt it into 17 yeah so a lot of those are mine one I wanted to flag is core 331 currently doesn't have anyone associated to it so that came out of the ask yes no roll over stuff something that we should do you know before we ship them our two devices but definitely but it doesn't need to happen right now I don't think we should put a tag on that as you know with respect to what it's what it's associated with right I mean that seems like it could be a user experience issue in terms of making it user friendly but it also might be a functionality issue you know if it's if it's too slow it might actually be unusable in which case it's not just a user experience thing it's a you know it's a real problem so for example like Ken was noting that we can't talk over Microsoft when it's talking to us we can talk over Microsoft when it's playing music right but you can't talk over it to say okay you know stop telling me that so that might yeah that might be well I mean anything you can do yeah I want to get into the detail there but yeah I'll tag it with user experience for now the best tag we have I think but from a higher level it sounds like what we're doing is we're putting a full court press on the wake word tagging process which seems to dovetail nicely with the fact that we won't have new hardware for possibly a week or two and then when that sprint when this sprint ends we can basically put a full court press on everything hardware related right exactly you meet Josh's goal trying to get this thing in a shippable condition yeah yeah so you know we'll get as far as we can with the wake word tiger and it'll you know and then we'll take we'll see where we are and what it takes to get it into production but you know they'll you know if that bleeds into the next sprint then it bleeds into the next sprint but what I really want to do is get it into a maintenance mode where it's just one of the things where we're you know occasionally adding features to or fixing bugs with just like you know precise or mimic or any of the other parts okay so Gez makes a good point a lot of his activity isn't necessarily reflected in the wake word collection so we should expand our eyes to the 20.8.1 probably what are your issues that you can be telling us? Well a lot of it I mean a lot of it at the moment is around improving our CI improving the stability of our CI builds because you know we've just got a bunch of little niggly things that are sometimes failing and so getting those lined out is going to help us move quicker and then the other things are like helping progress the community side of things you know merging the hours just slowly slowly getting through our community contributions I have to keep remembering to meet Josh all the time I don't know how noisy that stuff is I also wonder if it's worth doing a project roll over epic so that we can tag those things because they need to stay on there. Yes yeah good point yeah Josh we keep muting you because I don't know if you realize quite how noisy that stuff sounds to the rest of us so just be aware Michael I'll create a new ticket for the testing stuff my headset was muted so I don't know what's sending you sound so anyway sorry yeah it's great so for tickets on your side we should create some tickets just like you can a couple ideas so create some tickets around managing pull requests for the sprint whatever your target is or if there are specific pull requests that you're looking at I know you've put some of the major ones in there dealing with like lingua franca the modules plug-in system and stuff like that so those kinds of things should be on the board as well but if you want to create a generic ticket for like you know review the incoming pull requests it'd be good to know it just as a reminder that you're working on these things you don't have to like list the specifics of every single model that you're like cool okay so can everyone take a look through this real quick in the backlog and push anything that you don't think belongs in the sprint over to 17 and we'll let's say in 15 minutes or so we can push this we'll start this sprint actually we can go ahead and start this sprint now and you can always push things to the next sprint yeah we don't care about velocity and all that kind of stuff when things are moving but if we get any further into agile it'll make a difference I'm starting it now okay so unless there's any other business people think we need to talk about we can call it here and just spend the next few minutes going through the sprint and making sure that everything in there is is relevant and if not push it out and we will meet again tomorrow or Wednesday depending on who we are Chris B I'll get with you in the morning and go over some of the VK test stuff and what my strategy is and what I'm planning on doing to make sure it's okay with you okay sure hey Derek can we stay on for a minute sure okay thanks everybody