 Well today was another sick day, so she's actually here, but I'll still be first and quick I guess so mostly just working on stuff for the fundraising efforts in terms of some assets and such And then getting ready we have two big same meetings tonight To go over their quotes So that'll be like my entire evening Yeah, that's that's about all I'm up to so yeah, sorry I'm kind of absent from The skill side of things so if you guys need any help on anything big me now we're fine. I'm fine for right now at least okay, I can go next I got the Wiki skill converted to the Common query framework and put up a pull request for that and Waiting for Gezi's review and I started looking at the pull request for adding Video To the common play skill, but I'm confused because that's it just weird There's changes to core there's changes all over the place some of them that looks like of the merge some of them Haven't I'm not sure what to make of that pull request some No to clarify. It's it's adding the capability to play video to core Currently, we don't know not just a common place. Those are the things that he's been touched That's what's confusing to me. I mean if VLC is already supported in common play framework It plays videos. Why did core have to change? We we don't have like If you're if you're a random skill like yeah, but just wants a gooey You should be able to in my mind You should be able to play a video, you know as part of your experience That's not necessarily part of the the common play service Because you know imagine you're not actually playing a video in the sense of you know, you're playing The video, you know a music video clip to go along with music, but you're just playing video you know imagine your Well the the meditation example that I gave the other day like imagine you're doing that and you want to play a video that like Shows people a particular meditation technique So you want to you just want to play video as a part of my point is if I can't but why wouldn't you play video through the common play framework Because why would you want to be able to do it yourself? I thought we're trying to push everybody to kind of be common play or in certain cases common query oriented So giving giving skills the ability to play video outside of the common play play framework Kind of seems counter to what we're trying to do unless I'm missing something. Well, I think there's two I Feel like there's two things there's We need to me the common play framework is around disambiguating Particular utterances that use the word play or similar things and We need a separate Separately we need a way for Microsoft to know what is Being Played audibly and what is being shown on the screen and what is Yeah, what is being played audibly and what's being shown on the screen? I think those two I See those is slightly separate Well, where I'm confused is Common play skills don't Play their own audio they go through the common play framework to play it correct. Yes Why wouldn't they go through the common play framework to play videos? I think they would if they're a common play skill Then I'm confused as to why changes in core need to be hand made To handle playing a video well when you would my understanding was that there was no way to play video previously Like even for the common play skill you couldn't call But I could be wrong if I'm happy to be wrong No, I'm not I'm not saying the common play framework supports video today I'm saying it supports audio and I think it should support video in the same manner This seems like Different No, no, that's what that's what I'm saying is not the case in other words There's no difference between having common play play the audio and having common play play the video But it seems like for some reason changes had to be made to core and I'm not sure why so that that's all I was questioning If it's the case that I mean some of those changes to core are GUI related and I completely get that but You know you would think that if you're a common play skill Then you would basically say play a video the same as you would say play an audio You would say stop a video the same as you would say stop an audio And I don't see any reason for you to be able to go around the common play framework to do so well again, I see I see those two different use cases like You know that there are going to be things outside of common play that play video Just like there are things that will be things outside of common play that play audio. Yeah. Yeah This to me is similar to what we're talking about with project rule over like, you know Hey, we want to just insert this dumb Wavefile into the skill to give them some extra instructions. I mean that could also be a video Right that could be a video instruction. Yeah, I wouldn't wouldn't necessarily need to be Part of common play. I mean it might have some Functionality I suppose in terms of Paul's playing a lot, but might just be a snippet of a video dropped into the skill that's kind of Not even really interactive if that makes sense Yeah, I'm not protesting too much I mean, I guess it kind of makes sense the common play can call into court a play of an audio or video file But it just seemed like common player already had the ability built in to play audio files And I wasn't sure why it wouldn't just have the same Capability for video and then if it was going to do that by simply launching VLC player or Some other type of video player why any changes to core to facilitate that would be necessary at all That's all I was questioning. It just confused me, but again, I'm not sure of the state of the pull requests I don't know what's been merged. What's not what's been, you know, where it's at So, you know, it's it's got some history there. You know what I mean? Well, I mean one of the problems I think do we have is if if things Play Audio like play media without Microsoft knowing about it, then it limits what we can You know our ability to know the state of the system It limits that's how common play audio works today. No Well, but that's a problem. I think I See, yeah, okay Personally, I think that's a problem You're confusing me because I'm looking at an existing architecture and I'm seeing people pulling away from that and saying why and Your response is maybe it's the wrong architecture in the first place But personally, I think we need a system a service in my core that Knows that is in Control of I guess you'd say in control of the media that is playing and that the common play service would use that and Skills would use that rather than them, you know playing audio and video directly Okay, and I believe that's how it is today with audio. I believe it uses the audio service to play audio But things can just things can just play stuff themselves Well, but not if you're trying to give Microsoft control over the environment and know what's going on So then you should go through my crop services So if there's an audio service and common play uses it There should be a video service and common play should use it for all of these same reasons Yes, you want my crop to be aware of what's playing at the time right and be able to handle that a system level Yeah, whether we whether we want an audio service and a video service and then have to interact between them Or do we want a an integrated audio service and also the supports video? Yeah, like a media a media service, but I didn't see that in this poll request was I missing something? No, it's not in this poll request But isn't that the way it should be? Well, this this seems like a step towards that because you know, whatever does it? It's there's gonna need to be a way to you know play video But why not just put it in the audio service as a separate entry point and be done with it I mean that that's valid feedback if that's the you know what you want to say and Okay, okay I don't know you gotta be careful because we're not you start having an audio service that does things besides play audio That's gonna get confusing. Yeah. Yeah So my point is that I should probably rename the audio service audio video service, but then it'll break a bunch of shit So we can just know by convention. It's the audio video service This is where the specs come into play, which is where we're you know, we're trying to get to them But they're also very large bodies of work so I feel like if there are if there are steps that we can take towards that that are not in the wrong direction that provide You know a degree of functionality Then that seems like a good thing in my books I don't disagree with that But the reality is the adding the ability to play a video to the audio slash video service is not a lot of work Well, and if it's the case that that's the direction we want to take Then we should just probably do it and be done with it I thought it was added to a new sort of playback namespace is that but it's been a very long time Yeah, I'm gonna I'm gonna need to comment on the PR so I guess that's where I'm at but um, yeah I mean, I agree I think the audio service should be the audio video service that should have an entry point Commonplace should rely on it and that you work the same way as audio does except that it actually plays Visualized stuff as well as audio But respond to start and stop commands should still be handled by the common play intense and You know my concern is we don't want to you know if we know that's the case We know we want to be there and that's the right design and it's not that big a deal to add that capability Then before people start going off and coding up to a different interface or a different way of doing things Since we don't have a lot of video skills out there right now. Maybe now's the time to bite the bullet Sure sure. Hey, I'm researching that. I don't have I don't know the answer because I'm still Evaluating and this conversation gave me more food for thought So I guess my point is there's a wiki skill has been converted to common play that needs to have a pull request They're reviewed on it, and I'm looking at the pull request for adding video to the system Cool. That's my Next customer Well, and I might go and comment on the on the PR as well to make sure that we We are tying things back to the spec because if we're not if there's not a clear indicator about how things are Tied in with the broad direction that we're going and then that's not very useful either Well, what I can tell you is that the I'm a fan of the common play framework and the common query framework. I understand now the rationale there isn't doctor And it's good stuff and it's it's correct You know basically what you're doing is you're moving intense out of the skills and putting them in common like common play intense handle it and That's wonderful The less intense that are active at any given point in time the better the recognition accuracy is going to be number one And number two is it forces everybody to comply to a similar interface for things that do similar things? Not to mention the fact that I dropped the wiki skill size by 50% Just by porting it to common query because it's just a query return the confidence level and oh by the way Give me the data and I'll figure out which one to play most of that stuff being done up in the common query base class In fact, that was a concern too, although it addressed a concern I had I'll be I don't know if the right way Which is what if skills just return their own high confidence levels that starved everybody off Ain't gonna happen in the common play framework because they're all using and down to the same Calculation value algorithm because it's in the base class. So that's that was interesting But yeah, I mean I have appreciation for them now I understand what they do and I think it was a good direction to go I don't see any problem with it And you know, I think moving forward we should be encouraging people to use those frameworks for query skills and for audio and video skills because You know, it's just it's goodness. So, yeah, I just want to do the right thing with this gooey and the and the video playing and I'm not sure how to button all that up, but Yeah, the spec you speak to is the common play spec And there's an integrated media playback spec Where might that I'll add them is going to pick it up and then chat me a link so I can read it Yeah, yeah, and and then I'll continue my review because I don't want to get Impetuous and make her of the moment decisions But I'm just trying to look at things and say if this is where we're going and it's gonna cost me an extra four hours And let's just do it invite the bullet and be done. I don't know that that's the case I'm just thinking that way. So yeah, yeah, sure Um, well since I'm already talking I Did some more font stuff I'm thinking that we just remove no, no sands display and You know, if anything breaks for people, I don't even it's not even in stable yet. Anyway, right? Oh, that's not true the font exists in stable anyway I think we just remove it now and and if people are relying on no, no sands display Then it will break but the whole point is that they're dev kits. So it's gonna break a little bit But we now have no, no sands as the Derrick's request as the default font and I'll add in no, no serif and No, no sands mono so that we have a complete Range of fonts that are gonna be relatively consistent I updated the pairing skill and I think that it might have been the only one that that was hard-coded And so that should now use the system default So just need to do some testing on that Yeah, I've been getting back in and trying to get some of the Feature branches that we've been using on the mark to dev kits Into the marketplace so we can stop doing this sort of manual Injection of a feature branch during build time Still unfortunately seeing some Failures in Voigt conf that I thought we're gonna be fixed with the 2102 release, which is they seem to be having less often But still happening Which is quite disappointing And What else Yeah, I can't remember the last yesterday's audio didn't work. So need to I Fix that but there won't be any any devs in for the last couple of days. So Sorry about that everyone But you should be listening to this if you're hearing my apology. So there you go Anyway, that's me Chris Yeah, so I'm still knee-deep in intents I talked to Sean yesterday who was the author of adapt And He agreed to Go over a document. I'm putting together that is Basically my understanding of adapt and how it works so that he can you know a sanity check it for me Make sure I'm not making any bad guesses or decisions on What's going on So I'd like to have that document Available for anybody to look at you who has you know what to know about adapter and how it works so That's taken a little time to put the document together it's more detailed than what I put in confluence last time and I actually corrected a few Misconceptions I have the first time I went through adapt So it's a better document So hopefully, you know, I'll send that over to Sean and May take a little while to get to because he's super busy right now, but hopefully don't take too long and we can You know, I'll probably get get okay. I'll ask him to take a peek at it as well So and of course the team you guys want to look at it. I'll share a link to it, too What was your core issue though that you had pedacious High confidence intense stealing your intense Well, that was one issue and actually I just found a different issue so I Was I was trying to change the date timescale to use an adapt intent instead of a penaceous intent for Future time and when I did that I Was getting the current time back a lot Which I thought was strange so I Dug some more into it and I think I found a bug in I don't find a call it a bug in adapt or a bug in core, but it's a bug in how intense can be handled so Adapt when it parses in utterance it parses it into One too many One too many barking dogs So yeah, so One of any permutations of the words in the utterance so for example in one Parsing result it can you know you could be what is and then the other person result it could just be what you know Depending on what's in like these these vocabulary files, right? It's trying to match to that so in the case of a request for a future Time there are several of these permutations. I went through and So what it does is it calculates the confidence of each of these permutations And my my guess is at least in theory the best of all of that is what would you would present back as this is the best intent But what's really happening is We're looping through these permutations and then what core does is it takes the first of the permutations Perhaps assuming that's you know, it was returned in some in some in some a confidence order takes the first one And just uses that even if a future permutation is Has a better confidence than the first permutation does so in this case the very first permutation of The utterance was a current time request Now the future time request that I wrote the intent that I wrote had a better confidence But since it was processed lower, you know further down the list. It wasn't even considered by core No, I'm making sense so far. No Questioning so you're saying that when you got back your list of intent matches That since they're not ordered by confidence level if it takes the very first entry every time It's not getting the highest confidence value out of that list. Is that correct? Kind of because there's there's like a couple of loops through these that goes through one So each permutation goes through a best intent check. So that permutation, you know And each time you go through a permutation it goes through every single intent To find out what the best one for that permutation is Well, I'm going to prove a permutation being what some sort of relation on like words like A conjugated verb so if somebody said run it also looks at ran running Things like this. No, it's taking different chunks of this of the utterance. So maybe He doesn't agree I guess not So maybe one one one query doc vocabulary file could say what and another query doc vocabulary file could say what is Yeah, those are different permutations of the same utterance that need to be considered If somebody said what is it should return a higher confidence level than what Yes, but but the but the parser will return the what in in one set of Of intent checks and the what is in a different set of intent checks because it's a different So each time it combines the words of an utterance in a different way It's a different set of checks But the intent is the summation of all of them. It's the highest confidence of the set of all the vocabulary files First required and then optional. So it should favor Yes, yeah, but Because different Skills can be defined vocabulary files in different ways You can you can have You know one permutation of the of the utterance score differently in different permutation the utterance depending on I have a camera file files are defined sure So in this case You know, I if I say what will the time be in 10 minutes? Yes, right? The what and the time Get pulled in. Oh, that's that's a that's a current time query, right? Are you missing will in your vocabulary in your intent builder? No, no because the first time it goes through it It says oh, well, I have what in time That's a current time and I have a confidence of point three four, right? And it spits that back to core and core takes that well later on another permutation It says oh, I have what and time as when will and and In this duration, that's that's a future time. It's got a point four. Yeah But but cores already consume the point three five as a good one How's that happen? Well, I mean it's difficult to explain because I mean you really have to understand how adapts kind of Goes through this, you know in the way the different ways it loops through these and vocabulary files and intense Often just point it out. That's a bug if you say what will time and that matches what time height with a higher confidence than what will time That's what I'm saying. It's it's a bug Yeah, that's a bug. Yeah, so and I think the bug is that you know when you have these multiple permutations of You know how an utterance gets group now the words and others get grouped together You need to figure out what the best of all those permutations are before you send anything back to core Either that or core needs to figure out what the best of all those are there's a step missing I think is what what it is. Yeah, so the question is like whether it should be returned in it whether whether adapt should be returning at all in a in a Sorted confidence list or whether whether core should be handling that better, but it sounds like that's what you're doing Why we're returning all these things to core when really to me and To me adapt should be saying this is the best Asp and core it says well That's it. This is the best. I'm gonna do this other check and yeah, I hear you there but that would be a significant change but Point being that like yes said If it's the case that you're getting back a list and it's not in sorted order and the code's taking the first one Somebody needs to sort it before the code takes the first one and that would make that problem go away Yeah, the problem is is that it's an adapt. It's a using a yield statement. So it's like a generator. It's not a list To basically and then core takes that the results of that yield statement and does a list comprehension on it Just throws it in whatever order the yield statement is coming in. So So, yeah, well, yeah, I think it's important that we clarify. Yeah, you know, what is the Intent for one of a better word. What's it? What is the you know? What's the contract that we're the that adapt is delivering to Consumers of adapt whether that's whether that's my cross call whether that's someone else like, you know, we Generator you don't have any you don't have any you don't have any choice You're gonna have the receiver is gonna have to gather all the data and sort when it's done Well, unless we change the generator to be something different. You still change it at Anyway, it sounds like you're chatting with Sean about that, which is which is good. Yeah. Yeah That's one of the questions basically this document I'm building. I'm putting a list of questions. That's gonna be one of them Cool. Hopefully I'll work some of this stuff out, but You know, so I'm kind of glad I'm I'm really I'm focusing on this for a little bit because you know, I never would have found this Otherwise, I thought I was I thought I was right of my intent wrong for the future Yeah, yeah, but I wasn't I mean it's actually has a higher intent the current time I'm getting your own time there. Well, and I think that this and like the the weird stuff in PDACIUS probably You know explains why we get funky code in Even core skills because people through that through the you know, the last X number of years have like First done things in the most sane way and then when that didn't work They made these funky workarounds to like just make it work for the time being and then you look at the code And you go, what the hell were they thinking until you hit shit like this Right, so much easier just like different death exactly But yeah, I think it's I think it's useful for for us to be like getting into skills in this way So that we can feel this sort of pain because this is the same pain that that community skill developers are feeling as well So and I think I think intent parsing is such a core part of what we do that It's important that somebody internally really understands it. You know, it's great that okay does and it's great that Sean wrote it but I Think somebody internally really needs to know what's going on in there. So since I'm in this I've kind of taken it upon myself, but We're lucky because PDACIUS doesn't really work. I've had a match on You know a percent value when I would say change the volume and it misinterpreted it to Change the brightness because of the percent like if I say 75 percent or whatever But I've seen it match very rarely at a high confidence level If it started doing that if we started feeding back to it and started doing that I suspect you would see a whole lot of other hidden problems surface Because it's the second most important thing we consult Besides converse Yeah, and Michael made an interesting point. I thought that PDACIUS has a different confidence Calculated mechanism than ADAPT does correct. So are you really comparing apples to apples as far as you know How confident you are that a match is right, you know, that that's a little concerning to me as well Yeah, yeah, I mean one might argue Yeah, if if we ever did use PDACIUS properly what we would need to do is we would need to establish a normalization Criteria for values coming out of both PDACIUS and ADAPT so that they went through a normalization process Or at least one of them did to match the other Yeah Yeah, we're not we're not there yet. So so what I'm so yeah I've been converting a few PDACIUS intents to ADAPT because to me ADAPT is actually is You know while I've just found a bug in it, but it's you know, it's I think it's more proven You know that we know there's actually probably bigger bugs with no more foundational those with PDACIUS that are causing different problems. So I Don't agree with that. I just think it hasn't been trained properly and it's not getting trained It's a feedback loop, but no feedback. Well, I think it's I think it well I think it's a little green while there's there's at least one or two issues the guys and I have found out about But yeah Seems like broadly we Are moving towards use ADAPT for most general things and then use PDACIUS when you have very specific particularly very short Intents that you want to capture and Then that provides the greatest flexibility particularly if we're talking about these sort of more general skills that are always going to sit behind in the background But then it allows You know more specific skills to utilize, you know higher confidence PDACIUS matches to to get in front Which they should be able to like, you know, we want to answer what what the time is, you know, anytime someone asks it, but We don't want to capture everything that has the words what and time in it forever more for only the date time skill Yeah, yeah, it's gonna be it's gonna be a challenge moving forward because our Intent matching is basically very flat right now and until that's corrected then they're gonna clash There's some there's some potential solutions out there that we may adopt at some point But for right now we can make it go with conversion what we got on our hands. We just have to understand the Shortcomings of what we're dealing with And that's what I'm trying to do with ADAPT so So sorry, that was a long-winded status update. It's kind of, you know Yeah I mean it just seems like that's something that should have surfaced a long time ago, right? Maybe but I mean I You know, but in this case I'm replacing a PDACIUS intent with a ADAPT intent And PDACIUS intent might have just, you know, had to take confidence level before and beat it out, right? Sure, yeah All right As Giz said, you know, maybe it had happened before and people have just implemented workarounds on their skill or are structured their intents a certain way to get around it rather than rather than do you spend the three days I spent You know, deep digging into ADAPT and how and how core it is with the results that ADAPT gives it Well, the other thing is when you disable barge and then you become a single threaded application And it's not that difficult to work with that But when you enable barge in and you allow all permutations Now you have all the intent clashes that you didn't anticipate before because you would never have them Because you would never be able to interrupt these things. So yeah It's all tied in. Yeah, so that's me I'm probably will be another day and intent will land and then I'll share you with you guys my results of my work and then I'll move back to the timer scale cool Oh Can you do the if you're if you're switching things? Can you I'm checking the API methods on the weather skill too? Well, can we is the weather skill ready to merge is that? The only thing you might talk about is really only thing I'm missing is some unit tests Oh, yeah, I could add those and then the next record for me doing a pass in these I could add that in the next pass Yeah And we can just merge what I have which is pretty substantial. Yeah I Mean yeah, because we do want to have it as the sort of example skill that we point people to yeah Yeah, but yeah, if we can if we can throw some of those in that it'd be cool Because yeah, it's it's significantly better for the for the mark to you say Cool up up there and not call it the shining example until I get a chance to add in the test Yeah, totally. Um, if you happy for you know Like everyone's been using it. I think it's been battle-tested. So Yeah, I haven't had any problems with it personally Look at what we're talking about. Yeah, my my yeah my rework Make weather great again. I Think it has bugs But I think we should just go ahead and release it and it's current in cantation because Then we need to get it in the Derek sands and you can do more extensive testing You know user usability testing and things like that and give us better direction in other words My point is I don't see us not committing it Progressing any further, but if you commit it I believe we will progress further and I believe it has worked to be done to it But I believe oh, yeah, I believe that will become more obvious Once people are using it and giving a speedback. So yeah, I agree completely the current BK Test for me is in a better shape than it was before the current BK test pass I took a bunch of X fails out that are now passing. So it's it's in better shape But yeah, it definitely needs more on the time zone stuff in there is Definitely better. Well the interaction stuff is where it's not shining You tell it to stop what happens if it gets interrupted what happened? Yeah, what happens? Well, that was never in there in the first place. Yeah, if you tell it to start again, all right Yeah, does it interact with common play framework and my point is You're not gonna see all of this until you get it released and then people can run with it And then they can say oh gee how are we gonna fix this and then it's okay because we have those solutions waiting So it's all right. Yeah, yeah, that's the plan is to take a second pass at these anyway after we do Exactly. Yeah, so I think we're not sir. Let's do the first pass. Yeah, we're not served by not releasing it We were better served by releasing it and iterating through it. Yeah All right, well, I'll merge that then and then you can add the unit test later And then we can also add the API things at any time at that point that means Think I noticed on the wiki skill that I was not drawing from 20.0 80 more I was drawing from 21.0 2 which is good yep And if there is if you do run into a skill that doesn't have a 2102 branch then feel free to add it But yeah We're gonna have to when we cross this bridge of you know the next pass we're not to come up with ways to do vk tests for all these things we've been talking about like interrupting a skill and Skills interactions between multiple skills and stuff, you know vk tests are really kind of written like you're just staying in your lane here And just you know interacting with one skill So there's there's a lot of stuff that's really just not covered in the test Yeah, there look the fall project there's plenty of opportunity for improvement. That's the good. Yes All right, cool, I did I did forget one thing Finished off the the third well the first draft of the the third-party licensing policy, ah if you feel like having a look at that it's a you know If you if you know about any licenses and and have a look at where they exist it started out with just The same categories as as Apache where a being a B and X essentially, but it's kind of branched out to four So I won't bother going through it because licensing is gonna take us down a four-hour conversation, but uh I'll grab a bottle of wine sit in front of the fireplace with my wife tonight, and we'll go through it So romantic. I love it But but at least At the very least have a look at the four like what the four categories are and and see that they make sense because I think I think there's two sides to it You know one is like just the structure of the policy makes sense from a you know from a developer's perspective Is that actually gonna make sense for using it? And and then separately like are all the licenses in the right place and that's the freaking hard part that you know Yeah Cool. All right Are we done? I think so I think so everybody this time tomorrow and more fun and quality. Yeah