 So I've been playing around with this tool. I actually used the GitHub co-pilot for this And as you can imagine, it's generally music out of hand gestures. There's all sorts of issues with it And we are trying to resolve But look at the out screen. It keeps detecting a Hand on my neck to change the sensitivity of it, but essentially There are these musical notes mapped to left and right hand Till we have to modify this mapping because it's pretty annoying as it is in the morning so moving from lower frequency to high frequency and This will adjust the amplitude And the other hand is just mapped to a different range of notes And again volume up and down The other thing is yes, you can play both of them simultaneously essentially the idea was to have two different musical instruments mapped to each hand see Actually tried that didn't quite work. The other thing it does is if you keep your palm close it will do this Burst and if you keep it open it will do longer longer notes same on the You can make it lower this way There's something with the motion as well, but that mapping is a bit elusive at the moment. I have to Keep testing it Probably keep testing it during last weeks from now Let me know what you think and yes, it will be available. It's currently called chest jerk groove And it will be available on the side Later today or tomorrow Yeah, let's keep detecting a hand on my neck all the time. I don't know why it does that Yeah, those stuff here is before they were asking to my chest sensitivity and the lack Yeah, give up copilot a bit It's always have all the context It gave us this line that was obviously problematic to Remove it still need to play with attack decays stain and release or synthesizer for each note But yes, there's a length being Controlled elsewhere both by palm open palm close but also the speed of motion of Of your hand on the screen And this is obviously Related to this obviously a related to what we already have live the eG to music conversion We did using the same protocols Trying to create some Christmas chimes with it Didn't work quite well. It's actually sounds more Christmassy on Some of the channels sounds a bit downed let down less festive on channel seven I think channel 14 from memory Sounded more Christmas like some sort of volume volume control to this and other controls later and Yeah, I have no idea why does that fix as you can ask Get up copilot quickly. Are you able to solve false detections of hands? What the microphone is working differently to the open AI? Website one and there is no really loud Yes, it gives referring to the model complexity and Detection confidence tracking confidence. So I'll have to play If those Show what this magic number is actually mean, right? I was actually doing it for an increase meant mean and detection confidence and the mean track tracking confidence This parameters control the minimum confidence core for the initial hand detection by increasing the values You'll make the model more Conservative so probably probably send those to law spring back 5 It's for their hands. We should have interior less of those false Detections now. Yeah, I'm max number of faces as well. I'm not actually using the face and more Only the hands. Yeah, it seems like that false detection went away. Which is good Still work That other hand is annoying the palm open should the note should be longer. It's calculating the distance between So those should be longer notes They obviously this will need more work, but yes, it's less sensitive now, which is good So we're not getting false hand detections on my face anymore, but yeah to watch that number Maximum number of hands is to And what's the model complexity this part? What is it? If I will this course point to more complex model that can potentially provide more accurate results cost to increase computational resources So you have it's already quite Getting Yeah, we weren't getting before might as well Bringing back to one seem to work anyway Yeah, it was a false detection of the other hand So I don't know what this is Not the error message, but the warnings mean there's something with the browser trying to protect the user from notes being played is straight away and this In GitHub cupboards what it says Yes feature of the web browser or to play policies. It's actually providing audio from playing before You the user interacts with the page Sure, I'm recalling the stones that No, I'm not I need to do it mouse down Yeah eight n means short note it's in one eighth of a note Sure, this will not do much. It's the model model complexity for the hands How do we have it again? Ah, that's for the pose That was a false false detection. Yeah, we're getting errors when we Increase the model complexity to above one keep it at one Yeah, so hopefully we'll be able to try it out Very soon on the side. It's pretty much ready. It's all JavaScript Which and is the annoying one? one of them is producing a very high pitch notes that we could have dropped down the notch which and producing higher A frequency notes a scale left scale right fc four to c five and right C five to c six is higher than c or a higher octave number C five will be higher than c four Which suggests the different scale for both hands One that sounds more pleasant, right? We did the same for left hand I'm afraid that this sounds more pleasant or not. They now overlap to a larger extent Yeah, they overlap supposed to sound more pleasant and harmonious. Anyway, this is something we were playing with over the break I mean it is biomedical engineering related. We use it for exercising as well with dancing who doesn't love dancing especially after eating all the Christmas food Could you suggest how the head position could be used Mapping additional musical instrument or something? I didn't get the Yes, we can use translated pitch What we do if the x coordinates could use more to the left Yeah, we tried this before we have to download more musical fonts and things which we Having trouble with last time. It was I have a couple of Blog posts coming about the e g processing feature the types of feature extractions in e g So stay tuned subscribe like comment. I'll see you next time