 My episode, yeah, so it'll be a boring one, so. It's super interesting. We'll get it done in five minutes. I'm going to be here looking at my watch the whole time. Yeah, can we cut away when she yawns? Is that fine? So, you did your episode with the VR stuff and the AR stuff from the future. So I thought, right, I'm going to have to up my game here. I'm going to have to do some device stuff. So I am going to talk about this device. Did you just find the first device you had lying around in front of you? Yes, that is exactly what I did. I brought this from my desk as well, because I want to talk about handling user input. Now, here's a thing, right? The title of this video below will not be that. It'll be different to that. I'm going to put a link to that in the description for you and on some of the scene stuff, because at some point between now and the video going out, Aaron our producer will point out that this is the most boring title that has ever been written on the screen. He'll suggest something like the 20 most ways to optimize your input, the main your experience. Or do that in my video too. So that's what we'll be writing below now. But yeah, basically, I want to talk about keyboards. Cool. I mean, we've all got them. Well, we'll get on to that. So this all started with Guillermo posting about a Wordle clone written in Versaille in Next.js and all that sort of stuff. Great. And I replied, doesn't work, mate. And that was as helpful as I could be at the time. Yeah. That's the best feedback a developer loves to hear. Yeah. Yeah. Because he asked, whatever are you getting? He's like, I don't know. I'm on my phone. Bye. I'm out for the day. Saws. But we found out that the error was only happening on Chrome on Android or on Android in general. So let's look at what's actually going on here. This is Wordle. You play Wordle, right? Yeah. Everyone plays Wordle. Everyone plays Wordle. It looks a bit like this. So this is what the Versaille one looks like. It's a little bit different to the actual Wordle because it doesn't have a keyboard on the screen. Yeah. They were using the real keyboard for all of the inputs like on desktop and on phone. And that's the bit that wasn't working for me. So I thought, like, let's take a look at an implementation of how you might code something like that up. Yeah. First up, we're going to use a div. All right. Yeah. This is where we're going to put the letter. Then we're going to get the JavaScript going. Get that element. And then on key down, we're going to have a look at the key and see if it's like a lowercase a to z because we don't care about the other keys. Mm-hmm. And if it was one of those keys, we'll pop it in that div. Yeah. Click. Done. Surely you have to make that element focusable. Why? So that it has... Oh, because you'll get the... So you're at a vent listener. Is that... I'm guessing it's on the document, not on... On the window, but yeah, equivalent. So you're not doing it on a per element. But you're right. If someone could click on a letter and... The main problem here is on mobile, you're not going to get the keyboard. Yeah. So to fix that, we'll just throw an input on there and visually hide it but make it cover the real... the visual thing, like the squares. So when the user taps that on their phone, it'll open the keyboard. Would an alternative here be putting content editable on the div itself? Yes. So if you use content editable then you have to deal with all sorts of things like bold and of a styling that comes with the... Yeah. There is actually a content editable thing to say, I want this to be content editable but just text. But that is, I believe, Chrome only. So we can't use it here. But yeah, this implementation works fine on desktop. It does not work on Android, on Chrome. So if I play a game, which key was pressed? All right. It's the hot new game show everyone's talking about. I love you even had an animated introduction for this. Thank you for noticing. I will play it again. Oh, that's beautiful. With the little pause between each word. Spectacular. I spent a lot of time on these slides. I am glad you're appreciating it. But yeah, so you're going to play this game. I want to know what key was pressed. All right. So, here we go, round one. Duh, duh, duh, duh, duh, duh, duh, duh. W, assuming a quite a keyboard. Very good. I mean, yeah. The answer is, it is complicated. Yeah. But you have hit upon one of the complicated bits. So you get these two pieces of data on your, on your event, event.key. event.code. And it is split like this, which is hitting upon what you said there about like part of it is to do with the key and part of it is to do with the thing that's written on the key. So, okay, it's easier to explain like this. Here is a keyboard. This is the key we're talking about. And like you say, on a Quota keyboard, that is the one with the W printed on it. So key, event.key is W, the string W and event.code is the string key W. Yep. Great. But as you pointed out, if we are in an Azzerty keyboard, like a French Belgian keyboard. Yeah. The key is Z and the code is still key W. Ah, now that's interesting. Hmm. And again, if we're in this, do you know this layout? No. This is, I'll show the whole thing. This is the Vorac. Wow. It is, I have met people who use this keyboard layout. They will swear by it, say it's much easier to use. Yep. So this is the Vorac. So yeah, in this case, your key will be comma, but your code is still going to be key W. And that's the split one. Because these mappings are software. Yeah. Usually. You can buy super nerdy keyboards where you can override the firmware and change that, but most people don't do that. It's software. So the split there is that the key is taking that into account, that mapping into account, whereas the code does not. So I can see why this would be really useful for if you wanted to have a video game that used W, A, S and D buttons in order to move. But in this case, you don't want to be using W, A, S and D on the Vorac keyboard. If you were going to do a game and you were doing the first person shooter stuff, yeah, the thing you care about is the code. Yeah. Because you don't care what's printed on that key, you care about it being there on the keyboard. Whereas if you were saying in your app, like, oh, control in W toggles widescreen mode or whatever, now it's the key. Because you care. You want to follow that around the keyboard. You want it to be the one with the W written on it, no matter where that is on the keyboard. Cool. So next round. All right. Which key was pressed? Cool. Gotta base myself now because I think you're gonna have more tricks up your sleeve. Ace the first one. What about this one? Ooh. So I definitely say like the key is E and then you'd think the key code or if I were to take a naive guess because I've never written code for this before, I would either say it would be like if you short press it key code for like key E and if you long press it key three. So yeah, I did try and catch you out there. Here's the full video. Oh. So what do you think? Oh, I don't speak a language which has an accent to D. So. In England we don't use accents because we're worried they might fall off and cause unnecessary punctuation. So we skip that whole thing. So I'm not sure what you would normally do for the keyboard shortcut because normally if I need to do that, I go to Wikipedia, type in a word which I know has an accent and then copy and paste the character. Yeah. Cause I am very inefficient in all things. So I would say the answer here is, again, it's complicated. Yeah. As you hinted at. Unidentified. Yeah, this is on Android. It says the key is the string unidentified. And the code is blank. The code is blank. That's against the spec. That's, it shouldn't do that. There was an early version of the spec that said, if you don't know what the code is, use an empty string. The updated version of the spec says, use the string unidentified. There's a bug open to fix that. But it's been hoping for a long time. So. Okay, but in this case, how would you know that the user typed an E with an accent without reading the last letter of the value of the input in which it was typed? That is a good question. And we will come on to that. All right. That is absolutely the right question to ask. But on iOS, or iOS? Is iOS the way to say it? Or is it iOS? I don't know. I work for Samsung. Okay, that's fine. I work for Google. So we can say it however we want. Okay, iOS will say the key is E with an accent, but it will say the code is unidentified. So that seems very sensible. I think that is more sensible as well. Yeah. All right. This happens on Mac. If you hold down E, you get this pop up. Wait, on a desktop? Desktop Mac. So it doesn't go E. It gives you a single E and then this pop up appears. But you get repeat key down events all the time you're holding the E even though you're not getting repeat E's. Weird. Which I think, like, this has been a historical weirdness with key events where you get like, they're asymmetric. You get many key down events and one key up at the end. And I think it's weird that Mac does this, but then testing Chrome OS and Windows, if you hold down the shift key, you get many shift key downs, which I think is totally wrong because that key just goes, there's no repeat action for that. So I remember I used to make games back in the Visual Basic days where you would control it with the arrows and I would just listen for the arrow down events and you would hold down an arrow and it would fire multiple events. But I think repeat arrow works because yeah, if you're on an input and you hold down the arrow key and the cursor goes along, that's a repeat action. Yeah. And I think repeat actions make sense, especially when you want to adapt to the operating system setting of repeat frequency. Yeah. It doesn't seem to make sense with shift or in this case. I'd say the thought is, is this is why people now do keyboard smash rather than type in LOL with many O's because you couldn't, it's not easy to type many O's. You can't just hold it down. What's keyboard smash? Well, you just do like. Oh, okay. That's cool. Yeah, yeah. That would make sense. Yeah, cause a lot of, on Mac especially, it doesn't do the keyboard repeating anymore. Doesn't do it on phones either. Yeah. All right. And also you couldn't do like was up cause now it's really difficult to type. And I think that's great. I'm glad that that era has passed us by. I'm still stuck in 2002. That was my university days. Right, next round. Which key was pressed? All right. All right. Here we go. Wait, can you play that again? Nope. Yes, I will play it again. Boop. So you just tapped the E. Yeah. So if I'm thinking it's being sensible, then I'm going to say like key code E key E. All right. All right. It's complicated. Oh, shut up. I jumped both feet into that one. No, but I think, wait, what? So Android is back with its unidentified and I don't know. So for the soft keyboard, it just does nothing. So androids stand on this and this is not who Chrome's doing. It's not really androids doing, it's the Android keyboards doing. Because if you plug a real keyboard into Android, you will be getting different answers to this. I mean. This is the visual keyboard doing this. I could plug a phone into our HDMI cable here and see if you want. I've tested enough devices as part of this. But enough to tell you that iOS does a different thing. All right. And that will give you... Wait, you can plug this to our keyboard into iOS? You can as well, yes. And I think the answers it will give are different as well. Yeah, hardware, keyboard, software, keyboards are going to give you different answers in this system. So the way Android looks at this is, yeah, like this is a software keyboard, all bets are off. Like the rules don't make sense. And so we're not going to have the rule. Whereas iOS is trying to be more helpful. I think it's good. Yeah. It seems much more sensible. Like it seems like the websites which are built around this will continue to work in a way that it expects without necessarily needing to be built, targeting a particular mobile browser. Agreed. Interesting detail on this one. The shift key is true. But it was lower case, right? Oh no, you did up a case. Yes. So even though the key wasn't being held by tapping, by putting it into like the shift mode, it treated it as if the shift key was held down. Yes. So when you highlight an input on Android and iOS, it starts in a case mode. Yeah. So if you type a character like this, it is going to treat it as if shift was held down. But you do not get a key event for the shift key. It's like the shift was there for the letter, but it wasn't ever pressed. It's kind of weird. So it's interesting. So Android here is explicitly incorrect in saying shift is not true because you did press like the shifted character. Because in... Well again, it would say the shift key wasn't pressed. Like, so I think both are wrong here because iOS doesn't say the shift was pressed although it was pressed when the character was pressed. But the character itself is shifted in that it's shifted to use the set of characters in the uppercase set. And by uppercase, I mean literally the characters that are in the uppercase of the... The lowercase keyboard keys. Well, I was thinking of the printing machine from mechanical key printing. Which is what these keyboards are based on. Interesting detail. If I now press the shift key. Yeah. No event. Like the web page can't detect that. You can't detect tapping shift like you can with a... Well, if I now press the shift key again, iOS will now fire a key down event for shift. And if I now press a letter, it will give me the key down for that letter, the key up for that letter, and the key up for shift. So it's kind of different to that initial first state. It's sort of all... It's all a little bit weird. So let's say you just start typing and you just press D. Does it also give you the event for shift being let up? Not when it's shifting the initial input highlight state. But in this state, because you've got... You've actually pressed the shift key. You have to load it in order for it to... Like you have to start a key down before it will fire the key up. Yes. Yeah, that seems to be the way it's defined. But yeah, on Android, it's just like, this isn't a real keyboard, so I'm not going to do much apart from firing a key down event for this made up key. All right, next round. Which key was pressed? I'm trying to be smarter this time. All right, here we go. You ready? Yeah. Woo, woo, woo. Well, that's not a help at all. So that was hello in Swipey Keyboard Speak. So, if I wanted... So here, if you wanted to be awkward, you could fire key events for every key that was crossed by your fingers. Oh, wow, yeah. If you wanted. Yeah. But I don't think it will do that. That seems like a difficult thing to engineer and also not super useful. Yep. So, I'd say maybe what might be ideal would be just, would be once they've, like, once the word has finished, then fire key down events for all of the letters in the word. As if they were individually typed. Yep. But judging by all of the previous things you've shown me, I'm going to say it's undefined, empty string, and shift key false. Okay. Well, in this case, it's actually quite... I'm lying. It's complicated. They're all complicated cases. But you were correct for Android. Yeah. Yeah. Once again, Android is back with its... I don't know. But you only get one key down and one key up, which I like that, because it suggests that there is a hello key. Which is sort of lovely, isn't it? It's a lovely thought. Makes me want to build a keyboard that has a hello key. Exactly. And with keyboards where you can add your own firmware, I bet you could. Yeah. iOS? Nothing. There is no key down event for this. Huh. Which I would say seems more sensible, because it's not pretending there's a hello key. Yeah. But it feels like there's been a shift. Whereas iOS in previous times was trying hard to be compatible. Now, we've got Android as the one that's firing a key down event, and iOS is just like, nah, this is nothing to do with keys. But I guess it's the kind of thing where if you're listening for individual key press events, ideally you're playing a game that listens to single key presses, or using an application where it's listening for commands from the keyboard, like control P or something like that. Whereas if you are swiping hello, the developer in all sensible situations, I would sincerely hope is using some kind of input field or text area or content editable, where reading the value of that would be the more sensible way to find out what the user has inputted. You're 100% correct. Yeah. This whole thing is messy, dealing with key events when you're actually dealing with input. Here's me. This is me in 2010. So nothing has changed since? No. Yeah. Oh God, 12 years. 12 years ago, it's one of my first ever talks, and I'm talking about keyboard events. The talk is called Events Left Behind, and my rap of it is keyboard events are awful. And they're going to get less awful. And do you know what? Here's the thing. They did, because it was a lot worse. But then smartphones became a thing, and things like, you know, that's when we got all of these bumpy bits with on-screen keyboards, not quite. I guess 2010 was still in the bit where not everyone had a smartphone. I definitely didn't. The smartphone web was really terrible. I remember building web pages for, I'm about to say web apps, but web apps weren't super-considered a thing. Well, they were kind of, but yeah. Ajax web pages, web 2. I guess we called it at the time. I think at this point iOS had, still had add-to-homescreen on the early iPhones, but yeah, super early mobile web. I think you hit upon, like, the whole problem that we have here. So the reason this wasn't working for the Wordle clone was because this never worked on Android, because it was just firing that undefined thing all the time. And what you said before was completely correct. This split where we've got the thing printed on the key and then the key itself doesn't make sense for user input. It doesn't fit properly. And it's not just on-screen keyboards, like swiping and all of that sort of stuff, but things like voice input. Like it's not a key, right? Yeah. There are cases like this. H-E-L-L-P, missed. Sorry. And then I press space and it auto-corrects. Again, the input is completely detached from the keys being pressed. There's also just like command and V, pasting, again, the input is then completely detached from the keys you're pressing. Also, in that case, you probably aren't even hitting the keyboard. You're probably, at least if you're on Android, you're long pressing the input and pushing the paste button. Yeah, exactly. So yeah, you wouldn't even make an argument for a key being pressed there because it's not even part of the on-screen keyboard. Mariko did this for me. This is how they're typing on a Japanese keyboard. That's so cool. I love the way that it collapses once it's phonetically spelled. Yes. So that's what's happening, which I learned last week, is it's typing in a phonetic way and then you get the suggestions along the top for how to collapse that down. And again, if you're just listening to key down events, you are not going to get anything related to what the user is intending to be typed. We have these other events. And so I guess the kind of moral of the story I'm trying to get to here is that higher level events are always safer if they're actually related to what you want. Before we continue, so input and before input, use both of those before, selection change, use that too. The composition events. Yeah. Hold on to that. All right. We'll get to that. But if you've never used composition before, then neither have I either. Neither of most people watching this. But I will get on to what those are. But for now, I'm just going to stick to the input event because it's the simplest one is the thing that would have solved this, this case. And is hopefully five letters long so you can put it in your slide. Isn't that nice? Yeah. Isn't that nice? Yeah, fits perfectly. So the input event fires whenever the content of the text field changes, which is all we really care about. So we can have here our div and then a div for every letter. I'm going to give this area hidden because I don't think it's, it's purely a visual thing. We get our hidden input back. That is the thing that people are going to be interacting with. It's the things screen readers are going to be using. And then over in the JavaScript, you can just get those elements, get all those letters, and then here's their input event. When that happens, you can take all the letters of the input and then for each of these divs per letter, just stick the correct letter in there or an empty string default. Yeah. And that's the fix. And that's the Versailles wordal clone. They switched to using a pattern very much like this. Yeah. And it works. It works for swiping typing. It works for vocal inputs. It works for pasting. Which is much more accessible because there are many inputs that people will be using that don't necessarily fit to the smartphone input because that doesn't work for lots of people. Yeah. I think the equivalent is if you use mouse down and mouse up, you're going to lose a lot of users compared to using click. Like if you actually mean click, we've got this weird situation on the web where click doesn't mean with a mouse. Yeah. It's a kind of activation event. It has become over time. That works with a keyboard. These higher level events will work for many more people. Yeah. So in this situation, you want to detect when the user has finished typing, do you listen for a return event or do you wrap the input in a form and listen for the on submit? What would you do? I would do the on submit personally. So enter is on submit? I wouldn't. So I would have it so that I would just listen for the on submit because hitting enter in the input would fire the submit. Yes. So this is the app I made, the word analyzer, so I get to advertise it here. And so I use a very similar system. I also use the selection change so you can do selections in this fake input. It's just a collection of divs. Oh, that's nice. And you click on it and do all that sort of stuff. But when I sent this out to people, and it is you are just interacting with an input under the hood, like an input per row. I send it out to people and the first bit of feedback I got from everyone was, type my first word, I press enter and it's submitted and that was dumb. That's not dumb. That's how the web works. But it turns out in this situation that is not what anyone expected. Really? Yeah. Yeah, so I had to... You had to listen for the on submit event and cancel it. I canceled enter in these. In fact, I make it so enter goes on to the next field. And there's lots of other little subtle interactions I had to put into this. A form with multiple inputs where each line is an input. Yes. And so... Yeah, so I guess it makes sense if each line was a form and submitting it would move you on to the next one. Yeah. I think it's like... because I've made it, it doesn't look like normal inputs. People don't expect it to behave like normal inputs. So I'll put a link to the code for this because there's lots of little subtle interactions I've put in there that aren't like how web forms work, but it seems to be how people expected this to work. Yeah. You asked about these? Yeah. Just so happens. Got a few slides on it. Fantastic. I didn't know what these were. At first I dug into the spectre to find out what they are. And they had to do with this. Ah. So this is a composition. And you can sort of see how the text has a blue outline at the top there. So that's part of a composition. So the composition starts when the first letter is entered. And then it's composition update as these changes happen. And in the event it will tell you which character range it's covering. And then once it's committed, that's the composition end. Wow. So the input will give you all of that information as well. So does that apply for Latin alphabets as well? So if I'm typing in hello. Not like this. I even guessed the word. Yep. What do you think? I'm going to say... So it's not getting a blue box. So this is a different operating system. Oh, yes. Yeah. Assuming that this also supports composition events, then I would hope so. I would hope so too. So here's the event order. Yeah. Android composition starts, composition update, and then you get an input. Yeah. And your input type is insert composition text. And you get the HE. HE. And so these are composition updates. L-L-L-P. You even get it wrong. And then it does a composition update. And it's hello with the space. And that's that done. Cool. Over on iOS. Yeah. Input. Input. Input. Input. Input. Insert replacement text. Input. Input. So again... So this is... So this is off-spec behavior. Kind of. I will... I... The spec is so vague. Yeah. I don't blame iOS for breaking the spec here. It... To me, I agree with you. It does feel like a composition to me. It feels the same or very similar to the Japanese input. iOS will fire composition events for the Japanese input. Yeah. But for the auto-correct, it sees that as something quite different. Something different. Huh. I don't know. I guess it's also, like, like it has the same effect. I mean, it's like, it's always annoying when there's differences between browsers, but, like, this actually does seem sensible, though. It's... Yeah. Both seem okay and the spec doesn't say anything about auto-correct as far as I could read. So, yeah. I mean, I think the moral of the story is, uh, really, keyboard events are awful. Um... But for a lot of the stuff that I've shown here, I've filed bugs. Yeah. So hopefully, they'll get less awful. And if you want to come here in 12 years' time, we can see if that's actually true. If we can still come here and talk about, like, the web in 12 years' time, and it's still, like, a cool place for people to build in, cool stuff like Wordle, I would definitely be happy to come back in 12 years' time, even if the text events are still awful. Brilliant. So, is the keyboard a prop? Yep. Okay, cool. I was wondering what you plugged it into. No, it's not plugged into anything. It's looking to itself. No, it'll explode. What happens if I press this key? You're in a short circuit, the keyboard.