 So, I was going to start my presentation by gauging the energy in the room, but I think that just woke us all up since it's the last, I'm the second last presentation of the day. So yeah, my presentation is actually about Mark. Mark is a cool guy. We saw that yesterday how Mark could just go thanks to Aritz Medina's presentation, so here I'm actually going to present, joking aside, a love story. It's between Mark and Listin. Okay, this is so bad. I'm sorry, I'm sorry. Yeah, so it's actually about marking as a term, marking, selecting and listening, readability, things like that. And the focus of this presentation will be about switching from the mouse being your primary input device when you use a computer to the keyboard. So that's what the subtitle is about. So who am I? My name is Juan Carlos Corona Romero. I'm a software developer engineer in Canada. The engineer is a protected term, so I can't call myself an engineer there. I don't even know if I am. I like the idea of an engineer actually being held accountable for things because nowadays, you know, I could do something to screw up someone's digital life in a really bad way, and I want to, yeah, people should be held accountable. So yeah, I work for a company, a small company in Vancouver, Canada, known as Evident Point. My role there is that I, the web technology is what I like, and I've really grown in that role at Evident Point. So, yeah, WebTech is my jam, and they really like that. And Evident Point really is trying to push web technology forward. We like the idea of progressive web applications. Most of our products are steering towards that and embracing the open web platform. And that's my GitHub short username. So, as I mentioned already, a little bit of Evident Point, but Evident Point, we provide digital publishing solutions with eBook, with our numerous years of eBook and eLearning expertise. We provide, we contribute to open source efforts, mainly through this project known as Redium, which is to develop an open source ePub reading system, but now it's gone beyond that with keyword Redium architecture. I'm in a little bit of hypothesis on the side here and there. And, yeah, like I mentioned, ePub, PDFs, we used to be a PDF-first company, but we're, we're shifting towards HTML documents and things like that. So, we also have cool tools that let us, like, internally convert PDFs to ePubs and so on. So, I'm gonna set some context, and this, we have a product, a commercial product known as Active Textbook, eBooks plus annotations in some form. So, the idea, the premise is that you upload a book, you read in a web-based viewer, as I alluded to, you enrich the content with various forms of annotations, rectangular-based annotation visualizations in different ways. You can also create interactable elements, like images, videos, quizzes, but, you know, there's also the core highlights and notes. And I'm gonna show you a little bit of that. This is to set the context, kind of a little bit of a plug too. I mean, yeah. So, this is our sample. And as you can see, these are the various forms, point-based comments, discussions, you know, you have these, you can create something by clicking somewhere, comments. I think this is read-only mode, but I can't select, I can't drag and select a square here to create something. But you have these overlays. Like, these are enhancements on top of the book. This is an open-stack textbook. So, you can see you can, we can click on a video about, I don't know, referencing the mosh, but somehow there's a link and so on. But there's one problem, I'm using my mouse for all of this. I can, I can use our toolbar and so on. But then again, so what happened with this product over time, actually, and I want to send, this is, I will go into that, but I want to take a break here and look at these. I really like numeronyms, if you don't know what they are. So, this stands for internationalization. And between the I and the N, there are 18 letters. And that's kind of like an acronym too. I'm guessing people here in DC like acronyms. So, that's why I'm showing this here. There's also another one, localization. And then there's, of course, W3C, the World Wide Web Consortium. So, you can also use them for repeating characters. And there's this one, which is my name, because it's very long. And I think maybe this, I was hoping this would be a numeronym for something, but I don't know. So, maybe it inspired something? And then, but this one's really what I want to focus on. And this is our ally, this is accessibility. So, a company approached us, and that's to highlight that, a company, a consortium of higher level education institutions that develop solutions for education and so on. They approached us and they said, we like your product, but it's not accessible, and we really care about that. So, can we help you, can we collaborate and make your product, you provide our product, I mean, you provide the product, and then we tell you and give you some, you know, some requirements and can you go and implement those requirements in your product for increasing accessibility, and they did. So, I got, I didn't get directly tasked this, but members in my team did. I was aware from indirectly about what was going on, but it really started getting me thinking, as a web developer, how do I do this? How do I implement this? Or like, how do I start? So, and I'm a very practical person, pragmatic. I don't, I mean, nowadays I can, I'm more comfortable with reading specifications, specification level information, but, you know, the W3C published the WCAG, which stands for the Web Content Accessibility Guidelines. So, those are the guidelines that we followed, and here are some extracts from there. So, there is a principle known as operable. User interface components and navigation must be operable. Navigation must be operable components UI. And the one I'm choosing here is this guideline for keyboard accessible. Make all functionality available from a keyboard, and this kind of sounds like a rule. So, there's another one as well, that I highlighted here, I selected. Another principle, the understandable principle. Information in the operation of the user interface must be understandable. Now, this one was a little bit more abstract. What does it mean? Guideline readable, make text content readable and understandable, but isn't it already readable? But, you know, like, not everyone can read visually. What if I have a visual impairment, or I have some disability where I can't read it, or I have a learning impairment where things are just not understandable for me. I need to be, things need to be pronounced. I need to be able to use text to speech. So, let me show you some of this. But before I do that, I want to go back a little bit to what we did with Unison. So, let's just go back to this one here, the baseline. I have a hit tab. Nothing's really happening, which is how I interacted the keyboard. Tabable components. But here, we have a lot more now. We can go, we can tab through everything. And I'm just going to let go of my mouse. Actually, I need to turn this thing on so you guys can see what I'm hitting, unless that's too distracting. Yeah, so, okay, let me just, let me, let me just use my keyboard and, all right, let's, let's look at the table of contents. Sweet, let's go through that. Yeah, and that seems like I can, I can go back, can search. I don't really want to search for something. I don't know if it'll work well. And then there's a text and speech setting here, which is, and then I can, oh, what happened here? Oh, I can probably use my arrow keys now and select some more stuff. Actually, let's pick a color. I like, I like purple, yeah. Green, I don't know if that contrasts well with purple, but let's use yellow. Playback speed's fine. All right, so how do I do TTS shortcuts? Okay, now let me, let me get out of this something and then just go to the main content. And all right, so I want to start, I want to invoke the text speech. So I forget how to do that because there's a new keyboard shortcuts that need to learn if I'm invested in this. So there's element navigation on and off. Okay, let's turn that on. Oh, cool, I can now curse, use my go through and some elements here. And let's go see if I can read word level. Granularity is fine. Sentence, maybe that's default. Start reading R. Someone on his cell phone will gain an appreciation for the difficulty in reading small sections of growth. Yeah, it's kind of loud. Someone trying to read on his cell phone will gain an appreciation for the difficulty in reading. Okay, good. Yeah, so it's supposed to go through, you can like go through the whole book and I think it just broke. Oh, no, focus changed. Yeah, when you're dealing with multiple iframes, it happens here, the focus is kind of weird. But I, yeah, so I think you get the idea now of Texas speech and how I'm using the keyboard for this. So let me go back to my presentation. So I've shown you that. Now let's move on to text selection. So think about text selection. How would you be able to do that in a web browser with just the keyboard? I drew a blank initially. Is that an anchor really? I, without using my mouse without, yeah, I need to set an anchor. That's right. We need to think about the primitives here. I need to set an anchor. But how do I set an anchor? How do I, I need, I think what I found was that you need a new form of operation and that didn't need the mouse. So let's go to Chrome and let's go here. And I can tap through elements, but I just don't know how to select text. Anyway, I found out that Firefox has a carrot browsing mode that you can invoke with F7. And it tells you about it here. You press F7. Pressing F7 turns on carrot browsing. It warns you here. Like this feature does this. Are you sure you want to do this? It's something you might not know about. Are you sure you want to do this? And actually it asks you every time you click F7. It's not, there's that option here. Do not show me this again. But if you don't click that, it'll warn you. So now I can click and it gives me like an I-beam. It seems like I can, I can't edit the text, it's read only. It's like content editable, but read only. Now I can move the cursor around with my keyboard. I can hold shift and I can start making selections. And I can start adding modifier keys to like, if I select shift and alt, like naturally an operating system, I feel like that goes through the words. And I can do, I think I can do some more than that. But yeah, so now I have this. But let's go to Chrome, because we all love Chrome. Doing F7 doesn't do anything. I don't think this is, I should have done my research on this. And that's why I wanted to be here and ask about what's the history between this or about this carrot browsing mode. But apparently Chrome has an extension you can add. It's marked here as accessibility. It's by Google. You can add it. I'm sure they use web technologies to implement this, which I'm going to start like reverse engineering, reading the source code if it's available somewhere and seeing how it works. Maybe we can make that a built-in feature with something like that. Add it to Chromium. That's, yes, that's because I think the rumor was that Microsoft got them to want them to do that. So now that I have that, I can turn it on and then I get, I think I get a similar experience, but I got to refresh the page because I know how that's how Chrome extensions work. Now there's a weird visual bug here with this page. So I'm going to go to the client test. And here I think it's, let's refresh that. I don't know, is it working? Yeah, it's working. Okay, so now there's not, the visual bug's no longer there. So I'm going to start selecting some stuff. And as you can see, the very familiar hypothesis Adder pop-up comes up. And I hacked it actually to just a little bit, a little UX improvement so I can, when I hit tab, the next tab will jump into the context menu because it didn't do that naturally. This, the tab order is determined by various factors and the Adder is actually the bottom most in the element order. So you'd have to tab through everything to get to it. So what I did was I wrote this change in the hypothesis source code, like 20 lines of code in the Adder.js that just like listens for the next key down. If it's a tab, it prevents a default and then it does that and then it lets go. Like it goes back to normal. It could be improved, but this is just a hack. So now we can do that and we can continue using the keyboard. I had space and this is actually pretty nice. The editor here is, I can't tab through it. And ta-da! I created an annotation. I anchored something with Carat Mode and I selected some stuff and I tabbed through the interface but now I don't know how to get out. How do I go back to the main content? But yeah, the experience is still, this still needs to be developed. And I just want to show you that there are a couple of issues on other browsers too. Actually, I'm going to move on to my next point because I'm checking time. But I'll use the next browser to test my next point. So my next, yeah, so we already checked that out. My next aspect here is, or I forgot the word, also screen readers. So I also thought about, if you don't know what a screen reader is, it's assistive technology for visually impaired users that you can get, it can read the screen and it can tell you some output with text to speech or with a Braille display about what you're seeing. And Apple, I'm not that very familiar with a lot of the screen reader technologies or screen reader software. So I like that Apple has one built into the operating system which I can just invoke now. And this was my little, this is kind of how it looks like. And let's go look at how to turn that on. So if you have a new touch bar MacBook, you can hit the fingerprint key five times. Accessibility options. I really hope I can change the volume. I think you can't, but I think I can. Okay, if you can help me out, that'd be great. Let's see. Yeah, that's it. Welcome to Mac. TheValde, Mark and Listen. Google Slides. TheValde. Window. Mark and Listen. Google Slides. Frame has keyboard focus. Open search or Xamant. Inacopem. Inacopem. Inacopem. Inacopem. Inacopem. A-R-G. Example domain. Frame. So you are currently on a frame, inside of web content to enter the web area, press control, option shift down arrow. To HTTPS Conan's Example Domain. Frame. Hotkey invention. You are HTTPS kilnum slash slash en dot Wikipedia. Escape button. You are currently on a button. To press this button, double tap anywhere on the touch bar. Escape button. You are currently on a button. To press this button, double tap anywhere on the touch bar. Stop. Yeah, OK, good. Thank you. I need to calibrate myself here. So what I want to show here is that I'm using Vivaldi, which is Chromium-based or Chrome-based. And it doesn't actually know how to read this, because there is some black magic to me. At first, it was black magic as to how can a screen reader built into the operating system be able to dive into the DOM. And apparently, there's a lot of it. It's a very complex process. But web browsers really need to actually expose this. I don't know if Chrome's doing it, but Vivaldi didn't do it. But there's something called the accessibility tree and accessibility object model that's being pushed. So hopefully that, in the future, improves. But I'm going to have to go with what's native here, and that's Safari. Space with code, space with Firefox containing window example domain. Sys Safari, example domain, window. Example domain, example domain, web content. You are currently on web content this. OK, I'm going to skip into here. You are currently on a heading level 1. Inside of web content, this DOM. I can use this. I can use this on Airbeam. This heading level, heading level 1. Example DOM, this domain, visited. This DOM, heading level 1. Example domain, this domain is established to be used for domain example. Example DOM, new list, domain, sys, select, letter by letter, I can use this as well. U4, illustrative, xin.u, u, u, m, u. I think that's another way where you just need to be comfortable. Example domain, selected. That's another way to select text with the key word. You are currently on a heading level 1. Inside example. Let me turn on the hypothesis book marklet here so we can get into Safari. Example DOM, this select domain is established. Selected. You are currently on a heading level 1. Inside of web content, to exit this web annotate button. You are currently on a button. Inside of web content button, hypothesis and location viewer frame. I'm conflicted. You are currently on a frame. To enter the web area, press Control, Option, Shift, down arrow. Index, so it's a little bit of a disconnect. Should I be using the voiceover keys? New, hide toggle, or we hide. New, type of, type of, new, hide, new page note, button. I don't know, but maybe I'm just not used to this. Empty. Now in example domain, window, list. M, this domain heading level 1. Example domain, you are currently on a heading level 1. Inside of web content, to exit this web area, press Control, Option, Shift, up arrow. Vastly configurable. You can tune it to however you need. And there's probably, if a website or a web app does a really great job with implementing those principles that I brought up from a WCAP, then the experience can get better. This domain is ignoring the annotate, button, edit text, A-E-L-O, add to post to public, button, edit, share this page. G-M-A-L-L-N-L-V-S-E-D, add to post to public, add post to down point, cancel down post to public, button. You are currently on a button. To click this button, press Control, Option, add to text, add to text. Edit text, post to public, button. You are currently on a button. To click this button, press Control, Example. You are currently on a heading level 1. Inside of web content, application. And let's go back to my presentation. And I actually didn't really know how to end this off. But after having a great conversation with Doug, he told me about this thing called the curb cut effect, which was so in the past, historically, you needed to fight for increased mobility around the city. And so now we probably take this for granted. I took it for granted because I didn't know about the history. But the curbs are cut. And originally, they were for people who needed to use wheelchairs or other mobility needs. But now we all benefit from it. If you have a bike, if you have a stroller, if you have one of those scooters around the city, you can use it to get around. And you're no longer blocked by there not being a cut in the curb. So what I wanted to conclude with this here was that if we start implementing these accessibility enhancements or start thinking about it in the way that we develop or we implement or design things, there will be features that will be very valuable, not just for those who are disabled or who really need them, but there will be a benefit for us all. And I really like this because I really hold to a high regard a website or something that uses. I like using the keyboard. And you've probably noticed throughout my presentation, I have this tile-based thing that I can use. I can open up a, actually, it's supposed to be tiling. I think it broke. Anyway, I don't have to use the mouse to manage my windows. I can go through, let's see if I have one somewhere. But I think you get the idea. I really like using just the keyboard. And I am benefiting a lot from this work. So that's how I can end this off. That was great. Thanks for bringing us back from the tornado one. Any questions? I see Jeremy approaching the mic slowly. Will you approach the mic? Maybe I should turn this off. Hello, I'm Lee. Cool. Thank you very much for that demo one. I've not really tried, well, not recently going through the whole process of creating an annotation and everything else. Interestingly there, that some of the work we did a while back to make hypothesis work better on mobile devices. So when you were at the fact that the adder actually shows up in the right place when you make a selection, turned out this actually solved the problem with the keyboard, excessive with at least making it work when you make a selection with the keyboard, although that wasn't the original intent at the time. I think that's a nice example of when a browser has events or hints that doesn't specify exactly what input device you're using. So here, for example, we listened to the selection events. It doesn't matter whether it's keyboard, touch, et cetera. So that's a nice advantage of leveraging the platform. One thing I wanted to say about inspecting the accessibility model, I've found Firefox very helpful. There's a separate tab in the browser which shows you the accessibility tree. I don't know. I have a feeling Chrome might have. I think Chrome does have something. Yeah, that's the view I was getting at that I've found quite helpful for debugging this kind of thing. Yeah, it's a more reduced version of the DOM where the attributes that matter are isolated away. One thing I would say is that VoiceOver does have an option so that shows just the thing on the bottom left-hand corner without actually reading any of the text out. That makes life, for those of us who have, yeah, that makes life much less painful. I should have turned that on, but yeah, thank you. But cool, thank you very much for showing us that. And yeah, I'd probably be interested to catch up with you about the change to the adder and the tab order. Sure, yeah, I can send you a patch, yeah. I really just want to thank you. I think that's really awesome what you've shown us today and it's usually exciting for me. It's the coolest thing I've seen all week and I'm really eager for us to, that hypothesis to benefit from your work on accessibility. So thanks. Thank you, thank you.