 Hi, my name is Jacobo and I'm a software engineer at TIGALIA. I've worked on browser solutions for the automotive industry and, more recently, inaccessibility on the upstream Chromium browser. We at TIGALIA are open source experts and consultants with almost 20 years of experience. We are best known for our contributions to open source browser technologies in the Chromium, WebKit and Servo projects, and compilers to the major JavaScript engines, V8, JavaScript Core, SpiderMonkey, also Gile, also contributions to graphics and multimedia in the Mesa and Streamer projects. In this presentation, I would like to start with an introduction to what accessibility means. I will follow with a more thorough explanation of what accessibility APIs are and how they work. I highlight two important aspects, the accessibility tree and the communication between the elements involved. Finally, I'll present some ideas on how to use these technologies for safe driving purposes. So, first of all, what's accessibility? We can say accessibility is to make a product, device or environment available to as many people as possible. It applies to anything from building to software. It's a simple idea with a lot of challenges behind it. In the IT world, we reach accessibility through the uses of assistive technologies shortened as ATs, which are software, hardware or a combination of both, used to enable use of a computer by as many users as possible. ATs can be hardware or software provided. Examples of hardware ATs are adaptive input devices like special mice or keyboard or hearing aids. Most usually, you will hear about software solutions. Some are provided by the system like a magnifier to increase the size of the visualization area around where the focus is, high contrast mode for operating system and applications, things like setting a bigger mouse pointer or adding a trail, captioning to provide visual feedback when you cannot receive audio or the opposite screen reader to transform text into audible feedback. Notice that accessibility can be helpful for anyone in different ways. It's not just about disability. Captioning is very useful when you cannot enable the sound for some reason like when you don't have earphones in a silent environment or to access content in other languages. This is an example of an application, LibreOffice in particular, using a high contrast mode. High contrast is provided by the window manager for things like window decorations or common widgets like scroll bars or buttons. But applications can help provide a high contrast experience, for example, including an appropriate set of icons. In web content, it's harder because you have to apply high contrast to the many different styling choices made by web authors. There's an entire set of recommendations for web authors regarding this. See this example of Twitter when the Firefox browser enforces high contrast on it. Black borders, white backgrounds, black text and blue links to make clear what's a link and what's not and whether it's been visited or not. But when someone is set to work on accessibility in accessibility, they usually work on or around the system accessibility APIs. These APIs expose the application UI in a tree-like format for ATs to consult and even manipulate it. They are platform-specific. Windows, it's called iAccessible2. On OSX, it's called NSAccessibility. On Linux, for historical reasons, there are two APIs. One for applications, ATK, and the other one for ATs, AT-SPI. All these APIs are different but with shared ideas and purposes. They take ideas from each other as well as from other accessibility systems like the one in Java. They share the tree-like structure and common properties like name, description, role. Role is an important property that defines the expectations regarding the behavior of a UI component and also defines its API. Common roles have a window, container, button, dialogue, alert. They have different names in each of the different APIs but they are usually equivalent roles in each of them. How is the accessibility tree built? UI toolkits handle it for common widgets. For example, application developers don't have to do anything special to make a button accessible. Custom widgets and applications require explicitly handling this, implementing part or all the accessibility API for the widget. While applications with their own toolkit must implement the application-aside API themselves. This is the case of for multi-platform applications. Some have internal UI abstractions which are translated to native ones. That's the case of LibreOffice. Other have custom toolkits that render directly using low-level primitives, and that's the case of Chromium. This is a real example of an accessibility tree built out of standard widgets in Linux. The visible text is used as the accessible names of the items. The roles used are application, frame, which means window, some blank space called filler and a button. It required no special intervention from application developers. Web browser developers must build the accessibility tree out of web content. They use HTML semantics to create the tree and populate it. This is a simple example taken out from the Chromium project documentation. A simple HTML document is transformed into an accessibility tree. It follows more or less the tree structure of the HTML document. Notice how HTML semantics are used to fill the names of the items. The title tag in the heading is used as the name of the top-level web document node in the accessibility tree. And the text input is labeled by the item that is next to it. Sometimes HTML semantics are not enough, so we have the ARIA standard to expand HTML and fill gaps. It can be used to add accessible labels and descriptions from invisible attributes or from other HTML documents. Notice that HTML semantics should be used when available, like in the previous example. ARIA can be used to add roles, for example. A piece of text that briefly appears in a web application saying changes saved. The ARIA alert rule can be used to notify ATs about this message and make it read immediately. It can also be used to describe hierarchies not explicit in the DOM tree, for example, to indicate the correct place in the document hierarchy of a piece of floating text. This is a real-world example of an accessibility tree from the Chromium web browser. Notice how it seamlessly exposes the application UI and the web contents as part of the same tree. Roles are shared by the application UI and the web content, for example. A bottom is the same role, be it a native bottom in the UI or a bottom in a web form. The only indication of something being web content is the role of the container role. It's the role of the container element. Before we move on, let's make a recap about accessibility APIs and the accessibility tree. These APIs are platform-specific but based on shared ideas and functionality. They expose the UI of applications in a tree-like format, which ATs will consume. This tree may be automatically generated for standard widgets with no specific coding applications. But in the case of more complex software, app developers must implement accessibility APIs for custom widgets or toolkits. Web browsers are an example of this more complex software. In addition to their own UI, they must generate the accessibility tree for web contents. To do that, they use HTML hierarchy and semantics with the ARIA standard for advanced features. Now we have learned how the accessibility tree is built. Let's explain how ATs access and consume it. This is a high-level view of how an AT and a screen reader, for example, works. We have two ends, the screen reader and the software running on the system. And we require bi-directional communication between them. This is achieved with inter-process communication. On the software side, there is usually an accessibility implementation provided to some degree by the system or the UI toolkit. Like explained before, standard widgets should be accessible more or less out of the box. This is a more specific scenario of accessibility on a desktop Linux system. On one side, we have the Orca screen reader. On the other side, a typical Linux application built with GTK or Qt toolkits. These toolkits implement the ATK API. Finally, ATSBI is the bridge that connects both ends via IPC, D-BUS, in this case. There is a library for interaction with ATK. Let's see these individual developments with a bit more detail. Orca, the screen reader, is an AT. It's an individual piece of software running its own process in the system. ATSBI is the bridge that connects Orca with the software. It runs on D-BUS for IPC. It also keeps a global registry of system accessibility, so applications only need to deal with their own accessibility. For example, they may use identifiers that overlap the identifiers in other applications. On the application side, there has to be an ATK implementation. Our software may be built using the GTK or the Qt toolkit. Widgets provided by them already come with the corresponding ATK methods and properties. If we are implementing custom widgets in our application, we will also need to implement ATK partially or completely. Finally, there is a library that will translate between ATK and ATSBI. It is provided by the system to be dynamically linked by the software and run inside the same process. This is true for most Linux applications, but as we mentioned before, it's not the case for most browsers like Firefox or Chromium. They will not use an existing toolkit. Instead, they will implement all the widgets from their own, together with all the required ATK properties and methods. They will still use the rich library instead of using IPC themselves. Finally, here are some ideas of potential uses of accessibility technologies for safe driving purposes. These APIs are very powerful. They allow to access the entire application UI and contents in a structured way. Alter contents for presentation and even manipulate applications remotely, like simulate a button press or a mouse click. Since accessibility tools can access content and present it in many ways, what can we do with that? The most obvious idea is text-to-speech. This is what screen readers do, because it's useful to visually impaired people, but it's also useful to drivers who should keep their eyes on the role. Accessibility tools can provide a simplified presentation of the content, removing banners, menus, etc., and providing content in the least distracting way possible. A special case of simplified presentation can be the creation of a table of contents presenting only headings for quick inspection, so you can skim through a big piece of content and extract what you need more quickly. Combining this with text-to-speech would be very useful. We could adapt content to multiple screen sizes and shapes, in addition to providing a simplified presentation of the content. We could make it fit in a container that content was never intended to, for example, a teletype-like screen on the cockpit. We could even go further manipulating the content, like providing on-the-flight translation of contents. I imagine this could be useful if you are on a trip abroad and want to check the local media for traffic news. We could also apply accessibility tools to innovate on content navigation. We mentioned before we could extract an index for the contents, since accessibility tools can also be used for interaction. We could implement a way to shortcut directly to specific sections of the content, combined maybe with text-to-speech. We could use accessibility APIs as an interface to special input devices, for example, buttons on the steering wheel or even voice commands. There are of course multiple ways to implement these features. We can make them work through accessibility APIs, as we have mentioned, or we could implement specific applications that have voice control, text-to-speech, etc. We could even modify the browser engine to implement some of them. Think, for example, on how Chrome integrates with Google Translate services or the reader mode available in most browsers that strips non-content parts of the website. We have all those options. When would accessibility APIs be a good fit? I think they are the best path if you have to leverage existing applications and content. Accessibility support is already there in toolkits and browser engines. It would require to audit and test thoroughly your software, though, to make sure it follows best practices with regard to accessibility, especially web contents. All in all, my goal today was to let you know about accessibility APIs and how they work, so you could take them in consideration when designing your systems. Thank you for your attention. I will be available for your questions and feedback in the channels provided by the organization. Don't forget to visit the Igalia Virtual booth, where you can ask me or my colleagues about web browser technologies. Hope to see you soon. Goodbye.