 Good afternoon everyone. We'll get started here in a better minute. You all got very quiet It's one o'clock or no it's 12 o'clock might we'll get started now Thank you for coming out. What we're presenting on today a smarter way to test accessibility a Comparison of top tools light house 10 and IO and the wave API My name is Everett Zufeldt. I'm the director of technology at my planet Started speaking at Drupal con in 2010 at Drupal con San Francisco Where I did a core talk on accessibility I was the Drupal 7 accessibility maintainer. I participated in the HTML working group at the W3C mostly looking at issues of accessibility in HTML 5 and Now I work at my planet and I support a growing team of technologists using tools like Drupal and React and others To build cool applications for our customers My name is Laura Johnson. I'm a developer at my planet. I've been a Drupal developer for over 10 years and I'm also involved with the Drupal community. I help facilitate the Drupal user group in Toronto and I've for the past few months. I have been researching accessibility Tools and testing them. So what are we going to cover today? Just kind of list of topics make sure and you're in the right spot First thing we're going to talk about we won't do an intro to accessibility really will assume everybody here has a Rough understanding of what web accessibility is. Maybe I'll just touch on it for a minute We'll talk about the difference between automated and manual accessibility testing and what that really means We're going to give a really high-level overview of the three tools. We're looking at today lighthouse tenon and wave We're going to talk about the browser extensions Each of those tools makes available and how they can make your life as a tester easier or as a developer easier You can talk about the command line interface available through each tool and how those can speed things up for you It's the DevOps track So we're going to talk about continuous integration obviously and how we can take some of those command line interfaces and integrate them with the build and deployment pipeline to make testing more robust to more reliable and Then at the end we'll spend a few minutes talking about what the future could hold and how we might be able to apply some Concepts around machine learning To expanding upon what these tools are able to do so in a nutshell Web accessibility testing. There's a few different standards out there. The one we talk about the most is WCAG Which is web content accessibility guidelines that being said a TAG or a tag the authoring tool accessibility guidelines are also very important WCAG is a set of guidelines from the W3C which are primarily designed To make content on the web not expand that to say web applications more accessible to persons with The following types of disabilities blindness and low vision deafness and hearing loss Learning disabilities cognitive limitations limited movement speech disabilities photosensitivity and combinations, so Primarily the WCAG The guidelines are there to make Your jobs easier designing and developing and testing web applications and web content that will be useful by persons with a broad range of disabilities and I'm sure if you've sat in sessions before Intro or intermediate sessions about web accessibility you've heard over and over and over again Automated tools cannot fully test accessibility. It just can't happen and that's true I'm not we're not here to tell you anything other than that It's interesting Carl Groves who's the founder of the tenon IO tool that we're going to be talking about today tweeted earlier today and he said In response to the question can the tenon tool certify my WCAG compliance? Sounds like a question. He'd get a lot The answer is no Accessibility tools are diagnostic tools not judges now Carl's got a vested interest in convincing the world Automated accessibility testing tools can certify your compliance and indeed if they could and if his could He'd be a rich man, but they can't and we'll talk a little bit more here about where some of the gaps are so I did a few contrasts here up on the slide. We'll talk through each one to just Help dig into a little bit Where the gap is in some of these tools and this is clearly not an exhaustive list a Tool can tell you very easily if an image in your HTML or in your DOM Has an alt attribute That's easy that you know most most developers in the room would be able to to write a couple lines of code in their preferred language And answer that question What it can't do is tell you if the alt text is accurate given the context of the usage of that image So I can tell you if there's text in the field. I can't tell you if the text lines up with the with the image What a testing tool can do very easily is tell you whether or not your form field has a label Note will show you in a little bit with some of the browser extensions that there's a form field on the Drupal con Nashville website That doesn't have a label What it can't do is tell you if the label or the description for that form field is Clear and easy to understand The tool can easily tell you whether or not there's headings in your document and we can have Debates about whether or not there should be more zero one or two first level headings on a page for years and years It can't tell you the tools can't tell you today Whether or not those headings are in the proper hierarchy to support the meaning of the content that you've written a Tool can really easily tell you if your HTML is valid There's lots of validator tools out there to look at your document and tell you if the HTML is valid HTML What it can't do is tell you if the HTML is semantic It can't tell you if you've used the right elements to represent the right sections of your document And then this last note I've put here says custom implementation. So they're really a tool can't do anything for you at all here today Does the application user interface behave as expected? And we'll show you in a little bit here when we take a look at the Drupal con website We look at the drop-down menu none of the tools that we're looking at today Actually identify the accessible the accessibility barrier with the drop-down menu And that's because it's not strictly Speaking something that the tools can test for the tools don't know how the menu ought to behave So there can't be a rule set that evaluates whether or not the menu behaves in the right way We'll talk about that a little bit more when we do the testing Really what this breaks down into if you take if you think this through if you look at through Tools can test to make sure that something exists or that it doesn't exist for the presence or the absence of data Tools can test to ensure Tools can test to ensure that things are present in the document What they can't do is assess the meaning of the content and Evaluate the semantics that you've applied to it the tools don't know how to do that today So really there's two types of automated tool out there and I made this up So somebody can argue with me and tell me there's a third. That's fine There's really two types of automated is that accessibility testing tool in the market today there's tools that are primarily geared around making the job of developers easier and And I'd extend that to say developers and testers So there's tools that are designed for that purpose and there's other tools that are designed for conformance and reporting So if you're a developer or a q8 specialist in your role is to build and test accessible applications There's tools that are built to make your job easier and If you work in a reporting function a conformance or a regulatory compliance function within an organization Or within a department of a government. There's tools out there that are built to make your job easier Because it's Drupal con and we're we're on the dev ops track and we're talking to what I would imagine is primarily developers and testers We've chosen today to focus on that that former set of tools the tools that were Designed and purpose-built to make your life is easier Notwithstanding the fact that there's another set of tools out there in the world are Engineered around conformance and compliance and when Laura talks through some of the tools in a few minutes here You'll see that at least one of these tools Strattles the fence it maybe was more originally geared toward making the life of engineering easier But certainly has some reporting and conformance capabilities built in as well So we're looking at three tools today and we picked these tools I'd say primarily because they're the three most common tools that fall into this Making the developers life easier bucket and you can't really test all of the tools in the market and present them all One tool we're testing is lighthouse. It's a tool from Google Lighthouse tests much more than accessibility it tests other things like performance It's able to test whether or not your application conforms with the requirements to be a progressive web app It's able to test quite a few things One of the advantages of lighthouses is open source, and it's free. We're at an open source conference. That's important to many of us and It's built on top of the accessibility testing portion of lighthouse is built on top of an open source accessibility testing rule set called axe so I think that's a that's the one open source tool that we're looking at today and and Again from Google and they didn't build this rule set themselves. They themselves are leveraging knowledge within the broader accessibility community Second tool that we'll be looking at today is Tenon IO. I already mentioned Carl Groves and a quote from him earlier It's a newer entrant in the commercial market founded by Carl It's not open source. It's not free. There's a there's a pay-per-use with Tenon It has API support. I'm on the newsletter. There seems to be a fairly constantly expanding tool set and coverage of what's able to be tested and Tenon has built into it the ability to crawl applications, so You're not stuck to just being able to test one page at a time. It's got a crawler built in The last tool we'll be talking about today is wave or the wave API This is from the web aim community at I believe the University of Utah and Many people if not most people who have ever done any accessibility testing have used the wave browser extension The wave API really is just an extension of that And allows you to submit requests and automate some of that testing rather than having to test one page at a time In the browser again, it's a commercial tool It's not free to be able to use this tool But it's been supported by the by the web aim community for quite some time So now what we'll do really what we want to walk you through is showing you how the browser extensions for each of these Tools can make testing easier After that we're going to talk more about how the command line interface can speed up that testing job Obviously, you don't want to be testing many pages across many browsers manually each time you want to go perform testing And then we'll we'll dig into how we can integrate some of those command line tools into a continuous integration process Yes, so the browser extensions There is a browser extension for each of the tools that we tested and the great thing about them is That they're all free for each one even though Normally wave API Is pay-per-use and so is 10 and I owe all three browser extensions are free and And They are not there. They are automated accessibility tools even though you actually have to go and test each page One at a time they can help you reveal issues with invisible elements They can help you catch issues. You didn't know where issues On the downside you have to remember to use them and It is a little more time-consuming than Using a command line interface tool, which we'll get to in a little in a little while And you can only test a single URL at a time So with that it is live demo time and I am going to show you Take a look at the the Drupal con Nashville site We chose this site because we all probably have a little bit of familiarity with it already And also it's a good representative site in that it has a lot of different features here on the on the home page And so I'm the first browser extension you can see the the browser extensions up in the top right corner here We've got wave and 10 and I owe I'm going to show you wave to start off with Which I think is a fabulous browser extension It's the I consider it the Cadillac of browser extensions even though It is these these icons sort of make it a very colorful, maybe 90s model Cadillac So you can see that there's a summary panel that opens up on the left And it shows you right away There are errors there are six errors on this page and but the great thing about this browser extension is that Is that it actually annotates the page for you? So you can scroll down and you can actually see where you know at a glance where the errors are located So right here you can see what is this? We've got an h2 tag here with an error beside it if you click on it It says this there's a heading here that contains no content And if you click on more information It tells you it opens up this little information tab in the in the sidebar Tells you why does that matter? Some users especially especially keyboard and screen reader users often navigate by heading elements an empty heading will present no information and may introduce confusion And Another feature of this extension is this little code tab at the bottom here Which if you open it up will show you the annotated html So you can you can actually see the the code in question and we can see that there is just Just a an empty h2 tag there, which Which was probably added by a content editor? At some point and just didn't get caught they removed the text, but they didn't remove the tag So that is a very easy fix if we keep scrolling down we can see Uh-huh Here are two form fields and We can see that They do not have labels Now why is this an issue? If a form control does not have a properly associated text label that form control may not be presented to screen users and Form labels also provide visible descriptions and larger larger clickable targets for form controls So essentially Someone with a screen reader might not know how to use this field if there's no label here And if we look at the annotated code It looks as though this is a this is actually a Mailchimp form. So this is probably just been cut and pasted Here, but again, this is this is a really easy fix Probably the developer just didn't didn't realize it was there So if we keep scrolling down we see okay, there are no more visible errors, but we we do know that there are six errors so Probably the other errors are hidden behind theme elements but the wave Browser has this handy no styles tab, which allows you to look at the At the at the page without which without styles so that you can actually see all of the The the hidden elements there This This Mailchimp field has a third input element Which looks as though it says real people should not fill in this fill this in and expected things Do not remove this or a risk form bot signups. So clearly this is some sort of a honeypot type Field that prevents bots from using the forum if we scroll back up to the top We can see that what was hidden in the header was that the skip to main content Now these a skip to main content link is added for For a keyboard keyboard users meaning users that don't use a mouse This link helps them skip over the navigation because otherwise they have to tab You know 150 times or however long it takes to get through the navigation to get to the main content Well in this case If we look at the error The skip navigation link exists, but the target for the link does not exist or the link is not keyboard accessible and What has actually happened is you can see that it should be linking to a div of ID content But that there is no Content div on here. So probably what happened is the developer Altered the template for the page and just didn't realize that there was a link in the header that was going to be broken afterwards the skip to search Link is also similarly does not have a div that that it skips you So those are issues that you know if a developer Used a tool like wave. It's just really easy to to identify them in the case of the forum with the missing Labels, you know, it is if it is a designer that is asked for the placeholder Fields because you know off, you know, this is it looks nice to have placeholder text instead of title text But you know if the developer knows well, it's an accessibility issue and they can go back to the designer and say You know, is there a workaround for this that will that will work for accessibility Okay, so I'm just going to briefly show the other Browser extensions Tenon.io we click on tenon.io this one actually, oh I've used up apparently I've used up all of my My test now abilities So I can't I can't show that to you But it looks rather like these screenshots here that you see It's got it's got a little report at the top, but again these are If you were paying for the the API credits you would be able to to use that and have all of the other features as well Tenon actually has quite a nice Reporting like dashboard where you can see sort of your testing history and and all of that sort of thing But you know you need to pay for it so Now I will show you Lighthouse so unlike Tenon and Wave where you have to Install the extensions if you're using Chrome then Lighthouse is baked in To the developer tools, so if you want to use it you can just open up the developer tools and I've got the audits tab open here So normally if it was at the bottom you would see all of these tabs, but you'd open up the audits tab and Click on perform an audit Now you have all these different options to To choose from and they're all helpful, but for the sake of time I'm just going to choose the the accessibility one and Run the audit so and it's so it goes ahead and it tests a test mobile and It tests for desktop and it is going to Show me a report Here in the sidebar in the audits tab again so You can see that it doesn't annotate anything on the page the way that wave does But it does show you the errors so Here elements have discernible names is a is a category And it's going to show you that okay there's some buttons that do not have an accessible name and it shows you the failing elements and So I snip it here We'll show you we'll show you the offending element shows you Lighthouse is showing us a couple of things that wave did not show I frame elements do not have a title And I looked and that's actually the ads And here. Oh, here is the the form elements do not have associated labels and And it's also showing us that there are color contrast errors Which which wave does as well? There is a tab and we've that shows you the color contrast Errors and it shows you that there were also 16 past audits. Yay And 15 not a lot of us. Okay, and additional items to manually check now as As Everett mentioned earlier None of the the browser extensions caught That there there is an accessibility accessibility issue with the menu And I don't know if you want to speak to yeah Yes, certainly, so one of the issues with the menu and if if you sat in the session yesterday where we were talking about JavaScript and accessibility We did a bit of demo and maybe Laura can do a bit of a demo here as well Where for a keyboard only user or a screen reader user even the drop-down menu on the Drupal con Nashville website Really really creates a barrier and the reason for that barrier is it's not it's not really possible for a Keyboard only user to get to the sub menu items So as you tab across the top-level menu items, you certainly can see that the sub menus expand But there's no keyboard control or keyboard mechanism that allows you to navigate down through those sub menu items What's of interest is that none of these three browser extensions and none of these three? Accessibility testing tools flag that for us in the report in the audit. They're not able to catch this The reason is because this is essentially a complex user interface component It's something that doesn't exist in the html specification And it's something that the tools themselves don't know to look for they the tools don't know how to At this time, they don't know how to look at your page and say that looks like a menu I bet you something supposed to drop down from there I bet you that once it drops down the user needs to be able to navigate through the list the tools Just aren't able to make the types of inferences about the user interface That humans are based on the visual affordances that you see when you look at the page And so that amongst some of the things we were talking about earlier are one of the limitations of automated testing You know, not only can't we tell you whether or not the alt text of my cute cat is Applied incorrectly to your cute dog. It also automated testing also can't Infer what type of user interaction your complex user interface component is supposed to allow and Test for that interaction now you can write your own custom tests Not with these tools, but with others So if you were using something like be hot to do automated testing of your application You could certainly write tests to confirm that menus were accessible Now the reality is at the point in time that you're doing that you've almost certainly ensured your menus were accessible So it's less about diagnostics at that point and more about regression I'm going to show you an example of How to use the command line tools? and The one that we've chosen to to demo is lighthouse and we thought that We would choose it because Practically speaking we think that you know for for local testing. It's probably You're probably more likely to use it because a is free and B you can use it offline to use the light house command line tool you you install a Note extension and So and then to run it You just do lighthouse and then the URL so I'm just going to show you what it looks like to run it without any sort of Modification so Okay, and I've just shown you Okay, so I've created as a developer. I've created this beautiful This beautiful form to demonstrate the command line tools and So I'm just going to show you what it looks like There so I'm just going to do lighthouse and then my local URL and To show it to show you what happens when Just use a lighthouse without any modification You can see that it actually opens up the URL in a in a browser in in Chrome and it spits out all of this Information about what it tested it doesn't actually spit out any errors here by default It tells you that you can view the report if you want to that it has generated a report so The way that I found it helpful to to use it is that is that I used gulp To To run The command here, so I did lighthouse and then I just added a couple of extra Parameters I said output as json Now if you have put us as json you just get a big blob of json I'm just going to show you that so that you can see it So what you can do and what I did is as I let it save the report Here's the big blob of json. I let it save the report and then I I parsed it with Just in as a gulp task So and what that allows me to do You know you can you can sort of use these tools any way that you want to but what I decided would be helpful was that I I run the gulp command with as a pre-commit With a pre-commit hook so that you know when I'm altering my form When I commit it if there are any errors, it's going to show them to me so for example I Go back to my form here if I If my designer said you know I I like your form But I would like it more if you took out the label and added Placeholder attributes instead so I'm gonna say okay, of course. I will do that for you And so I'm gonna go to you get Okay, so here this is my little module here with my form get commit changing Title text do placeholder text and it's going to I'm using a tool called get guppy that that Allows me to call the the gulp command from pre-commit hook opens it up. Oh my mentor Need to do clear the cache. Okay, and it's gonna show me look I'm not gonna commit this for you because there's an accessibility issue And so here I've just if I'll just show you my my gulp now. Basically. I've just taken In the JSON report there's report categories and I've just Determined that The two is the category that corresponds to accessibility Audit and you'll recall that it has five different categories. So I'm just saying take the accessibility category and If there are items in the result details, then I know that there are errors so So it's just checks for that and if there are any It's gonna spit out the name help text description now again. You can you can parse the JSON any way you like But this is just an example of what you can do And then you know, I can say oh my goodness. Well, that has caused an accessibility issue so I'm gonna put the title back in and Then once I do that The arrow will be on and it will let me commit again So so certainly one of the advantages of lighthouse over the other tools The first is that it's free and the second is that you can easily use it offline with Ten and with wave The tests are even when running from the command line the tests are executed against An API in the cloud and so it's a little bit more restrictive about how the tests can be run That's right. So now We're gonna talk to we've just so that basically demonstrates that Command line interfaces alert you to the issues instantly They don't forget to test when you're in the inter rush They you you don't have to remember because it will remember for you to do the testing It'll help you catch the issues before they go live The cons are not all earth are free only lighthouse is free You can actually you can do this with both tenon and wave but you In order to you basically like you can't do it offline. So you have to use another tool Called and grok, which is something that Carl from tenon told me about Which it just allows you to create a tunnel between your local instance and the internet so that you can you can point to your local instance and And tenon or wave can analyze Analyze it for you There is some configuration required as we've seen And another con is that it can only test a single you are all L at a time And and that must be set manually So one of the things that we recommend Obviously if you're working on an application with thousands of pages of content You don't from the command line want to test all thousand pages at the same time and and honestly the change that you've made locally might not It might not touch all thousand pages. It probably doesn't We also Considered maybe some sort of magic that you put in the affected URL in your get commit message And it pulls the tool pulls it out and looks at only that URL That seemed quite fragile and once again, we're relying on developers who are I'm certain best intentioned But when push comes to shove in the last couple of days of the sprint come Sometimes testing slips the mind of it of even the best intention developers What we would recommend is that typically if you take a look at one of your application is you can break it down Whether it's five or ten or twenty Really clear key archetypal user interfaces What we would recommend is that you put together that list of five or ten or twenty Example URLs maybe the same URLs that you'd be using in your be hot tests when you're testing the functionality of the application and That you execute the tools against those now from a command line interface standpoint having that done on a git commit maybe causes too much testing to happen and So what we'll show you next is an approach to Moving this integration from the command line Maybe you still ask your developers to manually execute the test on the command line against pages They know they've changed But maybe we do the broader level of a smoke test against that top 20 URLs or key archetypes of the application during the integration process on a build server Yeah, and so one way that you can you can do that is to use Travis So we set up Basically within Travis set up Set it up to do essentially the same thing that I showed you with the command line and It's only testing one URL But but you can see that in the build you can get it to test the URL and And If there are errors you can have it exit and it will email you to let you know You can optionally have it fail the Travis build so that it won't push to dev and Optionally you could also potentially integrate it if you only wanted to to test when you're pushing to production You could use an aquia hook To only have it triggered when you're about to push to production Yeah, so I think there's a number of layers and levels of testing here and and again It's gonna depend on you and your team and the tools you're using what makes the most sense it Are you on a team where accessibility is a mandate and where? It makes sense for you to test every time you push code to dev And if that's the case you probably want to identify those five ten or twenty key Archetypal pages of the application and test and have the Travis build fail if there's accessibility problems What you might also want to do and Laura mentioned aquia cloud hooks is you might also say hey our team pushes to production Once every two weeks or once a month we push changes to production Maybe you want to run the tests again through an aquia cloud hook on the push to production to confirm That things are working correctly on production We mentioned earlier that tenon has a crawler capability now obviously depending on the size of your Application the number of URLs there are to test and the fact that tenon API calls certainly have a cost to them You might want to do this on a on a less periodic basis But you could certainly whether it's once every couple of weeks or once a quarter have the tenon API Execute or crawl against your production website One of the challenges that you'll have if you only test those ten or twenty archetypal pages is you're likely going to catch 95 98% of problems that are introduced through software development What you're not going to be able to catch if you're only testing a sample of the pages are the types of barriers that are introduced through Content so using a tool and this is where when we were talking about automated versus manual accessibility We said really there's two types of tools one that helps developers more so and one that helps conformance more So tenon has that nice feature where it's able to do the crawling and generator report and it kind of Splits the fence between a tool that's built for developers, but also has capabilities that help with conformance Yeah, and wave is going to have spidering ability this year according to them They're partnering with with another company that will That will allow for the spidering And wave also has a standalone API that you can Purchase it is not it's a It allows you to it can spider as well and it can also test offline Which tenon Can't do so with the tenon you would be triggering a spider that would occur once the Once the code got to production, but with with wave EPI you could actually potentially Trigger it before before it actually went live So you might be sitting there thinking okay This is good. It seems like these automated tools can make my life easier Maybe if I integrate them into my continuous integration pipeline, they can make the accessibility of my application more reliable But as we've been talking we talked about the menu on the Nashville website. We talked about All of the things that these tools can't test for And honestly that may feel a bit disappointing I've been I've been doing this for nearly a decade now and I'd have to say that I'm actually feeling quite Optimistic about what the future might hold for these types of tools. I Mean machine learning Artificial intelligence those are buzz terms they they You stick them in an article and all of a sudden all sorts of people have read your article or they've at least clicked the link I Do think that there's a number of things that Machine learning applied to the domain of web accessibility Testing will be able to do in the coming years To expand the reach not only of what we can identify Through an automated tool, but also what we can remediate through an automated tool so one of the first things that machine learning can help us do is Identify and provide Alternative mediums for images sounds and videos. That's already happening today if you go on to YouTube you upload a video on to YouTube And there's already going to be auto captioning now. Are those auto captions perfect? Absolutely. They are not But they're a whole lot closer to perfect than the no captioning that was previously on the vast majority of videos on YouTube Another quest another item that these tools will become better at doing and I think this is going to take more time is Identifying and conveying Accurate meeting meaning from those types of media. I've given example here Two adults standing with a dog in a nature landscape You know so that that might be something that if you were to look on Facebook Facebook does a decent job at this If you look on Facebook, you look at an image. Somebody's uploaded. You'll see that they've added Some auto-generated description to the images now True there might be two dogs and or sorry two adults a dog And they might be standing in a nature landscape and by the way that on its own is far better than nothing But what it doesn't really do is tell me is this a picture of two people getting engaged Is this a picture of two people having an argument? The tools aren't today as sophisticated as being able to pull out some of that meaning another thing and this really comes back to That example we were talking about the menu on the Drupal con Nash view what Nashville website is Being able to identify and resolve barriers through user interfaces. So if you can see The Drupal con Nashville website and you've been on the web for not even that long You you've probably got a pretty strong hunch That if you tap on or hover over those top-level menu items something that's going to drop down There's something called a visual affordance being used that Helps you to understand how the application behaves and therefore how you ought to be able to interact with the application This is absolutely a domain where machine learning Applied correctly will be able to build up An algorithm that would be able to analyze how web applications appear Understand the visual affordances and therefore identify where there may be barriers if we had a machine learning tool that was able to identify menus on a website then Automate the testing of those menus to ensure that they were accessible to persons with disabilities We'd be much further ahead tools like wave and lighthouse and tenon would be able to in that browser extension Show you that there's a drop-down menu on this page, but that the sub menu items are not navigable by keyboard Taking it one step further those tools in the future might be able to either recommend the resolution to the problem or Perhaps even using a client side include they may be able to Automatically remediate the problem and that might sound like wishful thinking, but if you ask around you'll find out that I am not typically Categorized as an optimist That was a bit too much of a laugh But this is really the future that's possible using some of these new tools and techniques the last item Simplifying content and interfaces and there has been some work done here I believe in the natural language processing Communities and I'm sure there's more work that needs to be done But being able to take all of that rich content and information that your authoring teams are creating taking all of those form fields and descriptions if if nothing else being able to automatically assess The level of the language and the complexity of the language, but also being able to recommend synthesized summarized or Simplified wording to ensure that the meaning that you're trying to convey is easily understood by the broadest set of of users So these are all things that I believe in the next three to five years We will see tools like lighthouse tenon wave and others introduce to be able to identify and Automatically remediate a broader set of accessibility barriers We have about five minutes left Are there any questions? Hey, yeah, so first of all this talk was really really useful. Thank you so What about the the sub menus? Are you guys aware of a way of doing drop-down sub menus so that they are accessible or Would a better answer be just find another way of doing the UI? Yeah, it's a great question about the sub menus in particular One example that I point to quite often is mass gov They've taken they've done a lot of research and they've put a lot of time into The accessibility of the site and I think they've done a good job with this with the drop-down menus So you've talked a little bit of sorry You talked about the testing tools and some of the AI and she learned that so are there things coming or people are working on about kind of bypassing those barriers from a user perspective instead of When there's already problems on the website using that AI machine learning to kind of make that information more accessible From like from a client site. I want to make sure I understand the question. Can you try rephrasing? Yeah? So I guess like we got screen readers. We've got those sorts of things are those improving to the point to where Accessibility becomes easier on the only developer standpoint That's a good question. It's not something that I have read about or heard about Happening, but it's certainly something Certainly something worth considering more. I think your question is how is machine learning and AI being applied to the assistive technology itself? To create more assistive or intuitive interfaces for persons with disabilities, correct? Yes? Yeah? No, I think that's a good question. It's not something I've given a great deal of thought to Thank you Thanks. I actually have three questions The first one is have you used or tested Pali? Do you want me to list all three or go one by one? Let's go one by one. Have you guys used the Pali tool or tested that or Pali PA 111? Oh, no, I have not I didn't touch that. Yeah Have you? We're currently using it Both a client-facing dashboard version of it because it is free and open source and then have built an internal Pali CI that we can test with it So I didn't know how it stacked up against these ones. Oh cool All right, I'll answer that one The second one is do you have any opinions about accessibility overlay tools because I know there are a lot of solutions for I See a smile up there But we kind of have our own opinion just getting yours It's it's an so I did smile and it's an interesting question Not withstanding recent events that may have happened with some of the vendors of those tools and defects You know it it really I'm gonna play a fence sitting here, which is on the one hand I think that sometimes those tools may be Misrepresented as to the level of benefit that they can have on On the other hand, I think that they provide some so What you shouldn't do at least in the next two to three years before some of these machine learning Techniques become more mainstream. You shouldn't believe a vendor who says that if you install this JavaScript on your website It will become accessible Okay, however some of those tools That you would install onto your website with some JavaScript Do provide some, you know minimal let's say assistive technology To users who might not otherwise have access to the assistive technology They provide opportunities to perhaps read some of the text aloud Perhaps to zoom or change contrast. They provide some tools You know that persons who don't have access to the latest and greatest and therefore expensive Technologies have so there certainly is something to be said and there's merit for trying to get that type of technology and Information access into as many hands as possible. I think really what it comes down to is Are you being told that the tool solves all of your problems Then you're being told something that's wrong if you're being told that the tool coupled along the side Training for your development design and testing teams a commitment by the teams to always test and build Accessibly a commitment by your organization to monitor its broader conformance to accessibility and To set targets for improving if you do all of those things together in a program and also Add these types of overlay tools onto your application. That's what's going to get you broader reach and better accessibility for all Okay, thank you for that confirmation not to make myself a black sheep, but I'm in sales and account management So I get a lot of calls saying why would I go with remediation or Drupal when I can buy this thing that makes My site and they are being told a hundred percent Accessible so it's it's been a challenge. So thank you. And then final quick question is what do you mean by automatically remediate code? So I believe that in the future the story that we've been sometimes told by some of those overlay organizations Might become true Not I don't think we're going to get to a place Anytime soon where there's this magic one line of JavaScript that solves all of the accessibility barriers in an application But I do think that we'll get to a place in the next Let's say three to five years where some of the most common patterns. Let's say the drop-down menu as an example will be able to be identified and Automatically resolved through a tool where the tool might be able to inject JavaScript into the application That is able to solve that accessibility barrier without having a developer go rewrite the code for the application And I'd say that that's always going to be the second best option the best option is always going to be that the application be Designed and developed and tested to be fully accessible, but I do think that for some of the more common user interface components drop-down menus calendars Etc that will probably see tools start to emerge that can both identify and Remediate those problems without developer intervention Great. Thank you so much Hi, there's a great talk. Thank you I'm just curious. There's there's you know a lot of discussion around the development perspective on accessibility testing and I was curious if you're Basically depending on the way a site is built The content authors have to think about it as well So I was wondering if you are aware of any efforts underway to address that from like a publishing on workflow Standpoint basically using any of these tools in that way Certainly, and I wouldn't think that it's these tools in particular being used for that But I I do know that the community more broadly speaking and I and I think that some There may have been a core talk yesterday where this came up and maybe check check in with Mike Gifford You could follow up with him certainly looking at the administrative interface and the authoring experience is a very important Thing I mean there's two like it like everything else. There's there's two ends to this one is proper training and tools for the authors and the second one is taking one of those conformance tools and Maybe doing a regular audit of the site to say, you know, maybe once a month once a quarter Let's identify where the barriers are, but I don't think we can ever avoid Training I mean tools aside whether it's development testing design or content authoring We really can't avoid the training aspect because for the foreseeable future There are going to be many many aspects of accessibility that where the testing cannot be automated Thank you One more question Does anybody else want it? It's already asked when I don't want to be selfish So I ran the wave tool against a site that we just launched and it performed really well Which is great except for the Google Translate widget and and like The client had two really important requirements one was that it be accessible and the other that it have The Google Translate widget, so I mean I could ask you a very like specific hyper technical question about like Oh, what do you do if there's like an undefined anchor, but the better question would be Is there some sort of approach that you can recommend when you're bringing in third-party tools to make sure that they don't break the accessibility on your site Well, I mean the short answer is no you can't make sure they don't but you can Remediate them and I mean this feels hackish, but I've had to do it before I remember years ago I was working with a calendar because they are you know the scourge of accessibility components but I was working with a react calendar component three years ago and I asked my team okay does this component have like a Previous and next month button because I would think it should but there doesn't seem to be one and they're like yeah They're right there. I'm like no they're they're not right there and what I realized is like yes This calendar component and react did have previous and next buttons, but they were so poorly accessible I not only didn't they work for me. I didn't even know they were there and So yes, we chose to move ahead with using that component, but we kind of had to Remediate the accessibility of the component now There's two ways to do that the first one's really easy Which is you open an issue in the github repository for that open source library? You put in a poll request that has the solution and the very benevolent author of the component Incorporates it and issues a new release The second one if you're working with something that's more closed source or maybe a library that's been a bit abandoned I mean outside of forking it and going your own direction if there's really nothing else you can do You need to write some JavaScript that loads after your component that remediates the component itself a Little bit more fragile of a solution But a solution all the same all right. Well, thank you very much everybody for coming up I Right well, I mean the shorter answer is because we just had to limit the number Absolutely like you know would be interested in in and testing it. Yeah Yeah Sure, it is older though, I don't know that is I don't know that that's a bad thing or a good thing It isn't but so is great for that matter. Yeah, the reality is we had to