 All right. Hello, everyone. We're going to get started on the core accessibility core conversation, and we'll do quick introductions here. Since I'm at the mic, I'll go first. My name is Kristen Pohl. I'm from the US, and I am a contributor to Drupal and an agency owner. Yeah, I'm Andrew. I'm one of the core accessibility maintainers. I should come over here to the microphone. I'm one of the core accessibility maintainers since last year, contributing to accessibility issues for a number of years before that. And I'm Théodore Biaudela. I'm a JavaScript maintainer for Drupal, and I work usually on accessibility issues because they have JavaScript problems usually. All right. So why accessibility? So we pulled this quote from the web. The power of the web is in its university, yeah, universality. Access by everyone, regardless of disability, is an essential aspect. So this is a quote from the inventor of the web, and I'll let Andrew talk about it. Yeah, so I really like this word. Barrier fry height is a German word, and literally it means without barriers, barrier free. And I love this because it's an example where the German word in English makes more sense than the English word makes in English. So barrier free. It's a good one to learn. All right. So our goals today are to learn some of the new things that are happening in accessibility, brainstorm some ideas of how we might implement these things, and then also try to figure out how can we get more people to help. Yeah, so I thought we would just mention a few important things since Drupal 8.0 came out. In the past, my colleague, Mike Gifford, has taught extensively blog articles and conference sessions about the things we achieved between Drupal 7.0 and Drupal 8.0, but since Drupal 8.0, we've kept going. And the biggest, biggest win, I think, is that the inline form errors module is now stable. So if you haven't experienced it yet, I recommend that you try it. When you're building websites for clients, I really do recommend that you turn this on every website that you build. It's a big, big win and it helps lots of different groups of people. We're extending the use of some JavaScript features that we put in. Drupal Announce is a little utility which allows you to make a little custom announcement which a screen reader user can benefit from. We're extending the use of this now in Drupal Core and, of course, any Drupal module can also use it, use it sparingly. And we've found out, we've introduced some keyboard accessibility regressions where some things would work with the mouse but would not work with the keyboard. And so we have managed to fix a number of those, but importantly, we've managed to build that into the new JavaScript testing framework as well. So we now have some tests in core which explicitly simulate a key press event. So today, what we're going to carry on with is look at a few of the things that we could do in the future, sketch out a little bit of a road map for where we go next. Now, this one isn't something we could do, this is something we must do. The Web Content Accessibility Guidelines are getting an update. And WebCag 2.1 is a significant expansion to WebCag that brings in, I think, about 20 new success criteria. I've been tracking this since the start of the year. The W3C is still a working draft, but now is a perfect time to mention it because the working draft milestone this September is the one where they finalized the success criteria that we're going to be included. From this point forward, the W3C are refining that so they can move it towards the recommendation stage, which means it's now the time for us to start looking at Drupal and spinning out issues about this and looking at our core themes, 7 and Bartik in particular and saying, is there any part of the design there that we need to tighten up? So I've picked out a few examples of what's coming in WebCag 2.1. One of them is target size. And that refers to things like when you have a mobile phone and you're using it with a finger, you might have some small targets next to each other and you're using a finger and you want to make sure that the thing is big enough that you don't hit the wrong one accidentally. Now we address this as part of the mobile initiative, but now it's been formalized in WebCag and what we need to do really is just confirm how well we're doing. I'm hopeful on target size we may already hit that. One thing that we're not doing very well on is focus styles, in particular for sighted keyboard users who utterly rely on these. We have focus styles, but some of them are strong and some of them are far too subtle. WebCag 2.1 brings in requirements for the strength of contrast on focus styles. And there are some other things as well where WebCag 2.1 will address more kinds of assistive technology. So an example of that is that there are some new success criteria relating to speech control. So WebCag 2.1, that is a must-do and as of last week the core product managers for Drupal agreed that this would be a major initiative. We really need to do that because Drupal is a strong contender for accessibility in the CMS framework area. I dare say we're actually a leader there and we want to maintain that position. This is something that we could do. This one is the W3C ARIA authoring practices. ARIA is a way to enhance HTML with a few things that it's not capable of expressing. In order to, you know, we build, we use HTML and JavaScript to build some interesting interfaces like tabs and accordions and date pickers and mega menus and things like that. And HTML doesn't express those very well. So ARIA lets you build in some additional information which assistive technology can use. An example of that is that in an accordion, some ARIA properties and states can inform a screen reader whether your accordion is open or closed, expanded or collapsed. But they have been implemented in a variety of ways and ARIA is often misunderstood. So W3C is putting together a set of design patterns which illustrates some best practices for including ARIA. Now, this will not be a formal recommendation. I think it's just a W3C note of what you can do. And we don't have to adopt it wholesale. We can look at any of the patterns that are involved there and ask whether it would be a good fit for Drupal. A couple of places I think we should do it are our vertical tabs. We've had those since Drupal six where they began in contrib. And they look like tabs for sure, but they don't behave quite the same as tabs would on a desktop application. So the ability to say switch tabs using arrow keys, et cetera. The ARIA authoring practices sort of describe a pattern and it's kind of an attempt to standardize, reach some standard patterns which can be reused across the web. I think vertical tabs is a good one we could use. It will involve rewriting a lot of our JavaScript and either updating our own JavaScript or maybe throwing it out and looking to see if there's a better piece of JavaScript in a third party library. It even goes as far as providing patterns for things like a tree grid widget. And I'm thinking, well, you know, do we have any of those? And should we, the another way of thinking is, let's not use tree grid widgets. You know, let's get rid of those for usability. So ARIA authoring practices is something I think we can bring into Drupal core when we feel it's ready or when we feel we're ready on incremental minor releases. I'm interested in the media initiative. I went to their thing that they did yesterday, their state of the media initiative summary talk. And the thing that the media initiative offers us is a way to extend our coverage of the authoring tools accessibility guidelines. These are, this is another W3C standard which describes how an authoring tool such as Drupal or WordPress or Microsoft Word or so on can provide features which help you produce content that is accessible. An example of where Drupal already does that is that for any image field, you can configure whether or not the text alternative field is enabled for that particular image field and whether or not it's required. It gives some flexibility to site builders and information architects. But the goal is to enable you to produce accessible content. Well, at the time Drupal 8 came out, we didn't cover much more media than images, but as the media initiative grows and we want to bring in capabilities for video, then we'd also want to start supporting content authors who are dealing with captions for videos and transcripts and see how we can build those in. I asked the media initiative guys how you would do this and whether it needed to be in the API or whether this was something you would configure using fields and I think it'll work. I think what we would probably do is use some sensible defaults in core but probably it will be covered in things like handbook pages, document how you set things up for that. Testing. There are lots of ways that we can do testing. I've put user testing at the top because out of all of the things that Drupal has done for accessibility, conducting tests with real users for accessibility is the biggest gap that we have. For usability, we've conducted a number of formal studies but for accessibility, we haven't done this yet. I personally am coming from a developer background and I don't really quite know how to organize this or arrange the tests, design the tests or even recruit testers. So this is something I would really love to have more input from the whole community and people who can take part in getting this sort of thing running. Automated tests are things that we keep talking about and haven't put very much in place. The functional JavaScript testing framework is great because we can write specific tests for behaviors that we've built. An example is we included one for that keyboard accessibility regression that we mentioned but we would also use it for things like when we're making custom screen reader announcements, we're running tests to confirm that the screen reader announcement has been made. We could start building in tests like when we have a modal dialogue, pressing escape, we expect the dialogue to go away and we can start building that into functional tests now. The out-of-the-box experience is underway and this brings in a new theme and accessibility review has already started but we're going incrementally and at the moment the current focus is on getting the colors right and then we'll move to things like the document outline. This was demonstrated in the address note yesterday and it takes a form of a ready-made website which is featuring cooking recipes and the content will be displayed in views using display modes for the content types and so at some stage the accessibility will look at the templates we're using and the way we've configured the view modes where we need to have headings and so on. But as well as being a technical challenge to build the theme, we are also including some specimen content and because the sample website is very image-heavy this means that we need to pay attention to the text alternatives that we include as part of the specimen content and writing good out text can be a fine art sometimes. It's very easy to write a matter-of-fact description of an image but if you want to convey why you have chosen that image it's like you can do an awful lot just by tweaking the words. Think of when you're trying to compose a tweet and you go oh yeah I've made a really really clever sound in tweet. You could actually treat alt text in that way sometimes and it gets across the the kind of journalistic intent of why that image is there. So what I'm saying is there's a good opportunity for copywriters to get involved it's a creative thing. I'm gonna pass over to Theodore now who'd like to talk to you about how what it's like when we go about implementing this stuff. Yeah so it's going to be pretty short because it's not the main topic but it's still important. So the what you need to know about the core process is that it's pretty long to get anything done and especially for accessibility because of a few things. So the process to get an accessibility fix into core is similar to a UX fix and UX issues usually have lots of comments because everyone has an opinion and it's a bit similar with accessibility because you know maybe it works for this type of users but not this one and there can be a lot of discussion so it can take some time. The core committers are not very specialized in accessibility so sometimes it's hard to make them understand that the fix is good and that it needs to be committed. I mean not always but it can happen especially when JavaScript is involved in that which is often. As Andrew said we like people to well we like people in the queue to review tests and make sure that the fix is good and automated testing is lacking as well because we just started with the JavaScript testing a few months ago. So it can be a lot of work to get anything done unfortunately but there are difficulty level to each issue accessibility issue that you want to well fix. So the easy topic are the ones that don't require JavaScript usually because it means you don't need to test it really much it's just markup or template modifications so that's fairly easy for the committer to approve and get that in. The higher walls because it's markup as well and the styling so I mean it can be hard to get you know everyone to agree on the right color but it's not technically hard to do. So an easy topic doesn't mean it's a fast fix but it means it's technically easy to get involved and try to contribute to Drupal core. We have difficult topics which are usually when anything JavaScript is involved so the widget the drop buttons models views UI that kind of things it can be really tricky because we have lots of weird and you say that's inconsistent use of some widgets. The screen reader feedback it's a hard topic so we made an API for that for Contrib to help Contrib integrate those kind of fix into their code and one thing that I did not really expect is that we had a user who was made sick by the tour module because the tour module sometimes scrolls a page in your place and this scrolling made him you know kind of sick of motion sickness so there are you know some unexpected things that you need to fix sometimes and lastly we have the very very hard topics which is always keyboard navigation. We have some codes in core to help manage the keyboard focus so you can't focus on all the links on the page but restrict the focus to some links for example in models and as always documentation it's hard because you need to understand the topic being able to write in a way that makes sense and is accessible I mean for people to understand what you mean and we lack documentation especially with the whole framework JavaScript framework and decoupled discussion that we started yesterday we need good documentation so that people don't break the accessibility work we've done in Drupal core when they implement their own you know rendering frontend basically. All right so we have a lot of fun things we can do and different levels of help that we need so who can help? So I've broken this out into different types of involvement that you can have the the actual organization whose site you're building whether it's for your your company or for someone else's some of the reasons why they might want to be involved could be you know increasing their diversity of their user base or just increasing their user base and if they're trying to get more people on the site get more leads conversion sales that kind of thing then they can't get more of that supporting people with disabilities within their company itself so if they have like an intranet or something like that then that can be useful for them they often need to comply with the law so this will be more relevant you know with government higher education but it is increasingly becoming more relevant to all sites that are on the web and ideally if you're doing it up front you're going to save time and money rather than trying to add it on later so some ways they can help you know basically become involved would be basically learning about accessibility in general trying to find the right expertise which can be difficult but you know hopefully in outside of Drupal there are organizations can help as well and finding developers that are at least familiar with it and or willing to learn about it if they haven't already and an important one is allowing contributions so if if they have developers that are working on a project and they are able to fix something to do with accessibility allowing that contribution to go back to open source obviously they can you know provide their own time and money maybe they have people who could help test or they can you know provide sponsorship for events such as this I'm an agency owner and one of the ways that we you know hope to you know some of the reasons why we like to be involved in this is in track attracting good employees good clients and also it makes it easier for us to sell Drupal if Drupal already has these things baked in so that if we're trying to do a higher ed project or a government project or someone that cares about accessibility which should be everybody but you know being able to sell it better is a good reason you can build things faster if it's already baked in which is good and you know giving back to open source and fostering inclusivity so how can an agency help make this happen you know it's it's hard but you can budget time for your employees to have some time to contribute back you have to factor that in obviously to your rates but it is an important thing another thing is when you're writing up your contract you should make sure in the contract you're allowed to give back any patches to open source make sure it's actually in your contract when you're having that with a client and then most often they'll be fine with it occasionally they might fight back but that is a good thing so build it into the budget so up front you can just say that we're going to try to make sure that you know this the site is accessible and that that you're going to do audits and and make sure just make sure that's part of your overall plan you can advertise it as a service sponsor events and run trainings as a community there's good reasons why we might want to do this as an individual contributor you we develop our own skills we become you know more marketable ourselves it helps us you know be more empathetic with our our community and it helps Drupal stay relevant you know if we're always trying to make things better with Drupal then it's going to be more relevant it's going to be easier to have you know it's sustained and stay have a job it's just good and then there's always the the warm fuzzies so obviously you're here so I'm kind of preaching to the choir but you know educating yourself learning more coming to these events you know there are lots of chats there's a lot of resources at the end of these slides that will be available so you can join the chats there's sprints on Friday you can actually go outside of Drupal and learn more there as well and if possible try to find a mentor and you might be able to find that at one of these events or in the chats hopefully that I know there are some here there's you know core leadership as well and this is a core conversation so why should core leadership be uh should care about this we want to obviously grow involvement within the community and we can't do everything so we need to try to find people who are willing to help increasing diversity in terms of contribution is good just in general we get different opinions and it makes the software better so you know obviously as leadership we want to make sure that the project succeeds and you know as Andrew was saying you know we're a leader in this space and we would like Drupal to be a benchmark so that other projects also use it as a standard and there's the warm fuzzies so some things that leaders can do to help uh you know get more involvement find a mentee if you can try to reach out to people who are willing to help uh write documentation which we are sorely lacking and you know articles whatever give talks webinars whatever just try to get the word out there because a lot of people don't really know about this space and um maybe they've heard the word but they don't really know what it means so just trying to educate people is good organize events tags issues in the issue queue and help out in chat or IRC so we've actually included a large number of resources and just a shout out to to Carrie Fisher who put together most of this list and we've got general resources Drupal resources these slides will be available I'm not going to go over all the resources whoops gotta pull it this out of place uh and just to quickly we'll say some of the other things that are happening this week at Drupal con we've got um some things that happened yesterday we did have an accessibility buff yesterday which was really nice and an out of the box um buff uh today there's there are two today and tomorrow we've got two buffs on redesigning the admin UI and the times are at 12 today and one o'clock tomorrow and then we've got uh accessibility session at 2 15 today and then we have there's two tomorrow um one at noon and one at 135 so and then sprints on Friday so if you're around Friday please come to the sprints there will be a table for accessibility and just join you don't have to code you don't have to know anything about code you're welcome to come and participate help with documentation with testing uh moral support whatever so just learning more is good so at that we'll open it up uh yeah so um this is a core conversation I think we've uh still about half an hour available for questions and ideas uh some of the things we just skipped over very very briefly we could go into more detail with um I'm just gonna leave that up there uh the sessions that are on later later uh the beyond accessibility inclusion I believe that's in the being human track and uh I'm looking forward to that I think it's uh talk about building culture of accessibility into uh your organization or our community um Everett uh a former core maintainer for JavaScript is going to be doing more detail on uh um accessibility in JavaScript sorry he's a former accessibility maintainer but he's going to be talking about the JavaScript and then Barris is going to be talking about um you know how you pitch accessibility to your clients and make them understand the value of it so we have some questions or one okay I can ask multiple questions if needed uh yes from the site building side or rather using Drupal as a tool to build accessible websites I frequently run into the question of how do we once we've started building these sites validate the accessibility of the site once it's underway or worse completed and you know it's hard to back into that right but is there a way or okay I have two questions one is what is a way that could that we could popularize to help ensure that more Drupal sites are built in a consistent consistently accessible way and two how do we beyond following guidelines validate that they truly are accessible how does one do acceptance testing for accessibility really okay so the we we talked about this in the boff session yesterday that one of the big things that we uh we would like to improve is our documentation especially around what we would recommend for themas and site builders and module developers you know you could even break it down that way beyond that um so that would be the guidelines part that we could put into place but the second part of your question is more interesting how beyond simply following the guidelines how do you validate that you're you're doing well um the important thing with accessibility is it happens right at the start this isn't something you do at the end for best success you know you need to look at this at the design stage when you're looking at the graphic design the the style guides and choices of colors to designing your your site architecture from a you know like the wireframes and information architecture point of view all of these things can have a knock on effect down the line because you know they'll dictate your choice of where you put headings and how you lay out your page templates and so on and so you can start with those but the important thing is start early and then keep monitoring at kind of each stage through the the build so once you've got your wireframes you'll be looking at right so these are tabs these are accordions do these tabs behave like tabs or they just links to other pages and they happen to look like tabs that'll affect your choices later on when it comes to the front end development and uh then if you are further down the line when you're testing it if those were just a bunch of links that looked like tabs but actually led to other pages well that's a fairly easy thing to confirm that they're behaving well but if you said no no this is tabs are dynamically updating within the page you then you say well that's something we have to test so um a key thing is web developers sometimes like to build really unusual widgets and so if you've got any unusual widget you need to say well that's something we need defined to behave before I guess maybe to put more nuance on my question how do I know if I'm using if I'm let's say I'm testing the accessibility of a component or a page and I'm reading guidelines and or using some testing framework to validate the choices I've made what level of confidence should I have that if I follow these rules and or get a green on a test that it actually is accessible to a user how how well do those guidelines map to real-world usage and is there a gap there still yes there is a gap there still that's kind of what the area author in practices is about because it's um um building a set of patterns that are reflect the common use cases and uh the idea is if more developers follow the standard pattern then the whole web would kind of become more consistent um the the thing about accessibility testing is the way that webcag is structured is that every success criterion in webcag is supposed to be testable but many people make the mistake of assuming that means automated testing uh and that you can just run it through some uh checker and see some red green pass fail type things but the testable in webcag terms doesn't necessarily mean automated testing it really means human review can confirm that it's behaving so the way that webcag is structured you it's um it's if you if you admire good hypertext it's a beautiful document but what that means is some people find it really hard to navigate but um imagine it the webcag itself is a really short document but then it has a large amount of supporting material and you'll find some documents called um understanding webcag success criterion 141 use of color whatever and i'll embarrass myself that would have been the wrong number uh and in there you can find longer explanations which include scenarios uh discussing the kind of um users and technologies that it's aiming to support and we'll also finish off with uh suggesting a list of techniques that you are advised to use because it doesn't say do this it says these are lots of ways that you can address this problem and they're not all appropriate for every single site you'll make choices between um well we could use this method or that method they also have a list of known specific failures which are things if you've done any of the if you if you've encountered any of these situations then you've not managed to address it properly so there are some well-defined failures um but every but there's no clear you have succeeded when you do this they the web the understanding webcag documents that's the more important one that to to learn to read that's the one where you get a feel for what's there and it does suggest approaches for human review on those and if i uh if i may just designate someone to kind of answer so Vincent just here he oversaw the updates of the french government of the french uh accessibility guidelines uh last year or two years ago so maybe you have some opinions or advice on how to do proper uh validation and testing um yes as a matter of fact uh i think automated testing uh can't be achieved perfectly there's always a gap and uh maybe one of the best approaches to to give to have automatic automated testing that tells you if it's 100 percent okay 100 percent false and what is in between at what needs what what requires uh human testing so there's some tools that can do this and there is also a lot of work that has been done um on accessibility by the french government like um maybe a simplified uh referential for testings and i'm just trying to get in touch with uh people that can point to the english version of it because there is some things in english and there was work done also on um uh javascript library uh to have uh examples of uh how to make an accordion or um uh a menu in in the way it's accessible with angular react and some other libraries so i will i'm gathering now uh where are the links and i will show it later with the other um so it can be put online thank you and we have a question here i want to add something automated testing is one issue i want to talk about in ph in ppp you can uh you cannot give the decision to the automated testing program but you have to check thoroughly what is the result of the testing and i have to decide myself and some people want to give the responsibility for testing to the automatic tool but i think this is the wrong way to do it yeah yeah thank you for that that's the um accessibility is about people and human testing is always the final uh um validation of that culturally if you're doing this say in a in an organization such as a an enterprise organization or a digital agency it can be a good idea to have an accessibility lead whose role is to provide advice but also to keep mentioning the word accessibility at every stage of the process our next question uh yeah thanks um first of all i'm really happy that the out of the box initiative actually not only makes it a really nice thing to look at but they can also actually have a gold standard on how would an accessible theme look like with the alt text and stuff like that but within the core development i'm also wondering whose responsibility is it to make sure we have accessibility and usability because in usability for example we have issues that are nine years old and are not getting picked up um how are we making sure that this and we don't even have proper guidelines we don't have any guidelines on what should be a page a tab or a secondary tab so how we do we bring these guidelines not only for usability but for accessibility within Drupal instead of saying there's a tool here there's a website there go check here how do we bring this as something that we do when we develop core but also when we develop contract modules and how do we make sure that this is a responsibility that we make up when we provide the software for others to build on how do we make sure that these things are all getting fixed and not waiting for 10 years yeah um i can't speak for the wider aspects of usability but um to say that there are issues that have been open for nine years but we can also look at how many issues have been uh fixed and changed in the process of nine years um you know like the the usability team have gone through several rounds of improvements with real user testing and and on accessibility we went from being a project where it was like developers were saying like what what is out text uh to going through successive versions of Drupal of five six seven eight where we've gotten better each time the low hanging fruit is now all gone and now we're on to the polish and the harder topics how do we um make it how do we establish a process or guidelines for core developers and likewise for contrib we we talked about this yesterday in the boff it is a gap and i think we do need to have more we need more handbook pages for site builders and uh developers and we need more um more of that uh edgy i i guess i guess we need to yeah more documentation will fix it the one i'm not sure about is the business of how do we decide what is a tab and what is a page and and so on i'm not really quite clear on that i suspect as we go towards more richer javascript develop javascript driven front ends um the answer will be that we want fewer page reloads as as in you know because it's like go to a next page go to a next page go through a confirmation page towards more immediate richer experiences with you know modal dialogues and little immediate notifications and so on so i think in the future tab would be the answer rather than page but it depends what we design and just on the overall subject of enforcing accessibility uh in core uh we we are very bad at making process for non automated uh topics i guess so coding standards we're really good at that but anything else that we can't automatically test we're very bad the mentoring aspect is one way we could address this um we would like to have more contributors working on accessibility issues but uh one barrier we face is that there is so much manual testing involved and there is a big learning curve for using assistive technologies especially if you're someone who is uh not requiring them as part of your everyday life so a developer could get involved someone could get involved in core issues but if you don't know what the kind of expected keyboard uh experience is like or you don't know what makes a good keyboard experience different from a tedious one um if you have never used a screen reader before there's a lot of things you need to learn before you can start uh contributing effectively and i think maybe we need to just get more screen reader training for contributors i was just saying that maybe at at camps we could actually have training sessions for developers on not only how did we break things but how does these technologies work yeah and actually something this is something we need to learn yeah to see how it works when it works right yeah it takes time to uh but yeah having demonstrations and letting people try it for themselves is a good thing last year um one of the core committers um i think it was um chris syliphen who had just become one of the release managers he was he was a new core committer he like came up to me afterward after a similar session and said have you got some time that you can show me how to use a screen reader and uh yeah we'll sit down we'll get the headphones out we'll have a play we jump through headings and and he had his first go at a screen reader and at the time i didn't realize he was a new core committer but afterwards i'm like that's good i've got buy-in from the top yeah i just wanted to say one thing about the documentation side um and it's a little bit telling so we're putting together the slides and there was a there's a page on Drupal.org for the d7 list of like contributed modules for accessibility and we're like well where's the d8 one and there wasn't one and we're all looking at each other well who do we how do we get that added like well yeah and so we're all looking at each other like well we're not sure the best way to get added so we we're like okay we'll just make a quick blog post and put them there and then once we figure out how to add it then we'll go and add it that's kind of sad that we're just kind of well we need some documentation and we don't even know like where's the right start so we need you know we definitely need so to educate ourselves of like what is the right process how can we have like our own place and and be effective at adding the documentation that we need yeah then we've got 10 more minutes available to us so should we move to the next questioner hi thanks for having this core conversation um I just had about to David's point I don't think that um wherever really I'm a developer and we're never really done with accessibility like we will we have our designers who will do their style guide and those past tests that they do but then when we start developing um we try to follow guidelines and then we'll get feedback that since such doesn't work so in applying this to core with aria roles or redoing the form system I think we'll just have to can we can try to do as much automated testing as possible but we're also going to have to do a lot of manual testing and as new patches keep going in we'll have to test again and like as something gets refactored we'll have to test again because that's the only real way to know that everything is still working and I don't have an idea of how to get that manual testing in as part of the process but I know from working on our client sites that that's really the only way you get to a place where your stuff is working yeah actually just I had a question for you is when you implement the website which what kind of resources are the most helpful to make sure it's accessible for you like is it actual testing from a professional or is it documentation or what kind of things um so the agency I work for we have a set of documentation internally that we use that are our standards specifically for a client that I was working on they were required by the government to follow the standards so they had a professional accessibility expert go through a key pages and give us feedback that we then had to fix but for me I will read the um the actual recommendations and notes um it is hard to follow at times because it can be verbose but I think the more detail the better and then I think it would be helpful for Drupal specifically to have examples because we do have a lot of widgets and interfaces that then when we're or sorry implementing um client sites or new modules that we could just pull from that instead of continuing to reinvent the wheel something I want to highlight in what you said was that uh you mentioned that your agency actually has a set of internal resources and you're doing a client project where they brought in an accessibility tester uh you know did digital agencies are notoriously um bad at web accessibility many of them have never really made it part of their process so in a way you're kind of already in the top tier um but you mentioned that you had some testing from an external tester and the interesting thing is when was that done was it after you'd done all your your website build or did they come in and get a chance to look at prototypes along the way which would be the point I'm making is that this would be earlier feedback would mean that there's less for you to unpick and rebuild that I can't speak to because I came into the project after it was built and I'm assuming that they even though it would have been nice to have multiple rounds that they didn't have that testing done until after the site had launched and then got feedback that they needed to make changes and that's where we were we came in to make those changes and fixes yeah um where that relates to core as well I mentioned that lots of people core contributors may have faced a big learning curve with using assistive technologies or testing for different interaction styles but it's not just that people who are implementing patches that face that if someone wants to come along and review a patch they need to have the same level of knowledge I think it's like maybe having at a sprint that could be something that people can there could be a table and people could learn about this you know actually learn some of the tools in order to do the screen readers or other assistive technology and we could do that this week we could do that on Friday we could uh let's have a little um boff during sprint time where people can experience some assistive technologies for the first time and take that first step up that ladder I'll actually be at the sprints on Friday and have um local setups so I'd be happy to show people if you're going to come to sprints on Friday which everyone should let's connect at the end then thank you hi uh on one of the slides previously you mentioned that needs tests can kill an issue progress yeah and uh at dev days earlier this year I hadn't learned how to use a screen reader up until then and it had really opened the process for me to sure I think you need to be very close to the mic at dev days earlier this year I hadn't learned how to use a screen reader until the conference like was basically almost over and it enabled me to work through more issues that felt stalled at needs review but I am interested in trying to figure out what other things we can do to help people like move issues forward and how we can make needs review not like kill an issue or kill a progress on an issue or so they don't last for like 10 years or something like this yeah some of this doesn't relate specifically to accessibility but it's a general problem in in core development is that um and an issue can be filed and it has its issues summary then it gets you know 50 100 comments and in order to move that issue along you've got to understand what the current state is and the that involves reading the whole thing and the the introduction the introductory summary has not been updated for a while so that's a thing that we share with every other aspect of core development but for testing in particular or what needs to be implemented I've started to as a maintainer I'm trying to find new contributors and encourage them to get involved and I realize that means having to teach people what it is and rather than try and fix accessibility issues myself I want to present them in a way that other people can learn by fixing them and I think the effort we need to make more of an effort to have better issues summaries describing accessible behavior and or linking to an issue where we've fixed it somewhere else first or you know saying well we've already got one place where we do this in core but we need to extend coverage so refer to this and copy the pattern there actually put in more detailed instructions in the proposed resolution you know what are you actually going to do but it does whoever's updating those issues needs more needs an awareness of what's going on and if you're coming to this prince on Friday I'll be running the JavaScript table and first time contributor stuff so awesome thanks hi how much time do I have we have a few minutes left so maybe we'll say this will be the last question not a question it's sharing some resources my name is holing poon and I work for the new york public library I am a back-end developer we have been learning the hard way of how to make our website more accessible so we have a design toolkit my pl design toolkit I just put it on the phone it is available on github it is our web style guide with accessibility notes embedded in each one of the issues that we have and we've been trying to follow it this is like a new guideline to our web standards it follows closely of web tag it's like a non exhaustive subset of web tag of things that we want to address first because this it's so deep and there's so many things that we need to address we also do user testing right now every other month we have a library dedicated to braille and talking books so we have visually impaired users that would volunteer to do testing for us so it's not just us checking manually with the screen readers basically joss on windows and voiceover on mac we also do like real user testing and get feedback from them so for what it's worth there's somebody out there doing this but we're still pretty short staff but once accessibility comes in it's just suddenly that there's not enough time and not enough resources to do everything so we're doing what we can thank you very much we should definitely follow up on this afterwards and one of the great things about issue queues now the way that core development happens is a lot of opportunity for organizations to take credit on fixing issues and it'd be great to have a new york public library on our list of organizations i had a super short question you talked about the gap between what automated accessibility testing can catch and what problems they they can catch and what a real person can catch as far as accessibility problems what percentage do you think that the the automated tests are capable of catching compared to the a real person it's growing in coverage it used to be the accessibility tools were like checking the html source but now the emphasis is on checking the DOM after it's been processed and you've got all the css and the javascript in place so you might well have a form label in the html but then your css has made it invisible in a way that you know the display non-gun we call it you know you you just shot down a really useful feature um testing tools that go against the DOM are getting a wider range of um tests you know the number of tests are growing because what they're doing is um you see this in the the tenon and the um ax core um libraries where you know you can configure which tests you want to run and some of them are getting towards not not just say html validation and stuff but are actually now looking for common patterns or anti patterns in what they're testing um and and you may well have one of those we where you say ah that's going to throw up a red herring but we'll say right that's fine we won't run that test on this scenario so they are getting more coverage and they are getting um they're getting richer ah how much coverage as a percent and does this let us off human review it doesn't let us off human review but um that yes there is there is a thing there is a an idea which said let's get as much of the automated testing involved because once it's in place we just let it go yeah yeah but um but what kind of scale are we talking about is it like it'll catch like five percent or is it like 90 percent of the problems i i think i know what harry's talking about is this the stuff that the the uk gds did yeah the uk gds has done uh sort of comparisons of of deliberate what what is caught on deliberately bad pages um years ago the w3c had a uh it's probably still there had a before and after it's the same website but coded badly and coded well and uh as as a learning guide inspect this be interesting to run the tests against that yeah the the gds did write a blog about it and i think it might be might have been about six months ago yeah we can post that link yeah we'll find that and speaking about posting uh that's not just uh tweeted the link to the french resources about accessibility so you can check the hashtag so for the video since you were off mic it was 30 to 40 percent of automated test yeah harry uh mentioned that the gds had done a test where they ran lots of tools against a page that was deliberately bad and they they estimated that about 30 to 40 percent of the problems that they had deliberately built were caught cool that brings us to the end of this conversation so i want to thank you all for attending and anyone who wants to get involved in uh drupal core accessibility we we're seeing more and more involvement as sprints and the slack channel the the drupal slack team that has an accessibility channel it's really good because we've seen within a few weeks of having that channel we saw more chat and discussion than we saw in like 10 years of using irc so it's it's there is there is a culture growing there is a we have a posse