 Good morning everybody and welcome to this week's episode of the Visual Studio remote office hours. My name is Matt Christensen and today I have a very interesting topic. It's about Visual Studio, of course, and accessibility. What we're doing with accessibility in Visual Studio as well as is there something that you as a developer can get into your apps from an accessibility point of view by the features of Visual Studio and so we're going to talk about all of that stuff and it's a super interesting topic to me. I've been in this space for quite a few years from the web side of things because I used to be a web developer and so this is a topic that is near and dear to my heart and today we have the absolute expert, the one and only Dante who is the basically who owns the whole accessibility story of Visual Studio. Hello Dante. Hey Matt, thanks. Yeah, my name is Dante Gagné. I am the accessibility driver for Dev Dev, so Visual Studio VS Code, all of our developer tools. My job is to kind of be a subject matter expert for accessibility. What does it mean? Work with teams to help them understand the impact of the choices that they make on folks who use assistant technologies and generally just try to be the voice of the customer in this one. This is a space that for a while hasn't been represented very well when it comes to developer tools and things like that and we are working harder to do a better job at it and I'm glad to be a part of it. Thanks. Yep, so for the people that are watching this live, you have a Q&A panel. If you check out in your, if you're watching in the browser or in Microsoft Teams, check out the panel on your, I think it's on your right hand side. You can ask questions and we will try to answer them here as they come in. And if you're watching on YouTube after the fact, then just write your comments below and we will answer them there as well. All right, so Dante, you know, one of the things I hear the most when we're talking about accessibility is that it has to do with making sure that blind people can use my desktop app service website, whatever it might be that I'm building. And I don't know how to test for that beyond sort of there are some tools I can use that I have to like basically Google for every time because I don't remember them. And so it's hard to get started, hard to understand what it means and is it just for blind people? So it's a great question. So we tend to focus quite a bit on visually impaired and blind folks using our applications because they're the ones that have a workflow that's most different from the keyboard mouse screen users and it's it really is a different mentality. I went through screen reader training about a year and a half ago and it took me well over a week just to do something simple like open outlook and send an email. It's a whole different way of approaching your application and understanding how those tools work with that, work with Visual Studio, whatever tools you're doing. That's kind of the first step. Accessibility on the whole though is the Microsoft mission is about empowering people of all ability to do more. I'm sure I'm getting Satya's words wrong there and he does a lot more eloquently than I do, but it's not just about blind folk. It's as well folk who have motion impairments. I've got a great video from a colleague of a quadriplegic database administrator. He has no motion in either arm, either leg and he is a full database administrator. He uses SQL day in and day out. It's kind of amazing the things that he can do with it. It's motion impairment, it's hearing impaired, it's more and more we're also interested in neurodiversity. So folks who have autism or ADHD like I do, it's how little things like distractions in Visual Studio that a typical developer can overlook can be very distracting for folks who identify as having ADHD and the impact that that can have on their productivity. How can we make things like that better? There's a whole spectrum of space there that we're learning. We've got a pretty decent idea of how to deal with screen readers and switches and motion impairment, but learning how folks who are neuro atypical, helping them learn to be more productive is a brand new space. We've been doing a lot of exploration there. So let me answer the question that you actually asked. There are tools such as accessibility insights and of course the garbage truck picks right now to pick up. Accessibility insights is a tool from Microsoft that will scan your running application and look for a lot of the most common issues. Like there's a property in WPF called the automationproperties.name that is what the screen reader will report when focus arrives. Accessibility insights will let you know, hey, you've got a control here that doesn't have a name. So when the screen reader arrives there, the screen reader literally doesn't report anything. All it will say is button. Well, great, I've got a button. I don't know what clicking this is going to do. I've got no information at all. I've got nothing. Accessibility insights will point out to you controls that have that. That's just one example. It covers a lot of the automatically detectable accessibility issues and we use that tool. Developers on my team use that tool for every check-in that has anything to do with UI. We're always using tools like that just to make sure that assistive technologies work and it's again, it's not just for blind folk. The automation property name is used by speech recognition software. So when I say click the apply button, it's looking at that property to find out which button is applied and it scans UI tree to find that. So accessibility insights will let you know if this is set up incorrectly, your speech recognition is going to fail, your screen readers are going to fail and things like that. And these are the simplest issues to fix. In many cases, it's a half a line of code to fix them, but if you don't know look for them, you're going to make those mistakes. In 2000... Can I just ask you a question real quick? Sure, sure. You're talking about accessibility insights. So we use it internally at Microsoft, which means it works in a WPF app, because it works on Visual Studio, which is WPF. But what about WinForms? What about native UI? What about on the web and all this sort of stuff? And is this something that everybody can freely download? How does this work? Great question. So first and foremost, it is available publicly. It was released, I want to say, about a year ago. I want to say it was around March of 2019. There are two versions of it. One of them for desktop applications, one of them for web applications. And the web one is both... If I'm not mistaken, it's a standalone version, as well as a plug-in for Chrome, so that once you're launching it in your browser, it will help you detect those same types of issues. The assistive technologies, screen readers and things like that use either MSAA, which I don't remember what it stands for, or UI automation, UIA. And these are standardized APIs that screen readers and other assistive technologies use to interact with the application. Those are WPF, Windows Forms, even older Win32 applications, all of those implement either MSAA or UIA, in which case they are conforming to the APIs that allow assistive technologies to work with them. Accessibility Insights uses those same APIs to read the application. So it's not going into the WPF to say, do you have the correct WPF property set. It's looking at, after all of your code has run, is the output conform with these APIs. So it can apply to Windows Forms, it can apply to WPF, it can apply equally to Universal Windows, .NET Framework, .NET Core. Any UI technology that works on Windows, as long as the underlying platform provides either an MSAA or UIA implementation, the tools will be able to work with it. And all of the Microsoft technologies, again, Windows Forms and WPF, UWP and so on, they all provide those APIs. So the very long-winded answer, which if you have noticed I'm good at, although all of them do work with Accessibility Insights except for the web. The web has a different way that it exposes those automation properties. That's why there's two flavors of Accessibility Insights. There's the one that works for web technologies and the one that works for desktop. So you've got to pick what you're working with. And if you're doing, I don't remember the term for, but a web embedded app, so you basically got a browser embedded in your app, that's, I believe you want to use the web version because it's still going, the assistive technologies are going to be interacting with it as though it's a web page, even though it's a standalone desktop app that's not in a browser, if that makes sense. Yeah, okay. Cool. So yeah, so there's a bunch of tools out there and Microsoft has one as well, the Accessibility Insights, which is really cool. And it's very, very helpful. I've used it in the browser before. You mentioned it works in Chrome. And because the new Edge, Microsoft Edge browser is also based on the Chromium thing, it also works in Edge, which is nice. Yep, absolutely. So, you know, I'm really happy that there are these tools that can automate a bunch of this stuff. So that, you know, I would say that a minimum is, I think it makes sense that we as developers, we try to make sure if we're allowed the time, which hopefully we can advocate for, right, that we at least do a Passover on the automated testing by these tools, just to catch sort of the low hanging fruit, you know, is your color contrast, right? Because, and this is something that people don't think about much, I think, color contrast, like if you're colorblind, or you don't have to be like completely blind, let's say, to have like, to be visually impaired in some way, you know, small fonts is problematic if you're, so if you're like me, I like, I have a relatively small like 13 and a half inch monitor in my laptop here. And I like keeping like my DPI such that I can see a lot of things on the screen, but you know, the consequences that my font size gets small. And so I'm right on the border of being like, Hey, if you're making on your website, or in your app, you're making your font sizes smaller than standard, then I might not be able to see it. And I'm like, okay, I'm wearing glasses, but I see just fine with the glasses, right? So, so when we're talking about accessibility, it is not necessarily about someone being disabled in some way, right? It's all these everyday things. And also the temporary you know, disability or whatever, you break your arm, now you can't use your mouse, but maybe you can use a dial. Well, how does that then plug in? Like, what are the different hardware? I don't know. There's I think there's so many aspects to it. I'm just saying, oh, it's for the very few percent of people that are, let's say, you know, can't see or have other severe disabilities. That's simply not true, right? It's almost for the majority of people. Like, do we have an idea of like, how many, how many people like percentage wise of the world population or something like that has some sort of condition or whatever where we have to think about accessibility for them to include them? You know, it's ironic he asked me that because I'm putting on a brown bag later on, talk about accessibility. So I was kind of trying to answer that specific question. According to the World Health Organization, one in seven people today worldwide have some form of disability. And that is disclosed disability. So that's not even talking about folks who show up on the autistic spectrum or identify ADHD or things like bipolar or things along those lines. One in seven. So over one billion people and the World Health Organization, as they're getting better testing, as they're getting more people feeling more comfortable disclosing how their own conditions, they expect that number to double in 10 years. So by 2030, they're saying two out of seven people are going to have some form of disability. And that's what we need to be thinking about. Stack Overflow does an annual survey of developers. So, okay, we're talking worldwide one in seven, but bring that down. I believe in 2019, one and a half percent of all developers identified as blind. Think about that number. That's in the scale of, what is that, three out of 200? That's a much larger number than you think. And when you think about like the user base of Visual Studio, you know, that's hundreds if not thousands of developers when you look at those levels of percentages. It really is kind of, those numbers are growing. And you talk about how you use your monitor. I use extra large cursors in Windows with contrasting. So I can see my cursor a lot better now than I used to. My site has gotten weaker over the years. And I recognize it, and I'm very thankful that Windows has got these tools in there to allow me to make things easier for me to see. I increase the font size in Visual Studio. And so it's not just about people who have a, I'm always hesitant to talk about disabilities because it's amazing what people are able to do with the abilities that they have. So I don't want to say it's limiting in some way, but just those folks who interact with their environment differently, they can be amazingly productive. One anecdote I pulled from the CSUN conference last year was a blind gentleman I was speaking with, and he said, you know, when he's learning, and he has to go through reports for white papers or anything like that, he can search himself to be more effective than other folks just because he has a screen reader that reads at four, five, six times what a normal human speaking speed is. So for you and I to read this, it might take us literally three times as long to go through a page as somebody who's used to working with a screen reader. So they've actually turned the assistive technologies that they rely on into a competitive advantage in certain spaces. And I just find it fascinating what it is. But the average age of developers is getting older. We have new folks coming in every day. The average age of new developers is going down. So people are getting into developments earlier, but the average age of the typical developer, when you look at all of them on the scale, they're going up and their needs are evolving and changing. And they want to continue to be able to use Visual Studio. And one of the single biggest issues is, like you mentioned, the color contrast. We have a guideline of what the color contrast needs to be. And to folks who don't have a hard time distinguishing color or have got reasonably good, corrected vision, you don't notice it. But for folks who have lower vision, that color contrast change, it can make things turn literally invisible for them. They will completely miss text that's there if the color contrast is bad enough. So we need to be mindful of that. And thankfully, we're doing a much better job of that. I see a lot less of those issues. But when we first did the evaluation on VS a couple of years ago, there were color contrast issues. There were keyboard issues. The list goes on for a long way. But we made a lot of improvements since then. Yeah. Wasn't there something about that we were not going to, or we were delaying the release of Visual Studio, something like that until we fixed a bunch of accessibility issues? And you were the PM that was leading that entire effort for the entire Visual Studio product, if I remember right. Can you talk about that a little bit? How did you do that? What tools? We have a question here from Alexander. He's asking, if we're testing Visual Studio using JOS or NVDA, which are two programs that accessible technology. So can you talk about like how did we take Visual Studio, like what state was Visual Studio in? What happened such that we had to make it accessible? And then how did we do it? That's a great question. Yeah. I was part of that journey and it's been very fulfilling for me to see that journey and talk to the people that this is impacted. So long story short, about two and a half, three years ago, the United States started changing some of the rules as far as what it meant to have accessible software. And as you may know, Satya is a big supporter of accessibility. He has always felt that that should be a priority. And even just from a business standpoint to think of ignoring one out of seven developers or customers in general, we're not going to do that. So he went to Scott got three and said, all of our tools, everything is now going to be accessible. And Scott said, we have no idea how accessible we are. Okay, I didn't hear Scott say that to Satya, but the message was we don't know. So a team got spun up and their charter was first figure out how accessible we are or aren't, then build a process in to get our tool get the tools out there for developers to be able to make accessible applications and make our tools accessible. In October of 2017, we started an end to end test of every square inch of Visual Studio. We put together a catalog of everything in VS, every panel, every window, how to get to them. And we handed them off to third party contractors who were trained in how to test for accessibility because we didn't have expertise. We didn't have anybody who knew how to even do this kind of testing. So they went off and they started filing bugs. And over the span of eight months, which is how long it took from October to May of the following year, they filed, I want to say it was 3200 bugs. Hang on, is that on Visual Studio alone? 3200? Visual Studio, not VS Code, not external applications, not tools or anything, just the Visual Studio application. And they would be small ones like the color contrast to this particular window is bad, or they would be large ones like here's the entire panel that is not accessible at all. I won't mention the panel in this case because it has been fixed and I'm very proud of the team that worked on it, but they were working with the developer and the developer was trying to go through a particular code flow and they said, why don't you just use the X panel? And his response was, what panel is that? I don't know what that was. And he knew where the panel was, but he couldn't interact with a single control on that panel. None of it was accessible. So as far as he was concerned, it didn't exist. And this was a somewhat important panel in that workflow. And they have since gone back and it is a great panel to use now. And they didn't just fix the panel, but they engage with the community to say, okay, now we've got the panel doing what we think is right. The community came back and said, no, no, that's not the way to do it. There's a better way to do it. Yes, you're giving us the information that we want, but we learn from them how to not just meet the letter of the law, but make it usable for the people who are trying to use it. I'll give you an example. Imagine if you had a list of customers, and you were going down that list, and it said the customer's name is Mads. The customer's name is Anthony. With a screen reader, you'd be hearing the customer's name is Mads. Oh, that's not the person I want. I hit the down arrow. The customer's name is Anthony. No, that's not. I have to sit through the customer's name is for each and every one of those rows, and it becomes Maddening. So what our community told us was just say Mads, Anthony, Dante, Tina, Kendra, whatever it is. It's just give me the information. And that is a lesson we would not have learned without engaging with the community and finding out how the tool is being used. And that's been a big chunk of it. So yeah, we did this end-to-end test. And in May of 2018, we had over, I want to say it was around 3,200 bucks. And it was the, I think it was the 15.7 release, cameras was 15.3 or 15.7. But Julia made the decision that we were putting a pause on every bit of new functionality and every new feature and anything that wasn't accessibility related, put everything on pause, get this bug countdown. Because as long as these bugs were there, there was a subset of our community, an important subset of our community who could not use Visual Studio, those particular workflows at all. And we just stopped everything. Any team that got all their bugs taken care of, they contributed resources to other teams to get their bugs taken care of. We found issues that required updates in Windows. We found issues that required updates in Windows forms, WPF, at the platform level. We uncovered issues all over the place as part of this. And as a result, Visual Studio is better than it's ever been before in terms of assistive technology compatibility, and I think overall, but that's my feelings. WPF is better, Windows forms is better, our compatibility with that framework is better. All across the line, Visual Studio is a better product for it, Windows is a better product for it, and the platform UI frameworks that people are using to build applications are better for it. We just recently, it was the 15, or 16.5 or 16.6, 16.5 I'm pretty sure, we finally hit our certification that we fixed all of the bugs that we had been tracking all the way from back in May of 2018. We got through that entire backlog. Now, new bugs kept coming in, so it wasn't just here's all the bugs, it took us two years to fix them, new bugs as new features are coming in, we put new features through accessibility testing to make sure that we're not releasing new features that are busted. A great example with that one, the new product, the new project dialogue, the new experience that you get when you first launch Visual Studio, the first preview of that had a bug that the big buttons that say, open project, open phone and repository so on, the automation property names were missing on this. So when you moved over to it, a screen reader said button, and then you hit tab button, you had no idea what any of those buttons did, and this was one of the previews of it, we found out about it, we actually did a service release on a preview, because we thought that that bug was so bad, and we put in new processes to make sure issues like that don't come up, but as new features come in, it increased that backlog, we had new issues that we were finding with 16.5 for the first time, we actually cleared the log, we cleared out all the bugs that we had, and from here, we're going through new regression passes every six months to make sure we didn't mess anything up, and we're always any new features, it is a requirement before release that they go through accessibility testing. May not always be there for previews, but by the time it hits GA, they are supposed to be clean, as from our trusted tester and certified and everything like that, and that's where we are today, and then we're moving beyond that for usability. So I do want to answer Alexander's question, for testing, we focus primarily on three screen readers, narrator, NVDA, and JAWS. JAWS gets the heaviest testing because from our surveys and studies, JAWS is the most commonly used screen reader with Visual Studio. We test with NVDA because JAWS is a product that has costs associated with it, NVDA is free for all developers, so we didn't want to optimize for a screen reader and tell everybody, yeah, we work great with the screen reader that you have to spend extra money for, we want to make sure that we had a good solution for folks who wanted to use another screen reader that was freely available, and we're also working with narrator. When we first started doing visual studio testing, we had a lot of issues with narrator, but we work with the narrator team now so that we've got better compatibility there, and narrator is starting to move up steadily among the screen reader that people use. When I first started talking to folks in the blind community about what screen reader they used, the not so complementary joke is, narrator is a good enough screen reader to use to get a better screen reader on your desktop. Nowadays, we have developers using narrator as their main tool. Now, on their home machine, they usually use NVDA or JAWS, but it's moving up where I'm on somebody else's machine, I can just turn narrator on and still be effective with it. So we test with JAWS, NVDA and narrator, but we do have lines of communication open to other system technologies, as far as if somebody else is having a problem with it, they can work with us to kind of solve those issues. All right, so that's a very interesting thing that you test with three. It makes me wonder if there's one of those three tools, you said narrator, NVDA and JAWS. So narrator is built into Windows, Windows 10, Windows 7, whatever. If I just test with narrator, is there a good chance it will work with the others as well? Like if I do a manual pass with narrator, am I setting myself up for improving the situation generally speaking for all of these different screen readers? So the short answer is yes, you are definitely doing better. I've worked with a dietician for years for weight loss and one of the things she always says is a few small decisions makes an improvement over no good decisions. So if you're doing manual passes with narrator, that's going to catch a large number of issues as far as keyboard navigation, as far as automation properties and things like that. It will catch a lot of them. It may not catch some of the nuances. One of the complaints I've heard about both UIA and MSIA, those the APIs versus technologies, is that they're not always consistent in how the assistive technologies interact with them. So the way that JAWS interacts with UIA properties may be different from how NVDA does it. So there may be some nuances to how these specific screen readers are going to respond to your implementation. But if you can get your implementation working well with any of the three screen readers, that is a huge step over not doing any testing at all. Yeah, absolutely. It's a great step. And again, accessibility insights helps there too. But accessibility insights won't help you realize that you've got controls that you can't reach with the tab key. Or it won't help. Well, I think it will happen with ones that aren't accessible by a tab, but it won't help with your order of your controls is wonky or you're jumping around the page or things like that. Those are things that you're only going to be able to test for those when you do it manually, which is why we had enlist contractors. who know how to test for accessibility. There are certain things that today we just can't test for them in an automated way. Yeah, so it seems like if we can do the automated testing, which I think I saw a survey or a survey like some sort of analysis that was made of all these accessibility tools, maybe it was two years ago, and they basically came out and said of all the accessibility issues, when we throw all the automated tools at a program or website, they can only find 37% of the issues. They can only identify them. I think, so I don't remember if it was 30%, but it was less than 50% of all the potential issues. And so it seems like if we can automate those using accessibility insights or, you know, lighthouse or whatever other browser plugins or whatever we've got, then we're setting ourselves up for major improvement there. And then if we add, if we then just start narrator, which is built into Windows and we start clicking around and actually doing the workflows that our users are going to use as well to just to listen. So is it doing the right thing? Can I use just my keyboard and no mouse? Like throw your mouse away for a minute and just only use your keyboard. Can you actually work with this? Those seems to be some really low hanging fruit as well. That seems like something that might be easy, something that we should just make part of our development story or is there more to it than that? Well, like you said, even accessibility insights doesn't claim to be able to find every bug out there, but the bugs that it can find tend to be the most impactful. They're the ones that are going to, it's the equivalent of a button that has no label and you just have to guess what it's going to do. Those are the ones accessibility insights is going to find. Even though it can only find 30 some percent of the bugs that are out there, those are the ones that are going to impact people the most and they're the ones that fixing those are going to be a huge jump in the right direction. Honestly, even if you don't start narrator and all you do is try to interact just with the keyboard, that is a big jump in the right direction. Turn on narrator to see the information that gives you is another big step in the right direction. Each one of those steps improves and the way I think about it is every bug that I find there, that could be hundreds if not thousands of developers who can now use that workflow that weren't able to do it before. Every issue I find that may seem like, oh, this just doesn't have a very good color contrast. I don't need to think about that. That could be thousands of developers who can't use your workflow at all because of that issue. Internally, when we're prioritizing accessibility bugs, one of the reasons why Visual Studio, actually, I don't want to say anything bad about the way we used to do things because I'm very happy that we've done better, we would look at a bug that had to do with assistive technologies and say, well, that's only impacting 0.1% of our developer base and here we've got this other bug that's impacting 30%. Should we fix the one for 30% or the one for the 0.1%? Well, when you scale them against each other, that way it's kind of hard for the accessibility issues to ever get prioritized. One of the bits of guidance that Satya, Scott, and Julia down the line have said is whenever you see an accessibility bug, an assistive technology bug, treat it as though every single developer is using that assistive technology, now how impactful is it? And okay, all of the buttons in the new project dialogue are blank. We would drop everything and have a fire drill to fix that. And when we found that bug in a preview, we did. We dropped everything and we fixed that issue. So that's where we stop prioritizing it in terms of how many people is it actually impacting? Instead it's imagine what percentage of the people who could be impacted by this are impacted and how bad is that impact? And that changes the number drastically. Now it becomes everybody who needs a screen reader, these buttons are completely broken. That's 100%. That's a bug that we would prioritize. That's a bug we drop everything and have a fire drill to fix. So that change has made a big difference in how we prioritize bugs against each other. And it's part of our culture now. Back in the days when we had offices where I could walk around, I would hear developers talking about what happened with Jaws. Or I would actually hear people using, hey, hanging at each other, their headphones. And I'm like, what are they listening to? It's like, okay, well, yeah, that's what narrator was saying here. They're actually using it at their desks, even in the team rooms. And this is before they saw me. So it's not just because, oh, here's Dante coming. Make sure you check. It's become part of the culture. One of my other responsibilities is I sit on the Visual Studio UX board. And this is the board that goes through any new feature and the UX of what is the user experience going to be here. And I'm the one, well, I'm not the only one anymore, but I'm the one asking, the color contrast doesn't look here. Have you checked those numbers? What is your keyboard order here? How is that going to interact? And if questions come up, I've gone to my representatives in the blind community and said, hey, here's a particular workflow. I'm not quite sure how this should behave. I've got an idea, but I'd love to hear from somebody who lives and thrives with a screen reader. How do you think it should work? Get that information, bring it back to the team, and they can do a better job with it. Yep. So Alexander also mentions here that in Visual Studio, intelligence and parameter end for Windows are examples where Jaws perform better than in the other screen readers. And that reminds me, I kind of forgot, but Jaws also has a Visual Studio plugin. Someone wrote a plugin to Jaws that speaks Visual Studio or that understands specifically what Visual Studio is doing. Do we contribute to that project? We don't really contribute to it other than as folks are working on those plugins, if they've got questions or anything like that, we help them out with it. So those plugins serve two purposes. One of them is if we've got an issue in Visual Studio, we've got controls that aren't interacting correctly, we've got an automation property that we're not implementing properly, the scripts can fix some of those issues for us. That's somebody else doing our work for us. I'd like to see less of those um, realistically there should be better way, Visual Studio out of the box should be able to do more of that. The other thing that they do is like you said, they speak Visual Studio. A screen reader can say I am on a control, the name of this control is login panel or something like that. It doesn't have any context of what that means. A script for a screen reader can say this is a login panel. I'm going to give you a keyboard shortcut as part of the script to move focus over to the login panel and enter your credentials for you. So it takes that next step to be able to translate from what a control claims to be to better improvements on how to interact with it. So it's an interesting space because in some ways Visual Studio 2017 has got a few advantages on 2019 did when it first came out because the people the community has been developing these scripts and helping refine them to be more efficient and speak Visual Studio better and then BS 2019 comes out. It's got new features. It's got new UI paradigms and it breaks some of those scripts because now we've got an extra click or a streamline flow or something like that and those developers need to go back and update those scripts to work with the new flows. Now if they're fixing a BS bug hopefully they just have to remove it and things are better but those we don't contribute to those but we definitely work with the folks who are trying to build them to help them be more effective with it and we try to make sure all the controls are compatible with the assistive technologies because it's kind of hard for you to write a script to fix an issue if the controls you're working with don't speak the UI or MSAA. If they don't talk that language now you've got to find a control that does and then you've got to say okay find this control hit tab three times hit down arrow twice and then enter the information instead of just being able to go to control directly and those are the kinds of things I heard about in those original Jaws scripts that I'm hoping are getting fixed and removed. So the fact that there has that there exists they plug in for Jaws is that does that mean that we have not been good enough on the Visual Studio team to make Visual Studio accessible enough such that one of those plugins are required is success for Visual Studio and accessibility that you know the plugin is no longer required and it just works in every screen reader because we adhere to some you know standard APIs or something like that. Is that fair to say that? It's a tricky question. I would say if you're looking at those scripts and you're seeing things in there that work around bugs in our compatibility with UIA I want to see none of those. I want it to be just the things that make the screen reader users life better. Do I think they should completely go away? No I don't think they should. A script for a screen reader is very analogous to an extension for Visual Studio. You might be saying you know do we want to get rid of all extensions for Visual Studio because the extension of Visual Studio means that this is functionality that VS should be doing. Somebody had to write an extension to fix it. I would argue saying no actually that's a custom workflow that somebody wanted to optimize a particular subset of folks and it was more useful for them to create an extension do it than it was to try to push that into VS. Now just like we do with VS and extensions if there's functionality in those extensions that we find you know we're finding more and more people are finding this useful is our way to get that into the basic Visual Studio functionality. It's a very analogous question so no I don't think those scripts should go away but I do want to see those scripts being able to focus more on how could we extend and optimize the experience in Visual Studio than trying to fix the issues with Visual Studio. So that gives me an idea because I do like to write Visual Studio extensions but I have never done an accessibility check not because I didn't want to but because I knew how to do accessibility checks for websites those automated tests at least but I didn't know about the narrator thing I need to go through my extensions and make sure that they're actually that they make sense and also from a keyboard perspective and all that stuff that's a really good idea maybe we should actually provide some guidance how do you add those that metadata if you need to add metadata to your buttons and click event handlers and whatnot such that they can be read correctly from from narrator we should make sure that that happens. Yeah absolutely the WPF in particular so before I came to the editor team I was on the desktop development team for 16 years first a tester then as a PM and one of the things that I really saw with WPF that I'm glad to see is their compatibility with assistive technologies out of the box. So when you create a button in WPF and you say content equals accept it will automatically set the automation properties so that a screen reader will know the name of this is accept unless you override it so it WPF and Universal Windows have gone a significant distance towards your default behavior your default code is going to be compatible with the assistive technologies unless you go in and override it and if you know enough to override it hopefully you're overriding it in the right ways so a lot of your controls I wouldn't be surprised at all if they're working right out of the box now they're best practices like for instance I don't know how many controls I had that the name of the button was accept a button or address list box or things like that because I had a tendency to put the type of the control in the name because that made it easier for me to control screen readers historically will give the name of the control followed immediately by the type of the control so as I'm tabbing around it will tell me address text box text box well I don't need to hear address text box text box it's the address and then the screen reader will tell me the type so getting the type of the control out of the name is just one step that you can do to be a little bit more compatible with the screen readers you can still work with an application that says accept button button it doesn't block you from working but it's not quite as usable it's not quite as friendly to the screen readers that's a case where the accessibility insights and actually powering up narrator and running through it will help you see those issues that are kind of the ones that are going to be grating on people's nerves and making it harder to be effective okay another point I do want to make there is both accessibility insights for desktop and windows have got plugins to be able to work in uh CICD pipelines so if you're you're talking about running those tests and things like that they have ways to get those into automated for desktop you need to incorporate into like a unit test workflow or something like that but for html content it can actually do static analysis of your html as part of your CICD pipeline so that you know you'll get just like you would get a build failure you'll get notifications of hey you've got an accessibility issue here here's how to fix it you can work that in to help make sure those issues are that much more shifted left and found before it starts impacting people's productivity yeah so um so there's all these different layers it sounds like to accessibility testing and to you know to fixing things up and all that sort of stuff and I know that when we did in visual studio we did our big pass we wanted to get to accessibility grade was it c or b or what was it like can you describe what are those different levels and um if you're out there listening and you're you have your own app or whatever what is the minimum level you need to to achieve sort of and how do you move between the level how do you get the higher levels and and what does it all mean that's a great question so when I mentioned before Scott and Satya kind of got together and Satya said we want to treat accessibility much more seriously we created a scorecard for ourselves it uses the american grading a b cdf scale uh where in f is we have no idea how accessible we are or we're not we haven't done any testing we have no clue where we are and every product in in microsoft started off as an f um there may have been a few over in the windows organization or the office or office has always been very strong uh supporters of accessibility they've done amazing things over there we kind of try to live up to their standard honestly um every started in f we went through our testing we got those back and then we moved up to a d which is we know where our accessibility issues are but we still have accessibility issues and when we define an accessibility issue we've got a whole system as far as how long it's been since it was found how impactful is it and so on but the long story short is if it conflicts with the WCAG guidelines the web content accessibility guidelines it's an issue that needs to be fixed and if we had any of those still around uh that were more than 60 days old that put us at a d grade a c grade is where we cleared the backlog we fixed every bug that we knew about that had that was impacting our assistive technology compatibility everything that was more than 60 days old because once we find a bug we give ourselves a little bit of leadway to fix it we can't find fix it that day and ship or release for it but everything that falls into that bucket we fix it and that's a c grade so that is barely meeting compatibility with the WCAG guidelines and with those WCAG guidelines or what the influence is the EN 501 the second in 301 section 508 in the United States it's affects a large number of the laws that are built worldwide in terms of accessibility the step up from C grade is B which is usable so we are compliant you can use our software usable means that we have gone to members of the community various accessible communities out there given them the software without instruction and said can you do these tasks can you create a new project a new WPF project in Visual Studio can you install Visual Studio can you edit the code and things like that very basic tasks and they would give us a yes you're usable that's five out of five know you're completely unusable I can't even figure out how to get my job done is a one and we have a scale there that says we've got to have better than a three out of five to be considered usable and we're on that journey right now which is kind of a challenging one because when you're talking to the community you're talking about people of different experience levels different things that they've been able to do in the past what one person can do fairly easily somebody else might have a challenge with so we need to be getting in there and we need to do everything we can to get that usability up so that any developer who wants to grab Visual Studio they're going to be effective with and that it's a challenge it really is but it requires a much greater engagement with the community it requires folks that we're talking to and trying and that's one of my tasks here that I do and I'm proud to do is being a part of these communities hearing from folks where the tough points are and then making sure that those are getting addressed there is a step above B and that is a which we would sometimes call the delight level this is where we're breaking new ground this is where we're going above and beyond adjust making sure we're usable but we're innovating in the new spaces the best example of that one I can give is the immersive reader in office if you've ever played with that when you turn it on it strips your strips word word is where I know it best it strips word down to just the text so all the ribbons everything like that gets out of the way so folks who are ADHD autistic dyslexic character dysmorphia anything like that they've got a workspace that just gives them the text it gives them a large rectangle around the line of text that they're working on and it uses fonts that have been researched and peer reviewed and studied to be better for folks with character recognition they've gone the step beyond just can I use the software but how can we understand how new readers or folks who have a condition that makes it difficult to distinguish letters or their their mind jumbles them up how can we make it better for them that's that a grade we would love to start moving from the C grade to the B grade we want to we actively want to make sure that we're more usable for our developers and I've talked to developers who have said DS 2019 is the best one that they've ever used um it is more compatible the workflows are better it just works better that's not good enough we want them to say yeah I enjoy this that I I am happy working in this space uh anecdote I tell frequently to the other PMs when they start wanting to talk to folks in this community was some of the first interviews I did with blind individuals and asking them how what are your feelings about visual studio I started off with on a one to five how would you rate visual studio as a blind developer and he said 4.5 like wow we're doing a lot better and I thought we were all right great so what are some things that we could be doing better and he gave me a laundry list of a solid 45 minutes of this could be better in this way and this is how I work with it so and we just kept going and afterwards I asked him you know I gotta say you've got all these issues all these things it could be better and yet you give us a four and a half out of five his comment was look I can get my job done the things I'm talking about here are things that I would like to see better but the fact that I can get my work done is a lot better than some of the other IDEs out there now this was a couple years ago and some of the other IDEs did make great strides but he said visual studio I can get my job done I want to do better so it's not good enough for us to just be yeah you can get your job done I want it to be enjoyable I want it to feel like this is something that somebody puts an effort into recognize my workflow and actually did the job I need it to do so I guess what happens then is that we slowly move not the entire product moves from F to D to then C which happened in I think you said the previous update to visual studio 2019 16 5 now we're as the visual studio as a whole is the C level and then like is it one feature at a time we then move into like B level is it like is that how it works so do we do everything in parallel or how do you actually yeah how do you actually do this work as a larger team it's another great question and we do break it up by feature area so we already went through the usability studies for the editor and they gave us we gave them a bunch of simple scenarios like can you copy and paste code can you determine where the this code has got an error in it can you figure out where that error is and we got those usability studies back and we're working through those usability studies on the editor to look for those improvements so we're going to basically pick out right now we're focusing on what we're calling the benchmark scenarios those are the most commonly used parts of the s the editor the debugging experience the project creation the shell itself we're working at those ones that pretty much every developer regardless of whether you're an asp.net developer or c++ these are the pieces that you're going to work with we're going to deal with those ones first and then as we continue to move okay when we're feeling really strong about the editor then we're going to start moving over to more language specific workflows and things like that and then they'll will reduce some of these usability studies and say okay well now we're going to do it from a c++ lens yeah you know what you're going to need to use the editor again and that may bring more issues up for the editor team they're specific to c++ that may not have been uncovered when we just did the editor but it's going to it's going to be a journey it's going to be starting with those main the trunk of the tree and then as we move out there we're going to start moving towards some of the some of the branches and I don't know how long it's going to take and vs is an ever evolving tool but what I really am optimistic about is I've got more teams today who are engaging with the community as they're doing their feature design I feel like we're going to be getting rid of more and more of that debt and the features that come out at least I hope are going to be more usable right off the bat which is going to reduce that we need to keep doing from the trunk of the tree outward and then again I'm very well I'm not very active on Twitter but I'm active on Twitter I'm active in developer community which is the visual studio feedback tool that we've got I'm always looking for these issues that come with accessibility and make sure that they're getting the visibility that they deserve so it's us engaging with the community it's the community engaging with us and us being proactive about new experiences and making sure they're doing the right things that's a that's a great segue because I would imagine that for a lot of people that need visual studio to be more accessible in certain areas or that would really benefit from it can we is there a way that's super easy for them to let us know the issue make sure that when they if they create a feedback ticket if they go into visual studio up to help and then send feedback to report a problem for instance is there a way that that will automatically be recognized as this is an accessibility thing therefore it will automatically be prioritized do we get enough bug reports coming in about these things do we want more like how do we is there something we can invite the you know our users to help us do you know for everyone's benefit really are we doing enough on that front too like I don't know many questions here in one but thoughts yeah it again it's a great one and I want to invite people use whatever avenues you're most comfortable with the feedback to a visual studio we regularly scan that for certain keywords that say hey this is an accessibility issue and when those get filed I get an email about it as a divisional lead for this if somebody is reporting an issue with color contrast or screen readers or keyboard accessibility or anything along those lines it gets flagged and sure I get some that are flagged for me that aren't accessibility issues but I would rather sort through a dozen issues that aren't actually accessible if that means finding the few that actually are so we definitely use that I have had people come to me directly on Twitter and say I've got an issue here can you help me with it there are better routes than that one but that's the only route that you've got available or you feel like you're having a problem we actually did have a problem with developer community at one time the send help workflow had a keyboard blocking issue and I had someone reach out to me on Twitter and say I tried to report the issue and I couldn't because the issue was with the tool to report issues worst case scenario honestly and that was another one we fixed very quickly but they reached out to me on Twitter and that's how I became aware of it and we were able to fix that and those tools now have got better workflows in them to prevent issues like that from recurring I can't promise they won't occur again but we're trying to make sure that we learn from those mistakes but yeah any of the avenues developer community is the best it's the most consistent and that's the one that I get a personal nag mail of hey these issues were filed but please don't be quiet about it if you're having an issue with Visual Studio if it's not doing what you need it to do or even you've got feedback on ways that it could do it better please send that our way make a suggestion report it as a problem however you want to do it I want to hear it my teams want to hear it we as the organization want to hear it I I'm more and more of the PMS when I first started doing accessibility back in 2010 nobody wanted to hear about accessibility because it wasn't prioritized now teams want to know about it they want to engage with this community the more we can hear about it the better there is no such thing as an issue that's too small there's no such thing as an issue oh I'm sure they already know about this no we don't we we like I said the send feedback it had a keyboard accessibility issue and we didn't know about it it got past us we put things in place but if nobody tells us if everybody assumes we know it's going to go away we're going to miss it yes so I think we're running long time that's right we are we're at the end here we're almost at the top of the hour so it's time to say goodbye and Dante thank you so much for for hanging out here with me this morning this was like so interesting thank you I really appreciate the chance to get out here and chat with folks I hope this reach out reaches out to more folks feel free to follow me on twitter or anything like that I'm looking to the community especially folks who fall into that invisible disability category we're really trying to do some research on how we can make visual studio a better product for folks who identify as having ADHD or on the autistic spectrum or things along those lines and feedback is the way we're going to get it so thank you for inviting me here I had a great time talking to and I look forward to chatting again and talk about this stuff more in the future I'm not going anywhere soon awesome well thank you Dante and thank you everyone for listening I see several blog posts in our future here on the visual studio blog about some of these subjects we talked about because I feel like this needs to get out there so that we can maybe as developers get some tips and tricks to how we can start doing accessibility testing and fixing bugs while we're working on our code instead of sort of afterwards this is the thing we call shift left we shifted from being an external process into our main workflow and if we can do that that would be fantastic so watch out for the visual studio blog maybe Dante is going to volunteer to write something otherwise I feel like I should so thank you very much and see you next week same time same place adios thanks again