 So I'm Peter Jonas, this is Mike Sabatella, and we're talking about accessibility in MuseScore. So what is MuseScore? Well, it's the world's most popular music notation program. It's free, open source, under GPL version 2. It's available on all platforms, translated into 40 languages, and it's written in C++ and Qt. And we also have a website, MuseScore.com, which is home to the largest online community of sheet music creators. And what is Qt? Well, it's Qt, but often called Qt, and you might hear us call it that by accident at some point during the presentation. It's a cross-platform UI toolkit that basically enables apps to run on lots of different platforms. It's open source under various GPL licenses. And it offers two, there's two parts to Qt. There's the traditional C++ framework, which basically adds event handling to C++ and widgets, which are like UI elements that are all ready to go. They look like native UI elements that you can use in your application. It also has a QML language, which is sort of a JavaScript-based programming language. And rather than giving you pre-made widgets like in the C++ framework, the QML gives you the building blocks to create your own custom widgets and custom UI elements. So now we'll talk about some general accessibility concepts. So what is accessibility? Well, it's basically inclusiveness, so designing for everybody. So in particular, people who struggle with the traditional interface of a mouse and a keyboard and a monitor and speakers. Because everybody's different, we all have different abilities in terms of sight and hearing, or even in terms of movement. Some people might struggle to use a mouse or a keyboard and various other different factors. And one in five people have a disability, so it's quite common. And we all know that technology is an essential part of modern life. But our focus is going to be on accessibility for blind people, and the reason for that is that it's one of the most difficult forms of accessibility to implement. Because we're not just sort of adapting our interface, it's a completely new interface that's not based on visual interaction at all. But it's also more rewarding than other forms of accessibility in the sense that when you get accessibility right for blind people, you often find that it fixes other accessibility issues in the process or makes them easier to fix. And it also leads to better overall design as well, even for sighted users. So when we talk about accessibility, we talk about assistive devices. So this is a device that helps somebody to use technology as the rest of us do on a daily basis. And the main one for blind people is a screen reader. So this is just a program that runs in the background and announces what's happening on the screen, on the computer. So it's a synthesized voice, like a robot voice, and it will announce whatever element on the screen currently has keyboard focus, what is receiving input from the keyboard. So imagine if you're on a button, the screen reader will tell you it's a load button, and there's two pieces of information there. Firstly, the button is called load. That tells you what it does. And secondly, it tells you it's a button, and that allows the blind person to know what they're supposed to do. So they're supposed to click it maybe by pressing the space bar. There's more information that it loads the file from your hard drive. So the experience of using a screen reader, you can imagine it a bit like using a telephone menu system, like press one for sales, two for marketing, and so on, in the sense that the blind person, because they can't see, they have to listen to all of the options before they know what to do, and they have to keep in their head a mental map of what all the options are that are available. When it comes to programming for a screen reader, the screen reader is not able to see pixels, so it's up to the developer to tell the screen reader what elements are available on the screen. But you don't tell the screen reader what to say, you just tell it what's there, and then the screen reader is clever enough to work out what to do based on that, and this allows the users to adjust various settings like its speed, pitch, velocity, and so on. So here's some popular screen readers on the various different platforms. So on Linux we have Orca, Mac OS has voiceover, Windows has a choice, and that's down to, in the past, the built-in screen reader on Windows hasn't had a reputation for being all that great, but it's actually improved quite a lot in the last couple of years, so we might see it start to overtake the other two there. Another way for a blind person to interact with a computer is via a Braille terminal, which is a row of Braille cells. Each cell is a pattern of raised dots, and the blind person reads the dots by touch, and each pattern represents a letter or a digit. The advantage over a screen reader is obviously this is silent, but it also means that the blind person can read it at their own pace and they can skip back and forth, whereas the screen reader just reads at its pace. And of course a Braille display will work for somebody who's deaf-blind, whereas the screen reader's not going to work for somebody who's deaf. But the disadvantage to a Braille display is that it's another piece of hardware you have to carry around. You can't get them built into consumer devices. You need to learn Braille, and they're extremely expensive. But the good news is as developers you don't need to get hold of a Braille display, because they mainly just repeat what the screen reader is saying. So you can test with a screen reader and you can ask your blind users to tell you if the Braille display is giving something different to the screen reader. So get feedback from the users in that way. So how do you program for an accessible device? So if you're programming for an ordinary monitor or keyboard or so on, you don't do that directly. That's all done through a UI toolkit. And the good news is that the same is true for accessibility. So you can use your same toolkit, so QT, and it will, in theory, take care of accessibility for you. And so no need to import any more libraries, but if a problem occurs it could occur in any of these layers in the tool kit, the framework or the device, and it's not always obvious where the problem lies. So that's a disadvantage there. So I'll give you a quick tour of MuScore's interface. So in the middle we have the score view which displays the current score. At the top we have some toolbars for saving and loading scores and adding notes to the current score. On the left hand side we have the palettes where you can add elements to the score. On the right there's the inspector where you can change the playback properties or display properties of things that you've already added to the score. And at the bottom there's a status bar that gives you information about the element that you've already selected. So that's the interface for a cited user. And now Mike's going to tell you about the progression of accessibility in MuScore. Thank you. So accessibility didn't happen all at once when we first were developing MuScore. It basically was not accessible at all. I mean we used the cute widgets and so forth, but if you don't give them appropriate names the screen reader doesn't know what to read and it'll read the value of a spin box but nothing else. And then our use of the widgets wasn't really well optimized in terms of their arrangement on the screen, how you could navigate through them. So really it just wasn't very useful at all for blind or low vision users. For MuScore 2, which came out five years ago, we really made a concerted effort and it was some collaborations with a few other organizations including Google Summer of Code where we had a couple of students who implemented some really good features for us. So we were able to implement what's called modified stave notation which is this kind of large screen or large print additions that's not just larger. It's like scaled in particular ways to make it easier to see. So we had to implement a lot of settings for that. And then just making the score view itself accessible so that you could read the score because the score view was a custom widget before. It had no accessibility whatsoever. So we implemented the ability to navigate it by keyboard to have the screen reader read it but still not for the palettes which is how you add symbols to the score. So editing wasn't very accessible. For MuScore 3, which just came out last year and we're continuing to improve, we've really taken it up a notch and now it is the case that you can really edit scores. And we've done a lot of things including removing the need to have to see the score to be able to edit it. I mean if you put symbols on the score they used to appear where they felt like appearing and now we resolve those automatically so you can be pretty trusting that you're going to get a good product just by entering the things with the keyboard. And we've totally re-implanted the palettes using QML and Peter will talk about that to make them screen reader and keyboard accessible because they weren't either. And we've really improved the navigation and screen reader output for the score view. I'll talk a little bit more about that. And the bottom line is now we really have a pretty good story where blind users can both read but also create and edit scores. Okay, so have a look at MuScore's New Score Wizard which is where you go to create scores and this is kind of the best case scenario in terms of accessibility. So we have some text input fields there and they each have a label next to them so title, subtitle this is all the basic metadata about your score. And these are all standard Qt widgets and as a standard widget they basically give you accessibility for free so the screen reader knows how to deal with these text fields but there's a problem though which is that those labels next to each text field are not attached to the text field in any logical sense so the screen reader doesn't know that title goes with that particular text field so when the cursor is in the text field it will tell you it's a text field but it won't tell you what you're supposed to do. But the solution is within Qt you can give the text fields an accessible name and an accessible description and that allows the screen reader to know what they're there for and you can also use the buddy system which does allow you to link the labels with the text fields. So this idea of accessible names and descriptions it works with other widgets not just text fields but any standard widget like a combo box or a radio button and the screen reader will know what to do as long as they have the name and description. But sometimes you need something a bit more complicated and you might need to use an item view so this is a way of combining a bunch of items together within one large widget so now rather than just the widget needing a name and description every single item needs a name and description. The items are not widgets this is a sort of technical distinction and it means that you can't access them with the tab key so you have to use the arrow keys to change between different items in the view and there's various other properties you can't set on them. So now we'll look at something a bit more challenging than the new score wizard which is the palettes. So the palettes here is how you add elements to the score and you can see that the design that we were given for sighted users is like a tree of grids so we have these categories of palettes, clefs, key signatures, time signatures and each one expands into a grid of symbols and the difficulty here is that there's no standard widget that will give you this appearance in QT so your options are to combine multiple widgets to try and create something that looks like this. You could create a totally new one by implementing the interface of Q abstract item view which is the base class of all item views or you could leave the widget mentality of Qt and try the QML framework and that's what we did and it turns out it's more flexible than the Qt widgets so in the QML world everything is an item and you're able to set its accessible name and description you can set whether it appears in the tabable whether you can reach it with the tab key or whether you can reach it with the arrow keys so on the left here you can see what the palettes would look like in terms of tabable objects if each was reached with the tab key then it would take 50 presses of the tab key to get from the search box at the top all the way through all of the items in the palettes and back to the top again but if we make use of the arrow keys as well we can combine the whole tree area with just one tabable object so now it's just four presses of the tab key to do a complete circuit and while you're in the tree you can use arrow keys to navigate so you can make things much improve the experience for blind users and keyboard users using that method and now I'm going to talk about the score view so the score view is the view of the score itself and here is where we really had to think outside the box of how to do things we want it to be keyboard navigable but we also need to have screen reader feedback so the basic model we knew we wanted is if the score view is showing you a particular note is selected and you hit a cursor key like the right cursor key it would move the focus to the next note and hopefully the screen reader would read and uh... there you go on uk english that's why I said crotch it instead of quarter alright so um... the uh... navigation model we can like use a text based model um... for this to some extent I mean text is linear music is at least somewhat linear um... so we can have characters in text kind of map to notes in music words where you use control left or right to navigate by text that's a measure or a bar in music and then lines of text can we can compare those to staves in music but it's unfortunately not quite that simple um... like for piano music this is piano music there's two staves here in parallel one for the left hand one for the right hand there's nothing linear about that they happen at the same time and even within a single staff there's multiple notes happening at the same time and there's other symbols that are sort of outside of that uh... flow like this symbol here is spanning several notes so you have to come up with a way of linearizing that so that people can understand how to navigate it and so what we ended up doing was coming up with a set of commands that just do exactly that they linearize this so that as you move through it it'll know I'm going to go from this note this rest to this symbol to that note to this and then back here in a very standardized way um... in which you can learn to read the score that way it's not dissimilar to how Braille might linearize it now that we have the keyboard navigation getting well we um... now you need to get the screen reader working and so the uh... the main thing you have to do is make sure you're presenting the right information to the screen reader so it knows what to do and so uh... qt provides what's called the q-accessible interface and if you define certain functions then in theory the screen reader can access those functions the most important is to choose a role for your widget and this is how the screen reader knows how to interpret it like Peter was saying a button means oh that's something I can click so we had to choose which widget which role fit the best and trial and error told us static text worked the best we pretend the score is just text and then the screen reader knows how to read text so um... what we have to do is provide that text and there's different aspects to that you want to have that widget give it a name and then there's value and description that go along with these widgets so we are using for both the value and the description information about the score and so what that might look like is at the end of every command when I hit that right arrow key and the note moved at the end of every navigation command every edit command we run uh... something that will uh... calculate what to give to the screen reader and it's basically a description of the selected element it's time position what beat and measure it's on and what staff it's on but we also optimize this for the screen reader the raw form is in the status bar but for the screen reader we optimize it so that the information is first uh... a blind user will typically we'll start moving through very quickly like that so they can just uh... I heard what I needed, I'm ready to move on so we want to make sure that worked and so get the important information first and really avoid repeating things unnecessarily if you're navigating between notes within a measure you don't need to repeat the measure number we already know what measure we're in so we avoid that and then once we've calculated that information you send a Q accessible value change event and this is what notifies the screen reader the value of that widget has changed in an ideal world that would be all you need it's not an ideal world, the reality is different screen readers choose do I care that the value of this widget changed because it doesn't know what a score view it doesn't know what anything is it doesn't know what's important to read and what's not so NVDA works out of the box NVDA is an open source screen reader for Windows and so this allows all Windows is our biggest platform, NVDA is free so it allowed us to hit a lot of users at once but the other screen readers and the other platforms they have different interfaces, different APIs for how they work and only some of that is abstracted by Qt so we were able to create scripts for Orca on Linux and Jaws which is the most popular screen reader for Windows which is the most popular platform so now that we have those scripts there we can now say we fully support Orca as well as Jaws as well as NVDA within MUSE score so the bottom line here is that implementing accessibility we're not talking a ton of code, there's nothing about it that's hard except wrapping your brain around what needs to be done so it can be hard to find the information, the toolkits are great they provide some information but there's still a lot of platform dependencies in here so you just have to be prepared for some trial and error, there's a couple of tools exercises and accessibility insights will give you some information about your score I mean about your application what the widgets are, that's what you're seeing at the bottom of the screen and what's going to be presented accessibility wise this is a link to a whole lot of some of the different standards, the different toolkits and so forth I want to just wrap up by talking about a real use real world use case, I teach music theory at a university this is a blind student we had this last semester right as we were wrapping up this work so she got to be kind of our first really important test case and I was able to create all my teaching materials for the course within MUSE score and then print them to give to the sighted students and they took their tests on paper and pencil this is her taking the final exam in MUSE score right next to everyone else in the class taking it this way and she was able to read the test fill out the answers within MUSE score and I was able to take the results and grade them and add them as comments within the score and hand her back her grades and all that and it all just works beautifully and as you can imagine it's really actually kind of life changing so thank you so much and I guess we have one minute for questions even yes do you have any idea of bringing this kind of line of design to the score to the accessibility yeah I mean to some extent you have to assume the blind user is maybe creating music for sighted musicians because we're not going to create braille scores for them and so you hope that we do a lot of good things by default and we do but there's definitely situations where you're going to want a sighted person to act as your final editor to insert page breaks in reasonable places and things like that I guess that's the best way I could answer the question about making that accessible we had another question over here somewhere yes yes it it has its own cursor for some reason so with all the other screen readers you have a tab cursor you press tab something gets focus and that the screen reader thinks that that has focus voiceover has a separate cursor so you can have more than one thing that has focus on the screen at once also it was not scriptable I think it was mentioned on one of Mark's slides so we haven't been able to get it to work with the score view because it's not seeing this value update event that is being sent what I'd say about that is you can issue the command in voiceover that says what's selected in the score but that value change event is ignored even though the apple documentation suggests it's the right thing to do and so we have no real recourse there to understand what to do okay one more yes yes we will linearize everything in your score for you so yes there's a lot of things that could be nonlinear multiple voices within a thing multiple markings attach to things we come up with an order and say we are going to walk that tree for you in some linear order