 So, there is chocolate. I was not lying. Okay, so we're going to all play a game. You are not going to look at the screen. You're going to look at this piece of paper. I'm going to put a question on the screen, and I'm going to have you guys try to answer the question based on what he's about to tell you. And what you're going to do is you're going to take this chart and you're going to describe it to them and hope that they can answer the question. You all have 15 seconds. And 15 seconds. Okay, what was the trend of that chart? Down is the correct answer. Admittedly, he could have just said Bitcoin and you would have guessed it. So, just so you all know what the picture looks like, here you go. I hope you had that visually in your mind as he was describing it. We're going to do this again. Okay, so I'm going to have, we're going to change the picture to that. And I'm going to post the question now. Go. And time. Great. Okay, everyone. What was the price last week? That is actually correct. Good job. That's not what I wanted. Too good at audio description. Okay. So, assuming we didn't guess from nearby values, it would have been impossible for you to know what the price was because usually the exact price on any given point in time is not expressed in an alt text or text description that is what he just provided. So, thank you for your audience participation. You can have some chocolate. Thank you, everyone else, for guessing along with us. And Richard, can you tell us how was that experience of coming up with those descriptions? Yes. Yes. And that is great. Just so you all know, for this chart, I asked a friend of mine to come out with the alt text and he said, some charts just don't deliver. Which was unhelpful. And taught me something about my friends and how helpful they would be. Which is that they think dad jokes are the correct answer to anything. If we can't get that level of precision from alt text, if it's more of a high level, here's the direction, or even like here are the major beats, then how would we get that level of precision that a user might want? So, you provide them the data. So here's data table. So, everyone, what was the price of this Amazon stock on Black Friday? Why yes, everyone was shouting the correct answer in $93.41. Thank you. Great question. And so, like, you were able to find it. You find the column, you find the row. It's fine. Great. Okay. So now that we have precision, we now need to figure out what the general trend is. That was what alt text was good at. And so, what direction is the price going? Exactly. And so sometimes you'll go through, you'll see the dates. You'll see that the dates are kind of in the opposite order. You see kind of like the vibe, the numbers are going in. And of course, I've let you all cheat because you're seeing all of the data at once instead of one cell at a time, which is how you would be doing it if you were, of course, blind. So, that's great. It's nice to have that level of precision, but it's a trade-off between getting precision immediately or getting your high-level overview. So, let's do this one more time. I'm going to, now I want you all to listen and tell me what the trend is. Yeah, what was the trend? It was down. Excellent. So, but we also, that was the general overview. What was precision? What was the exact point in time? So, I'm going to... They probably want me to use the microphone. So, what is, what were we just listening to? And that is sonification. It's when you turn data into sound. It's been around, I think, like sonification, I think it was coined in, like, the 1920s. It's been around for a while, but you don't really see a lot of examples out there of it. Highcharts has something, Apple did something with audiographs. There are a couple of implementations out there, but you don't see them getting a lot of traction. And part of that is because they're all very siloed. If you want to implement something, you're implemented to get yourself or you're using one of their systems, and it is fairly difficult to implement, you need to have a lot of experience around, like psychoacoustics as well as coding, user interface design, it can be complicated to get all of those pieces in alignment, so there hasn't been a lot of development in the space. So, welcome to church music. That's what you were all listening to, and that's what we were all interacting with. So that was not only sonification where you hear all of the sounds, you're converting data into sound, but it was also interactive. So you're going through point by point and getting individual sounds, as well as background information that you would get to a screen reader. Does this exist besides just like it's fun to listen to weird sounds? I wanted a magic wand for developers. I've worked with developers. I am a developer. I've worked with people to help remediate charts and graphs over the past for the last decade, and everyone wanted to do something accessible. They wanted it to work, but they also were busy. They were doing a million other things, and you don't have time to become an expert on a whole wide array of subjects in order to just get the accessibility to work for something, especially something so complicated. So I wanted just something really quick. If you've never even heard of blind people, I still wanted to be able to work for you, where you just know your JavaScript, and it just works with you. I wanted it, and it's so hard to find that. You're never going to find magic wands, but I did it, you guys. And part of the real reason that I think it's going to be so successful is that I included my partner who is blind in this process. He's a data scientist with a PhD. He's been using data... He's been using sonification before in order to get access to deep levels of data and really understood the space and understood the requirements he would have that he would need to get out of, which... So having him involved from the very beginning as part of design and development and the testing and the feedback and including him as well as many other members of community was part of why I believe that this is going to be really successful to be... And one of... One of the most important parts of that is being able to customize things. It's being able to fully appreciate everything that you're working with and doing so completely independently. So what is the developer experience? If I'm calling this a magic wand for developers, what does that actually look like? Well, here on our left is just some code that makes a chart. I found it from a demo site of a thing when I googled JavaScript library charts. I'm sure you're wondering what this looks like. It's that. It's a bar chart. The old standby. The old warhorse of all chart types. So let's say I have this bar chart and I want to make it accessible. How do I do that? You add about four or five lines of code where you're like, by the way, this is a bar chart and that data already handed to the charting library. Here it is. And there you go. That's what you do. And so magically, that bar chart you just saw is being accessible for the blind. That's all I want you to do. You already know it. If you already know JavaScript, you're just like, yes, obviously. That'll do it, man. And that's what I want. That's what I want your developer experience to be. And it's partially working because it's living alongside the chart. You could honestly have a static image. You could have a completely blank space and you'd still be able to get that interactive notification. So I have, honestly, the most difficult part is making all the visuals look pretty for sighted people, which, honestly, the blind people are usually not going to care about. So here are some examples. If you go to our website, you'll see all of these different charting libraries already up and running with chart to music, usually with visual syncing to just, like, really make the demos shine. So it does not matter what charting libraries you have. It doesn't matter what silo you've already put yourself in. You don't have to leave it in order to be able to get the accessibility. We can bring it to you and you can use it whenever you want. So what have the responses been from the blind community? It's great to know the developers are fine with it, but what's the response from the blind community? So the first thing that we got from people was, like, I'm old and I can't hear all of those sounds? Can you change them? Yes, we can. So we added an options library. It was actually very important to me right now because I felt like I couldn't hear the lower sounds when I was in this room. So I just bumped up the Hertz values for the lowest level then until I felt like I could comfortably hear it. And that's what we did just moments before you all came inside. So if you feel like you can't hear the upper level or the lower level, you can just adjust it yourself and it'll remember that adjustment on your page. It'll remember for all the other charts on the same page. We're just using local storage. And it just goes with the way you want it to. You can customize this to be as successful as you want. Was the next thing I heard from people? I was like, can we use this with stocks? I was sharing people's like, here are some fun facts about the census. People are like, we don't care. We want stocks. So if anyone here does anything with stocks and charts, there are a lot of people who'd be interested in that. But finally, my favorite response I've ever gotten from anyone for sonification was this fully blind woman. She worked for the government in Toronto and she played it over and over and over again. And she said, no one decided people like charts so much. And this is why we do it, guys. So just to recap, what are the different ways that you can make charts accessible to the blind? You can use alt text and thank you Richard for being significantly better than most alt text writers actually are. It's the industry standards. It's been the industry standards since the 1990s. It's a really low cognitive load. If you're going down, you're like, I get it. It's going down. But it has fairly low precision. Normally. But it's very expensive and difficult. Can we all imagine Richard coming up here and having to write thousands of alt text because he has that level of experience? No, normally you get randos who show up and they're just like, I don't know. It looks like a mountain, I guess. It's very weird and craggy. There's also automation technology out there. Plus, it's hard. It's trying. Maybe it'll be there in a few years and it'll just generate these long paragraphs that will be the same as a person but cheaper. But you still don't get the level of precision. Data tables are around. They're an industry standard. The cognitive load is very high because you have to hold all of that in your brain just to process it. But the precision is very high. You do get access to exactly what you want once you find it. And it's pretty easy to generate scale. Generally, there's giant asterisk on that. Data cells will crash your screen reader. Please don't do that. You need to implement pagination at that point and also formatting. It's theoretically easy, especially if you only have a couple hundred data points. Anyway. And then we have interactive salvagation. It's not the industry standard yet. But it's user-adjustable. The user can decide how much cognitive load they wanted to have and it'll remember everything for it. The precision can be very high because you can go find the exact data point you want. And then you can generate scale. If you were able to generate the chart, you're able to generate the interactive sonification. So in conclusion, chart to music. It's a better user experience all around while also being a better developer experience all around. Easier to implement, easier to use. Any questions? Yes. Yes. Absolutely. Yeah. And you can never rely on knowing what the end user actually wants to get out of it. Besides hoping it's general vibes. Vibes are very easy to write down. Anyone else have any other questions? Yes. Great. Great. Great. Great. Great. Great. Great. Great. Great. Thanks. Thanks. Great question. So the types of charts that we support are bar charts, line charts, candlesticks, hichoo, Bannplot, etc. and pie charts. The way we and heat maps. We support a lot of things really now but the thing is for our bar charts and our line charts we basically pretend they're all the same thing. We don't care about the fact that in a row, and we let you change between those lines, so if you have like four or five lines we'll let you change between them, but it's still fundamentally the same sonification style, the difference between band plots and open, high, low, close, it's still fundamentally the same kind of interaction. This does mean we don't support things like maps, we don't support scatter plots, which is potentially an issue for some people if you really need scatter plots, but we have all of these other chart types that are all using the same general style, and part of the reason for that is how cognitively easy it is for a user once they have climbed the learning curve of when I make a sound, like the sounds mean this, and here are the interactions I do to get that. Once they've climbed that, they don't have to climb it again, it's always the same interaction, so I was trying to keep sonification as consistent as possible between the different chart types for that reason, but thank you so much, yes. Do you mean are there other accessibility challenges or can interactive sonification be used outside of accessibility? Yes, so I have been told that sometimes traders will look at a picture and have a five and there's the shape of the line can mean something to them, and that if you're more of an audio learner that may be the sound of something, if you just hear something really quickly that the sound will become that shape, it'll become that feeling for you, and so certainly some people have told me that they feel like traders would just use this whether they actually could, like whether they could see or not, but it would be just really convenient to be really fun to have like a different way of doing this. You can also set it up on another monitor and just have it running, and you'll be able to hear the sounds as the sounds are going by and you can like zone out until it starts spiking or until it starts dropping and be like, oh shit, I need to go look at it. Like there are a lot of different uses that you could have if you don't absolutely need it as your only way of getting access to that data. So people have told me it's possible. I think that would be really cool. I also think it would be really cool for blind people if all traders decided we're not looking at anything anymore, we're only listening to it, it's going to make it really easy for blind people to get into the space. So if you think that that is the case, that traders might think it's cool to listen to stuff, please do it. Some people will appreciate it. So there's a lot of the other accessibility issues that are out there, wind up being around visuals, the accessibility issues with charts, the biggest one is color blindness, and that's really going to impact how you built it. And so we have to become a question of what exactly did you do with your chart? You're going to need to learn some stuff about how the math of colors work or get some best practices. I can't really do a lot about that. I can do stuff for screen readers like this. This is what I can do. Anyone else have any other questions? Great question. So we have a couple of different things on our roadmap. One of them is adding ear cons. So we just have a single set of sounds that are already pre-programmed that we're thinking we should be able to add things like dings or other items that users might think are particularly useful. We also were looking into letting people export the audio sounds. We've had some people say, oh, I really like this, or oh, I really like being able to make music out of this chart. I want to construct a chart that plays this particular song. And I don't understand that vibe. But if that is what they want, I can write code. So I'm looking into maybe letting people export stuff. This is particularly important to me when I am listening to podcasts. So we have this chart, but unfortunately we're in an audio medium, so there's no way to show this chart to you. So let me just tell you, this is a chart that's going down and it's making everyone very sad. It's like, what if you were just playing it? What if you just had sound? That'd be so nice. I want to make a tool for podcasters to stop awkwardly trying to provide audio alt text on the fly. They're all very bad at it. Thank you. Anyone else have any other questions? I don't. I do not drink coffee. And I have never drunk coffee. And I don't like it. Yes? We're going to get to the floor. Oh no, you'll have to send me to another conference, Phil. Thank you all. If anyone has any other questions about accessibility or you want me to play any other random chart types for you, you can find me at the FDC3 booth downstairs. Thank you, everyone.