 Hi, everyone. I'm going to get started now. I'm so pleased to see so many of you here for a talk on video accessibility. Accessibility is one of those things which I have to admit myself. I am not an accessibility expert. I have resolved to try to get better at it starting with video. And I'm Jason Rinalo at NCSU Libraries. I'm the Interim Head of Digital Library Initiatives there. I'm at Rinalo. And if you want to, you can go to the handout at Rinalo.com slash VTT, which would work on your mobile device too if the wireless is being as flaky for you as it has been for me. And we'll see if that works. I have some handouts there that if I go along, as I go along, you're interested in anything. It's okay if you drop off to one of those links and explore that. That's fine with me. Yeah, so this session is part of the open source digital preservation and access stream. And I feel very fortunate for AV Preserve and Chris for bringing me here today. So accessibility, what am I talking about? So in one way, I'm talking about alternative or augmented access to video for those with a range of disabilities. So alternative text for the deaf and hard of hearing, alternative audio for the blind, contrast control, you want color contrast for those that are colorblind, and also graphical interfaces that universally aid people that have motion disabilities as well. But I'm also talking about improving access for a range of use cases that are really unrelated to disability. Search engine optimization, actually getting to the needed content, make working with the content easier and more efficient. So I'm talking about accessibility in both of these ways. So when we're talking about accessible video, I'm really talking about it on the web. I am talking about digital video, internet as the delivery mechanism, but I'm not talking about special purpose clients on desktops, apps on mobile devices, that aren't web views and set-top boxes for the most part. I'm really talking about video and browsers. So Firefox, Chrome, Safari, desktop and mobile, some mobile apps that use web views as well. So if we look at the history of video and the browser, we're really talking about plugins, and you'll have recognized a lot of these plugins, but the big one is certainly Flash. But we should consider Flash harmful at this point. There are problems with accessibility as far as keyboard traps. So for someone who wants to navigate a site or needs to with their keyboard, Flash content can sometimes trap you within the Flash player and not allow you to get back out of it. And then we have devices like iOS and later Android that drop support for Flash, and we have desktop Chrome and Firefox that are also deprecating Flash. And right now this is what you would get on some pages, get Adobe Flash player when you're on your iPod. So this is what your users are going to be encountering if you're still using Flash. And so the solution to this is HTML5 video. This means native support for video and browsers without plugins. And it's as simple as this to include video on your site now. There is a video tag in HTML5. HTML5 is kind of an umbrella term for a whole range of new web technologies, but it also brings some semantic elements to HTML that they didn't have before like the video element. So now it's as easy as it is to embed an image on your site as it is to embed video. And how this works is, and this is what's called progressive download, or what I sometimes call fake streaming. It uses the HTTP protocol. It's easy to set up because you can use a standard web server for it. You don't need any fancy streaming server for it to work. The video is stored on the server as a single file and requests are made by the browser for sections of bytes of the video. And as long as the metadata is in the front of your video, the video can begin playing once that first section of the video is delivered to the browser. But this leads to some choices that you need to make about whether you can use HTML5 video. If you need a simple solution for video delivery and you can allow users to download the video, because it is just a single file that you're linking to the source of it, it's really easy for folks to download the video. Now, that's not quite true anymore because there are standards like encrypted media extensions for DRM that are under development, but I'm not going to talk too much about those because that quickly becomes more complicated. So what else can we do with the video element? Well, we can add a poster image and we can add controls. Unless you add the controls attribute, your video doesn't actually have any play or volume controls to it. But one of the problems with HTML5 video has been that this is the kind of thing that you would see if you go to Firefox, some older versions of Firefox on some platforms, if the only video that you're trying to serve up is an MP4, because Firefox did not support MP4. But there's a solution to that. There's a new source element which allows you to use multiple sources of video. And then you can see here you have the paragraph element which in this case is serving as the fallback to let folks know that they really ought to upgrade their browser to something more modern so they can enjoy the web as it is today. And you can also then instruct the browser a little bit about what those sources of video are. So you can say that this is an MP4, this other one is a WebM, and the browser gets to decide which one of those sources that it can play. So you don't have to do things, practice in the past, which you might have heard of, which are called browser sniffing. So this is a better way to do that. And you can even get more exacts. So we know that MP4 has all sorts of different profiles for it. Well, you could tell it exactly what kind of MP4 you have, what's in there, and the browser can decide whether it can play that type. And then there are fallbacks. Yes, flash folder browsers becomes the fallback or you could use a download linker button. Okay, so here's where we get to accessibility. Each of the players, each of the browsers have their own native controls. And if we look at just these examples, Firefox has really poor color contrast for the play button, for the volume button in for the full screen. Chrome and IE have much better color contrast. And IE does a good job because it also then labels each of those buttons so that you have a better idea what you're getting when you click those. And so these players also have not always been the best as far as keyboard navigation through them. That's getting better. But what helps with all this, these accessibility concerns, is that there's a JavaScript API to video now. So I can add my own buttons to pause the video or play the video. And it just takes this really trivial piece of code to be able to do that. And so it's possible then to create more accessible video players. So this is an example that was just released. It's the PayPal accessible player. It has some neat features to it where it's using an HTML range input here for the volume. It uses the progress element here to show the progress of the video. And the idea is that if you're using the HTML that's already available to you, then screen readers will be able to understand that better than if you use something that's outside of that like Flash. Okay, so there's all sorts of events in the life cycle of watching a video that you can watch for. And it allows you to do things like getting the current time. And you can see you can get the current time about four times every second, which means you can add that then onto your video player. And again, this is the minimal amount of code. You don't need to read this or understand it. Just know that this is a short little snippet that makes it really easy to be able to get that current time out of the video. So one of the things I wanted to see about the video that I'm publishing is how engaged are users of the site with the content. And it's easy to get analytics on things like how often people press the play button or how often they pause or if they get to the end of the video. But I really want a deeper insight. Do users watch the whole video? At what point do they drop off? That's a good length of a clip that you want to show without losing someone's interest. And so using that JavaScript API, I'm able to collect that information and then chart it and then have an interface that allows folks to be able to watch the clips back and see the number of people that watch to that point in the video. And, okay, so another cool thing that you can do is now that you have video native in the browser, you can do things like style it. So I kind of... And there's just a little bit of JavaScript. The rest is just plain CSS styling. And I really like that. I'm going to watch it again. I like the kind of spinning that way. So there's all sorts of cool things that you can now do because video is in the web. Support for HTML5 video is very good. It has been for years now. Okay, so when most people are thinking about accessibility for video, they're thinking about timed text tracks. So everyone's familiar with captions. But there's also audio description. So you see, audio description like that is a real art to be able to insert within those spaces a description of what's happening in the video to make it accessible for folks. This is from Magic School Bus, which my kids love, all about science. And to be able to make that content accessible to other children that maybe can't see the content as well, the audio description can help there. And so when we're talking about text tracks, it needs to accommodate all these different use cases. So that was one, if I felt it was getting a little hot in here, I thought that would be appropriate. I don't know how good that translation is, but we also think of subtitles as timed text tracks. And again, it's as simple as including a track element and telling the browser where that content is to be able to provide that content to the browser. And then there's some other attributes. The one I want to focus on here is the kind, and the kind of tracks that are in the standard are captions, subtitles, chapters. So like you have chapters for DVD, audio descriptions, and metadata, which is really a neat one that we'll see some examples of that you can do some really cool things with. So the standard that's used, that's kind of most embedded in the HTML specification. It's a separate specification, but it's referred to a lot in the HTML spec, is WebVTT. So WebVTT is a W3C community group draft report. It's not quite a standard yet, but it's being adopted by the browsers. It's a timed text format for HTML, but it could be used in other places. It's a plain text file that just has some special formatting rules, and it's intended to fit the needs of all of these different kinds of tracks. So some of the benefits of using a plain old text file is it's very straightforward. Anyone with a computer can edit it. It's simply human readable, and it allows for good data portability. It has its genesis in sub-RIP, which wasn't really a standard so much as OCR software that would output, would extract subtitles and insert them into a file. But that format has some limitations. All the measurements were based on pixels. There was no ability to put in header metadata. There were only numbered queue identifiers and no ability to kind of notate that this particular queue wasn't as good as some of the others. This is our first example. Very simple format here. Every WebVTT file starts with WebVTT at the top, an empty space, and then a timecode, arrow, a timecode, and then the next line is the queue. Queues can break over several lines. So you can see how it's simple enough for me to be able to go in and edit these queues to correct them if necessary. So what are some of the features of this format? There's comments, notes. You can add voices, queue settings. We'll see some examples of those. Regions, we can do CSS styling, and you can add metadata header. So this is an example of some comments. You just use the note in all caps and it can break across several lines. And this can help. This is not intended for browsers. Browsers throw this away. It's intended for humans. And so having formats that are human-friendly is one of the goals here. And this is an example of some of the different ways that you can style a caption. You can use bold, italic text. You can position queues at the top to the left and right. This is actually the Research One reactor at NC State University, which was the first academic institution in America to have a nuclear reactor. You can style them red. You can style based on voice. And you can do really ridiculous things like make the text super big. And so this all gets into one of the goals in Times Text Tracks when you have queues for captions is you need to be able to place them appropriately. So you want to be able to place a queue near where the sound is coming from or near the person that's speaking. And so you need the ability to be able to place the lines in different places within the video frame. And here's some... From our example, here's some examples of what that then looks like in our file. So line 5% puts the queue positioned at the top. We can position 5% and a line from the start for a left to right language would be left. Position 95 puts it over to the right and we aligned everything to the end. And we can also then size the width of the queue. So again, pretty simple stuff and easy enough for humans to understand it. And then we can do some styling of this. We can add a class span here to be able to then in CSS use this pseudo element, colon, colon, queue to take the class red queue for that queue red. And you could take any portion of that queue as well and style it appropriately. There are some other tags that are built in like I for italics, B for bold that you can use as well that are kind of shortcuts. You can add voices. Now voices by default don't show up but you can you can style them. And if the voice goes from the beginning to the end of the line you don't have to close that tag. So very simple way to add voices. You can also, there's a feature where you can identify each queue. So here we have a queue ID which comes before our timestamps and we can then use that to change the font to a very large size and color the background red. There's still some work to be done because there's a little bit of hack where you have to add other the other class in there the other tag in there to make it work. But all of this is continuing to be under development in the browsers to be able to improve this. So what's browser support for web VTT captions and subtitles? Look like right now current versions of all major browsers now support web VTT for captions and subtitles. But other kinds of text tracks are not supported natively in any browser. And this is where what we call shins, polyfills come into play that kind of fill in those gaps and custom players help with that. So as far as chapters go while no browser supports it it's truly easy in JavaScript to create chapter track controls for a video player. This is an example of media element.js which is a player that I've used and these are the other ones that I know of that include an ability to create chapter tracks as well. Clouds move through the sky and still more clouds blowing. Nothing but clouds. This is boring even me. Can you please stop the clouds? Yes. So what's behind that is a web VTT file that has a track that's embedded in the page that then links to that file and it has a kind of descriptions. Now while that's not supported natively in the browser what... While it's not supported natively in the browser there is a Chrome plugin that you can install that will read those description tracks for you. It's text to speech so it's not as artful as many audio descriptions are but it can still suffice to make your video more accessible. Web VTT also has this newer feature called regions which is I think mainly being developed to help support roll-up captions. There's some disagreement about the implementation and browser support is lacking for that but you can take a region of the screen of the video and say that's where you want your captions to be. So that's kind of where things are going with Web VTT. I mentioned before alternative transport of it so it's all fine and good if you can use your Web VTT on the open web but you can also there is some support being worked on for using Web VTT in a WebM container. Apple has some documentation on how to use Web VTT segments with HTTP live streaming and there's been some talk about MP4 containers, AUG and MPEG Dash but I'm not certain where those efforts are right now. So this is good to the fun part Web VTT and some tricks we really preference access before discoverability with our content we don't have time to do all the metadata work that we would like we often will not be able to watch every video that we would want to make accessible so thinking about automated summarization services this video when I saw this poster image I was like really excited about what this video might possibly have in it and it ended up being a woman and a boy sitting on a floor talking about a lot of rockets which is kind of cool but not exactly what I was looking for and so you often see people hover over time rail and they'll see the time hover over the time rail like on Netflix or Hulu or on YouTube and see a little thumbnail so I added that feature and then you can see oh yeah that's what the video is about and one of my favorite finds here is JFK really likes to move his head when he talks about how enthusiastic he is about milk and so you can see yeah here's another one you can watch this film don't use pesticides around your pets but it also tells you you can then reuse this in other ways to then say oh yes this is really this whole video is really about boiling water I don't really need to watch this one yeah but then you also find some interesting things like this is about irradiation in food some news report and in the middle there's a nuclear blast there's some footage of some of the men that would be going into the wake of that blast to be able to investigate what had happened and so how does that how does all that work well you take all those thumbnails you stitch them together into a big image sprite and then you can use the coordinates there to position which of those thumbnails you actually want to show to the user and the specification that you use for that is called media fragments so you can get the x y coordinates of where you start and then the height and width take your daily votes now oh yeah that's a okay it's a work day and so and you can also do that in the temporal dimension as well and this can be helpful for things like annotating your video researchers then can say I'm just interested in this spatial dimension or this segment of the video you can also do things where you can just embed something like JSON in the file to reuse other ways to reuse then your VTT you can then why not just put it on the page if you've already got it and then allow people to read it and jump to that section of your video you can then also do a search so you can search for music and you can get to all the sections that have captions that include notes about music so as it goes through it's coloring the words that you should be singing okay I won't subject you to any more of that but here's what it looks like for each of the queues you have time codes that are within that queue and then you have the ability to be able to say whether that queue is in the future or whether it is past and style it appropriately and this is another example where the whole span of the queue is the length of time it takes for it to get from the beginning to the end anyway so there's some tricks and this is just an example where I coded to be able to take many different ways of positioning a queue and run through it just as a test to see how the browser would handle some of the different values that you could do for queue settings so one of the cool things that you could do is you could potentially do live captioning because you have the JavaScript ability to change captions that's awful so we're going to try something if you have any questions and you've gone to the site to this site renello.com slash vtt I made a mistake there there's a slash vtt you could see if I click this link I can then change what the caption is right away and we'd like to ask a question anonymously you can do that so are there any questions oh hi not a question anyone else no does this work yes it does work if you have wireless it does still not a question anyone have a question yes please do honestly what we're doing right now is as far as when we do captions like this there are students at the desk who have downtime and they have a transcript and they can do some cut and paste with a little web tool that we have yeah so these federal requirements for web accessibility I did have a slide in here about some of the captioning requirements and the file format has nothing to do with it being accurate or complete but it can be synchronous and properly placed and I think properly placed is like the really difficult thing to get right and so there's a lot of work still happening in the specification for that any other questions so we have some oral histories that we have made clips available of so our student leadership initiative where we're documenting student leaders through the history of the university we've made those available that thumbnail example of the boiling water that's on our site live so any of our videos that go there you can see on the time rail where you'll be clicking what you'll see as well as get that overview well this is implemented in browsers now it will continue to improve the players that support the things that browsers don't support I think that's probably actually these guys are talking about soy products for human consumption is what they're really talking about but it might be time for a vodka drink any other questions do I have any time? I'm probably probably passed my time yeah if you have any questions at all I'm happy to answer them any time yeah