 Good afternoon. We were lucky enough to have Mario and Sam at the conference and it was an excellent opportunity to get aware of accessibility features and lack or lack thereof or misfeature or whatever of our system which is something that we basically never managed to see at the conference and so we kind of improvised this event where Sam and Mario will show how to use a computer and cover and tell us about how it works, what doesn't and well you know what you're going to talk about because I don't. So I'll stop making it up and let you speak. So hi I'm Sam Hartman. I've been a deputy developer since 2000 or so and I've been blind a lot longer than that and so I think what Mario and I like to do here is talk a little bit about how we use computers and it's actually very different. You'll see a very different interaction style from the two of us and then we kind of like to go into discussing Debian accessibility and discussing what we can all do to help. I am a user as far as accessibility is concerned in Debian. I do report bugs you know occasionally I'll generate a patch to some software but I am not maintaining any of the accessibility packages in Debian. Mario is doing a heck of a lot of work to make it possible for me to use Debian and I really appreciate that and appreciate the work of everyone else in the accessibility team and so you know I want to stress that I'm just I'm not a developer of this software so you know take that grain of salt when you're thinking about accessibility. Also there are certainly a lot of other resources for this. Again I'm just a user I haven't really been studying accessibility other than you know is it directly affects me. Jason White who's a Debian developer and who's also blind gave a talk at LCA that Martin Kraft pointed me at that seems like it would have been a really interesting talk going into accessibility going into sort of the evolution of various technologies and stuff like that and apparently he and Mario have done a lot more research into this and so there are a lot of different resources going on. So one thing I'll say is that how I use a computer has changed drastically in the last few years. About four years ago I gained this totally new feature it's called a cell phone. Well I had a cell phone before that but people have been telling me that cell phones have these cool features like the ability to send short text messages to people and the ability for the cell phone to remember a phone number for you. I didn't have that and I wasn't sure I really believed them nor there were certainly a bunch of complicated clicks you needed to do to change the ring tone and you know had to memorize them and all that sort of thing. But about four years ago a German company came out with some software that allowed a cell phone to talk and I could do cool things like have my phone book and my cell phone instead of in my head. I didn't have to remember everyone's phone number and that meant I could call a lot more people. I've been using computers with speech since 1981. But up until last May I didn't run an X server because there wasn't a point there weren't any the accessibility of X applications just wasn't good enough to make it worth doing anything. I did use a console application and you know I'm from MIT so I used Emacs speak which is a collection of ELISP on top of Emacs that makes your Emacs talk and we'll talk about how you get from there to general accessibility in a really frightening way in just a moment. But first I'm going to start out by showing what I do with X and it's a program called Gnomorka which will. So which has this great you know exciting UI with three buttons. Now one of the interesting things is I haven't actually gotten the help button it pulls up a manual I'm told but that doesn't seem to be accessible so I really actually don't know what the manual says that way. But you know I've actually got I've read the XML and so that's how I found out what the manual said but there's probably got to be a better way. Now this is really a help button really preferences. So I'm going to pull up the preference and get this to the rate at which I normally use it. Orca preferences, tab, speech, tab, tab, speech system, cope, tab, speech, synth, tab, voice, tab, person, colon. Now this is great I mean it has lots of different speech you know you can get it into Latin or Lodzban or you know perhaps even some more usable language. Now what I'm doing here is I'm just using normally. I'm using normal gnome keystrokes for navigating a slider. That I can actually do. That's about what I've been doing for gnome. Now of course I might want to also use console applications so we're going to pull up emacspeak. Now by the power of also which I also gained one of the things you'll see here is that since these are using a completely different interface to generate speech they have different volume settings and that sort of thing. Up until last May I actually didn't have the ability to play music and use my computer at the same time basically because the speech systems I were using weren't compatible with ALSA. Why are you doing that to me? And I'm sorry I don't know how to adjust the volume here. One of the problems with emacs is that it's single threaded. So you don't actually want to do very much in this outer emacs especially anything that could touch the network. So we're going to pull up a shell and we're going to we don't really want x applications to come up here and then within this we'll run screen. So now we have emacs or screen within emacs. Now of course I want to access my mail and that's also an emacs but I don't want to have that in the outer emacs because if the imap connection ever hangs then all my console applications would hang. So we have emacs inside of emacs. Now one of the things you can note that these have completely different interfaces to the GNOME applications. Of course we can go over to screen readers. We can go out to our print of our site and look at you know this is ice weasel. Which is reasonably accessible. And I can do these things. You know I can basically look at any of those links. We're on a full link. Then come the pending link. Then come the website link. Then come create a count link. Location text. Then come heading level one. Welcome to the paper submission system. Heading level two. Well that's not actually the screen I wanted. Where do I get? Then come the website link. Create a count link. Location text. Please wait. Search using Google tech. Then come the website link. Create a count link. Then come the website link. Location text. Then come the heading level one. Location text. Search using Google tech. Logging link. Create a count link. Logging link. Please wait. Finish loading. Then come the website link. So someone has redesigned the print of our site this week apparently. New events. All events link. Travel dates. Free run for them. Confirm these. Logger link. Then come the website link. General panel text. Type in text. First name text. Then come the ongoing Settling. General panel. First name text. General. General text. Okay and you see we have these tabs up here and I actually don't know if it it all makes it clear what I'm I'm locating but we have the tabs you know general contact etc etc. Unfortunately iSweazel doesn't seem to be good enough that you can actually click on those javascript on clicks which basically should basically that's the sort of state of accessibility of of you know web browsers under uh under x. And so for a lot of websites unfortunately I find that I am still forced to use Windows. And this will be a little bit amusing because what we'll find here is that it's yet another speech synthesizer with a completely different screen reader so now we've got three screen readers and you know three different access technologies involved for three different things. One of the um one of the things I and this stuff works pretty well but there are some problems like the fact that you know that iSweazel has a little bit to go and I'll demonstrate our dear friend. Oh yeah I uninstalled gksu because it was uh okay I will not show that um basically if you run gksu even with um its keyboard focus grab disabled it will um it will hang your entire genome session apparently any single misbehaved application will take out the entire axis of accessibility module. Um but on the other hand things are really really looking up. I can use open office well enough that if I want to um I can hopefully Microsoft won't mind that um you know um but um I can still use most I can use a lot of software for example I can use ecaca I can use rhythm box I can use um not that many vlc doesn't really work that well totem sometimes one of the things like we had risk into various software might not be installed latest volume to fix this problem. So we have an entirely different speech synthesizer help them control um with arrow um but basically this will allow me to most windows software works pretty well and it's really nice to have both of them alongside um but um but basically there you know there are a lot of genome applications that are usable open office has some words for example uh you get a dialogue in open office um it doesn't read the text of the dialogue it just says open office 2.4 yes no and you can click on yes or no um and so but but on the other hand the main body of open office is actually quite usable and um is you know reasonably um I mean I can write documents in it um I I actually find it pretty useful for writing webpages uh when for some reason I want to write straight html or like that the other big gotcha is that there is no kde accessibility to speak up at all uh not there yet um but I want to stress that for something that that basically wasn't worth using at all in my opinion a year ago yeah this is really amazing progress and I'm very pleased with what the community has done I think there's a lot to do but this is great um and basically one of the things you are learning here is that accessibility is composed of a lot of different solutions you find everything that's out there you get it all working together and then um now I think Mary is gonna talk about a totally different way of using the computer um do should I step down at this point or and does Mary have a microphone or do we and am I going to hello ah yeah it works hi okay um my name is Mara Lang and uh actually joined Devin back in 2002 just to uh get some accessibility into Devin and first thing I did was to create a Devin accessibility project inside of Devin that's basically we have some mailing list and yeah trying to coordinate things and then I went and uh packaged all sort of obscure software I would say that uh yeah just blind people or people with disabilities are going to need and use um as Sam has already explained for a long time for blind people the only way to access Linux was basically the console so yeah this a little bit I guess I'll start with this yeah maybe to explain how I'm using the computer I'm actually the person that really likes to use Braille output uh I I know how to use speech and I sometimes do it for like reading long text uh but yeah I'm more used to to to reading Braille and I'm really more comfortable if if I yeah I can touch what is on my screen so uh how does this work actually technically I mean uh I'm using for a console I'm using Braille TTY which is basically a program that accesses the devvcsa device on Linux and uh reads from there the current console content so um yeah um a Braille display actually has like uh in my case 40 characters uh and one of these characters consists of uh like eight dots so you you end up having like eight dots like in eight bits so you can represent 256 different characters in one position position um yeah Braille displays if you want to have a portable Braille display you end up being uh limited to just 40 characters because everything bigger would be like no not portable anymore yeah sure sure that would be great does it work try to run x run that's that's auto huh x x run x run that's that's auto yeah maybe on the x now so now I am I guess there's one of the problems uh the performance is really not good yet with the uh gnome solution so on my laptop this is really slow so yeah that's the big question if it's still working now I guess not does it come come come is searching oh okay no okay can I do a fan at uh five try it maybe turns on the audio the output maybe not not okay not okay sorry uh so okay where was I um yes okay uh so since we can't really demonstrate this I'll just talk a little bit about um how it's supposed to work you you've got like 40 characters and uh you've got like navigation keys on the display so that you can navigate uh across the screen because uh yeah the screen is way bigger than uh what can be displayed so yeah that's basically how Braille works on a console it's no technical problem after all because text consoles are kind of like accessible by definition yeah on on x like this so I'm trying because there's actually uh for developers of gnome orchard there's actually a way to uh show what the Braille display is currently showing so I'll give this a try network yes okay so I guess we've seen this already um if I now go to the configure or something and say we want to have a Braille monitor it should actually display one which is outside of the screen so oh okay ah it works right so um this actually displays what is currently uh on on my Braille display so if I scroll around you can actually see on the Braille monitor uh just a small portion is highlighted so in this case we are in the console um and if we are like switched to I don't know you can see that uh whatever is highlighted where the focus is that's uh that part is displayed on the Braille display there's also a way to scroll around in some sense but yeah most of most of this is uh still in development I'd say um now that I'm here there's probably one interesting thing uh maybe I should talk about what's actually going on here um in as I said in console mode it's pretty easy to to get at what what is displayed in in the graphical mode it's actually not so clear uh how to decode what's on screen so you can just go and do an OCR of the content on the screen that would just not work so you need to go a level deeper and actually have a way to query the application and ask the application what what information it is displaying and what type of widgets those are so um the GNOME project actually started to implement the infrastructure necessary for doing this on on on free software systems in general um and uh the basic thing is called AT-SBI it's the assistive technology service provider interface that is kind of uh a demon uh which uh where assistive technologies like screen readers or yeah other tools which we might show later to uh connect to and they kind of like get a uh a listing of all applications that were registered and you can imagine it like a three like structure where all this information is uh yeah distributed to the screen reader on the other side we have on the other side we have the toolkits which somehow need to communicate this information to the AT-SBI and they do the and they do it via the accessibility tool kit it's the ATK um now if a toolkit implements accessibility it's usually done uh based on standard widgets so in GTK for instance or the standard widgets already implement uh the ATK course necessary to distribute the information to the AT-SBI so if your application programmer and you're using uh standard widgets you can usually assume uh that the information should actually be distributed and should be reliever I mean testing is is always a good thing but as long as you're going with standard widget that should actually work if you're designing a custom widget uh the AT-SBI has no way of knowing what kind of information you are displaying or what kind of of widget that is so in this case you actually have to link to the ATK and do manually some function course when you create your widget so that you can tell the AT-SBI what kind of widget this is for instance if it's a value slider or something like that you just choose the value interface define okay uh minimum maximum value and uh this way the AT-SBI can still present a widget to to a screen reader and the screen reader can do something useful with it um for developers it's probably pretty interesting there is one application uh you can use to debug this information if you have uh um some application like written in GTK um and you want to explore the the accessibility properties of that you can use exerciser um which is packaged in Divan um I'm just going to show the interface for for a few seconds I'm not actually a user of this because uh yeah but it's really useful tool for for sighted people and interested in developing in this area because you can uh extremely fast uh look at the three that is currently exposed via AT-SBI so in this instance for we we can just scroll down the top lever applications that they have read registered to the uh to the service provider interface and to have something familiar I guess we choose the orca which has an interface as well and then I can just hopefully expand it yeah and for instance this is like the quit button which registered itself so you could now go on on this button and and examine uh what interfaces have been registered uh what that button can do like it's it's an action interface and if it's an action interface you can invoke the click action or something like that so that's basically how you would uh look into the accessibility infrastructure without actually using it yourself that's probably the best debug tool that's around um regarding toolkits what is currently available and what does work um GTK being the first uh to to actually implement this since version two so GTK two standard widgets are supposed to work um as some of you might know the java swing uh api has for a long time always had internal accessibility and this is also a bridge to the AT-SBI these days on linux so if you are running a standard java swing application and uh do some minimal configuration then uh yeah it actually works and uh what is pretty new and kind of exciting is that um the mono people have actually done a big effort to make winforms accessible on linux so if you um have some winforms application you can bridge that these days to the AT-SBI that's not not package that in devend but i'm actually uh waiting for the mono team to to uh yeah sponsor the packages which are basically already built i tested it a few days ago it worked and uh yeah it basically works uh what is really missing just as as some has already said is qt for a long time now um this is basically an implementation problem uh since qt for they're actually claiming in the documentation that they are accessible on linux but the reality is they aren't because they are not bridging to something so it's internally accessible i guess but yeah um but this is actually a technically easier it has something to do with the with quabba and uh kda or qt not wanting to talk to quava and not wanting to have the dependencies um currently there is a big effort to rewrite the complete uh AT-SBI uh on top of debas and hopefully we will then all be fine and can talk to each other without having problems so that that should be the the the next big step i hope uh to make a lot of other applications accessible um yeah what else am i supposed to i sam and i talked before and and i i think we really don't want to have this as a as a talk session alone so it's it's more like a buff so i i guess what's uh yeah i guess i mean i think the questions right i guess um so if there are any questions out of the audience we uh i or we are really happy to answer them because it's yeah the accessibility field is is um hello yeah yeah okay thanks well something that came early to my mind started when uh when sam was explaining his settings and also should be uh uh well an important point with the braille access methods is uh i guess you have a even larger problems most users do regarding internationalization i mean what what i i can think of and you may tell me i'm way off is that when you're using text to speech well the some some strings may not be and you you may have your preferences in one language and then things get read all garbled and uh mario for you i guess well 256 is not enough for everybody but i i mean uh how how how do you cope with this uh this situation um yeah it's obviously not possible to to to to uh yeah represent a full unicode set uh there there are solutions in in in braille at least to to represent uh the few characters that are outside of your character set so you usually do it with preset and prefix characters or something like that so you have a character which indicates that this is something special and maybe two characters should be rated once or something like this so there are solutions but yes it is a problem yeah you can you can just um have all all the available characters and really read them at once that's it's kind of like the problem under linux with consor you have got like 512 font entries so no matter what you do on consor you are limited to to to a smaller amount of characters so you have to choose what what you are working with okay i'm relaying a isc question because uh there was a question about how how is the integration of the debian installer like how does it work or does it work anyway ah okay uh thanks for reminding me actually um uh i think one of the one of the big steps we actually made in devon accessibility project was to get devon uh installer accessible at least for braille uh that's something that works partly since edge if you know how to do it you could actually install a devon system completely on your own and it works even better since lenny uh since we have now wispy auto detection so if you have a braille display with an wispy caver you can just connect it to a random pc and throw a di lenny installer in and it will completely auto detect everything and you can just install it yourself that's actually something that's like on windows will never work because they don't have the screen reader integrated into the system so on those on those operating systems you will always relay on the help of someone else so that's probably the reason because why i'm really proud of that we got this working actually yeah christian pey speaking um we yes there has been a very good integration of braille display as far as i know as far as i can follow or as far as i can understand a great work by by samuel tibaud and also mario of course what we are needing probably i would guess is to integrate some um speech outputs uh there have been in the past quite two or three years ago i remember around here in extrema dura we met with some orca people along with france pop and we began talking about maybe integrating orca stuff into the graphical installer at that time we were having this graphical installer and it's based on gtk stuff i'm mostly ignorant about all this but it seemed to be feasible so maybe if we just had some people working on that uh which is probably easier for non-blind people to to cope with uh maybe we could integrate i i think orca into di or to the graphical installer yeah it would be a very good idea i guess the the main problem here is actually to get a sound card auto detection and uh mixer auto settings working so uh if you have software speech synthesis and uh it's trying to talk to you but the sound card is the mixer is not up and you have to configure this yourself it just doesn't work so what we need is really complete auto detection of sound cards so that the correct modules are loaded and then uh uh yeah the mixer needs to be adjusted so that it's really working i guess it's this uh one of one of the biggest steps in this direction yeah yes uh isn't this kind of auto detection already in debian live cds and usb stick images that you can generate yes so if you could run the installer from this live cd you would boot to an environment where you can actually roast the web and then run the installer um yes i'm sure you can uh uh some people actually work like this i mean you can always boot some some already accessible system and and and use use the bootstrap or something like this to to get the system uh up and running uh but in my opinion to to get the i accessible by itself is is a goal by itself that this does uh you can always on linux do it do it do it some way and and do it with some tricks or something like that but i would really like if if disabled people could just use the same installer then everyone else and just use it like this um so i mean i can confirm that the live cd if you boot it with the i want you to talk parameter which is amusingly access equals access equals v3 um it it works i don't remember if orca's on the default image or if i had to burn a separate cd um one one option to look at for di honestly is so it's my understanding these days di can run out of a live cd um that might be an easier option than um then getting everything working directly in di and still allow you to use di a question most mostly unrelated to debian but uh one thing i would like to have when i have the knowledge in the room in norway there's been a lot of discussion about open standards and the use of uh standardized formats in the government one of the arguments that has been put forth is that the doc format from word is a lot more accessible than odf and o xml is claimed to be a lot better for blind people than odf uh pdf is also said to be a really bad format for uh blind people um well and i can confirm this based on how the government actually are using pdf printing signing scanning and then sending it off it is actually providing images not text to blind people um but can you say something about the accessibility features of different file formats go for it word is word is far better um i mean basically i i mean i'm not sure it matters in terms of formats um because you can go translate you know formats every which way and that's not such a big deal but the accessibility of microsoft word is far better than the accessibility of open office today um open office making huge strides though um and so i i really think for the most part that's a red herring people can use either um and i i mean i think i could get work done it basically open office is good enough that if there were um non-accessibility reasons that you use it instead of word that would be fine um word is good enough that that's certainly claiming that open formats are more accessible is is is just not true um so i i basically think accessibility is a red herring in the debate about what formats our government should use um pdf if it is actually text in the pdf i can generally get access to that and it doesn't bother me too bad too much um i can't i do have solutions using proprietary software for taking it for you know ocring a pdf um i do that often i would rather not have to um but it did come up yeah i guess to to it really i mean i think it's not a file format issue but it's really an issue of the accessibility of the applications you use to access these file formats so i wouldn't say one file format is more accessible than than another it's really it depends on the application and how accessible that is any more questions you want to comment anything else we can finish the talk here hello um if you like i could show some other accessibility method uh that means how to um type some text without any keyboard only using a mouth um a mouth yeah um and there are apparently people who can only blink or see in one direction and so on and they can uh type some text using those techniques so i'm i i'd be trying to show that yeah do it that's actually a pretty important point i forgot to say uh accessibility is not really about blind people only we we are just the example here right now but uh accessibility is really about uh all sorts of disabilities so uh while we were the example now the scope of the dev and accessibility project is actually to try to get all sorts of software uh into dev and which helps uh in all sorts of situations so um it's probably also uh as sam and i demonstrated there is no single solution to a single problem while we have both the same disability we we use the computer in in completely different ways actually so um yeah that's an important point about accessibility it's always about choice you can just say okay you have this disability now you use this tool and and that's it so yeah okay so on the top of the window you see the brain monitor from orca again so that was already explained so i'm going to exit orca and we have geo key which is a gnome on screen keyboard on screen keyboard you can then click on compose and then you get a virtual keyboard on which you can click the character you're interested in so it's pretty clicky clicky but it's a bit lagging and performances are said not to be that great yeah so i'm going to move to another piece of software which is called dasha so we have some boxes and we have uh different areas on the top you've got the lower case on the middle uh the upper case on in red the little red point is numbers in green punctuation and at the very bottom uh stuff like uh white uh white spaces so you only have to move in some direction to select a character you the character you are interested in so that would be an h and then a e and l l o space okay so we can stop a bit that was easy because i already typed that word before so it's predictive and uh the next letters are um automatically closer to the pointer that uh that the other one so i was going to say hello again oh i forgot to say um the text actually type is displayed on the very bottom oh the very top of it so i'm going to continue my little sentence but really said that was cheating a bit so i'm going to spell mario to show you that it's not that trivial so that would be somewhere yeah there searching for m then a a in lower case and then r i and o so you can click to stop and possibly you can move to another point and then do some correction like inserting a new character so we could try to add a question mark there but it um i only spent some dozens of minutes so i'm not quite good enough that would be there punctuation stuff and then and then question mark and that's it any question about that um as far as i know it integrates into norm so you can control applications can you show that uh i don't know that much about integration but the goal is to have that uh text uh input to grab some text and then uh mostly copy pass it into whatever uh form or input text uh text input uh you want in your application i don't know i didn't wet the manual yet no wait wait wait please aha that's a pretty good question and the question was if you can only move your head up and down uh how do you select the text that you type and do copy and paste uh well okay it's probably written somewhere but uh that that's the meaning of how the really interesting part of geoc is to have at the keyboard you can you can have the lines uh moving uh automatically and then select one of them and uh and you only need one event to select anything you have a have as you as you show and cable but you could control all the menus and all things in the genome with only one event okay is a really interesting i'd like to have something that is not um related to disabilities but many people over here have an open moco and um i think there may be a few things that we may sympathize when we see an on-screen keyboard um accessibility is not just to compensate with disabilities is um is to improve what you can do um technology for the blind can be used in satellite navigators in the car when your eyes should actually look at the road um technology for entering a keyboard on the screen uh for people who can type can be used by people who don't have a keyboard so work on accessibility work on at spi is something that comes out really useful when we are actually trying to go beyond the normal screen and keyboard model of computing and we're building into something else work on ocr work on um text to speech uh is something that we can reuse pretty much anywhere and that makes us accessibility technology extremely exciting because we are basically using mario and sam as the pioneers that kind of make sure that we'll be able to use our web browser into something that fits in our pocket and we can just control with like earphones and a couple buttons um and and and that could be a call for for people who actually don't need accessibility technology right away to actually have a look uh and maybe kind of get their hands dirty with it because because it may be useful for many other things if if there's no more questions then we're done with this talk thanks for coming and next talk we'll be talking about hardware design with devian thanks