 Okay, good. Well, today we're going to talk about desktop accessibility, you see, and a thing called UI Automation. Probably none of you have ever heard of it. I'm also going to try and impress you with the OpenOffice transition, so you can enjoy that one. Excellent. So you may have noticed or not that we recently announced another collaboration with Microsoft. It actually was tapped on right to the end of a press release, which no one read. So maybe you didn't notice, but anyway. Turns out Microsoft is, we're collaborating with them to provide UIA, which is UI Automation. I guess this is what it stands for, which is a Microsoft API on Linux. This is what various people say about it. This one is a lovely lady, and she's been involved in Linux accessibility for a long time. When we announced it, it was a bit sad because we knew they were setting up this industry association as well. It would make it even better than it was. But basically, the punchline is that Microsoft is disclaiming all of their patent rights on UIA. That's sort of the business rationale. And putting it all under the OSP. Better than that, they're setting up this body, which has several working groups in it. These working groups. And all of the output of these working groups is also going to be covered by effectively a covenant not too soon. So we can work to merge the Linux infrastructure with Microsoft and take the best bits of either of them, standardise it all in here, and there you go, we're done. So there's lots of companies involved, Adobe, and these other people, Oracle in a way. But yeah, I guess Microsoft is leading it at some level with Adobe and AT vendor. There are loads and loads of AT vendors. AT is the assistive technology for those who don't know. People who make rail displays. People who make software like screen readers and so on, to make computers accessible. Basically, we want to drive convergence. It's a relatively small space with relatively little investment in it, such that it's possible to know almost everyone who is everyone in accessibility and get them in a room and not have many more people than this. So it makes no sense to have multiple competing APIs and how to choose which one to implement. And the net result is clearly worse accessibility and wasting, actually relatively limited resource pool, necessarily. So it's a good thing. So I'm going to try and talk you through briefly the various different designs in this space for doing accessibility and how they're different. So this is the existing Linux accessibility stack. And you'll notice at the bottom here we've got a whole load of core IPC interfaces. These are defined on the store, but it would be a good idea to standardise the IPC. I think that's actually a terrible idea. But at least, you know, there is a language agnostic IDL language you can do that in. And it looks like a good idea to start with. So, you know, whatever. You've got to work with something. I think it's better to standardise at this level, the ATK level, and get interfaces that can be wrapped there, and then hide the IPC so you can optimise it, cache it, and rework it later. You see, virtually everyone is using ATK, unless they're using Python to go directly through Google. It turns out there's some really beautiful Python bindings. So it's very easy to write screen readers natively in Python. And of course, the Java people go directly. They've got their own orb, so I guess it's kind of attractive for them to bind that natively. And so you've got Java accessibility. This is the news story. Of course, before I invested in moving this here, OpenOffice used to sit on top of the Java stack. They used to map between OpenOffice's accessibility and ATK, which made it horribly slow, and it got really disastrous. So anyway, that's where we are at the moment. And it works pretty well. I'm not going to do any demos, just so you know. But, you know, the punchline is that you can use your screen reader, and you can introspect your widgets, and virtually everything is accessible, or the increasingly few things that you can't do. Which is good news, if you're up there. So hopefully the other thing you know about the stack is relatively simple. It's a nice bust to pollute at the bottom, and things talk to each other fairly normally. This is the UIA overviews documentation of the simplified architecture that they have. And clearly Microsoft have got a long legacy tale of compatibility, and they have to work hard to make sure that they don't break stuff and so on. But here's basically how it works. The process boundary is much less well defined in the Microsoft world, because the IPC here is really not very good. It doesn't perform well. Interfaces are not very, you know, not suitably granular for doing lots of IPC. And so consequently, broadly, I'm trying to try and see, broadly application writers and AT writers who are maybe on this side, yeah, they tend to shove a whole lot into the application itself. Ah, I forget. I can't understand the picture myself. Anyway, they take their code and they go and shove it and export it into other applications, where it can run quickly and it can use the native common interfaces, and then they can control the IPC themselves going to their screen readers. And so that's nice. And of course, they can add new sorts of other components into the process. So say if it's Microsoft Office, they can add things that understand the DOM, and they can get all sorts of other different bits of information that can group around inside the application. They can even use, you know, programming APIs that are exposed by various applications. So there's Excel or something to, you know, get even more information and then crunch this all together, shove it over an IPC and main pipe or whatever, and talk to their screen reader. And that's broadly how it works today. UIA, of course, aims to throw away much of this stuff and a nice new clean API. So hiding behind this picture here. How to proc UIA clients. Okay, that's the screen reader. Aha, how to proc UIA clients. So this is where you should be, basically. So you then have a very nice API that is UIA-based that is maybe much less granular. So you can do, you know, sort of more advanced constructs and reduce the IPC around trip load. So you can start to do a lot of this stuff without needing all this horrible hacks in the same process. And so it's, in theory, it's nice. It's a beautiful architecture. I really like it. And interesting, you'll notice that this guy here doesn't really need to know about all of this mess here. That's the other thing. So from an API perspective, they get quite a nice clean API here. It's all the evilness is hidden from you. Including all sorts of bridges that convert the old API to the new API in various pieces here and so on and so on. So, in theory, the new APIs is really a huge step forward. In practice, of course, if you've invested millions of man-hours in making this old thing work and all of your business value is in defeating the, you know, the demons of complexity that were there before, you can understand that someone else doing that for you might not be perceived as pregnant. So that's just an insight for my life. Be helpful. So into this mix, there is also a thing called iAccessible 2. So it's created by IDM, and it's very interesting. So Microsoft have an interface called iAccessible on their platform. And as we explained, iAccessible has a number of problems and it's not as rich and functional as it could be. So IDM essentially embraced it and extended it. They created iAccessible 2. Unfortunately, in common, we can't really do interface versioning except by a penny or number, so hence iAccessible 2. And, yeah, they just inherited from this interface and then added a whole load of new stuff that was good, you know, and people want it. And so that's fine. So they got an interface, but of course interfaces by themselves are not extremely useful. They're like, I guess, document file format specifications by themselves, they're not that useful. You can prop the door open with them or not, depending on how lengthy they are, you know, so you should get more pages in so you can hold the door open. But you really need software that implements it, right? So you can go through the interfaces here, but more than that, IDM encouraged sort of seeding in this industry. And they took, I guess, what was the OpenOffice, you know APIs, which are very similar to the Java APIs, very similar to the ATK APIs, there's also family there together, and they caused them to come, and then they implemented the middle-order symphony, which is, for those that don't know, a fork of OpenOffice based on OpenOffice 1.1 that is slowly going to get resolved in OpenOffice 3, hopefully, free something or nothing. And they paid the major creator of screen readers in the world on Windows to also implement support for their API. And they paid other people inside, again, Aaron Leventhal, who's a very clever guy, who wants to support Firefox 3. So suddenly, there's actually applications using this, and there's actually screen readers using it as well. But of course, it's very nice and incremental. Of course, if you don't even like accessible too, that's fine, just fall back to your old code and use it as accessible. And you can query interface for it, it's either there or it's not. You can extend your application. But of course, there are still some pretty horrible, horrible design problems in it. As we saw on the previous page, probably it doesn't change these nasty proxy problems and all of the evil that lives under the sun. It doesn't make it much cleaner, but of course, it fixes many of the problems that UIA is designed to help with, arguably. So yeah, so there's no review there. Huh, so what is our angle? But based on this, what we're trying to do is we're trying to move UIA to Linux and we're trying to clean up, I guess, some of the stack at the same time. And from our perspective, it's interesting accessibility. IPC is fine, getting a beautiful infrastructure is good. But we have something that works and we don't want to subvert that. It doesn't make any sense. Let's build on top of it. So the new stack we proposed nearly leaves existing screen readers talking to Python, all of the existing applications. And then initially in the first phase we're going to be adding the UIA provider API. So this is an API that allows you to add extra accessibility hints to your widgets if you implement them yourself. But better than that, we're going to be implementing in WinForms. So WinForms is, I've got some pictures later of what you can do with it, but it's sort of a legacy toolkit that's positionally laid out and it's not particularly wonderful, but it's there and you can write your applications for it. And if you've written it for Windows already, you'd really like to just move it over and suit it and run it and it just works and it's just accessible. So that's basically the first phase of the project is to do this piece of the stack. And that will mean that using your existing August screen reader hopefully you'll be able to talk to your WinForms app. Ah, raw, excellent. This man has the D-Bus and the, that's by D-Bus here. You'll be able to call the raw D-Bus. You can have either. It can be raw, so there you are. So that's good. And so in the second phase of it, having done, I guess, arguably the most useful piece here, we're then moving on to doing the client piece. So you could, in theory, write your own AT. I just shall do it. You're coming. There's another door on the back there. I don't think I'm welcoming enough. Anyway, so you can write your clients then using this managed thing and hopefully ship it between Windows and Linux. I think that's probably slightly less useful, but in a completely serious view, it's quite nice. And the other thing is that if you look from a social conscience point of view out there in the world, ATs are extremely expensive, like you wouldn't believe, you know? The third Microsoft project costs $1,000, but ATs are really in another league for charging people. It's like pension providers. They like to steal money from retired people who aren't retired yet. So wonderful. You get a huge lump sum when you sign up for a pension. It's like thousands of dollars. That's all taken from your pot in a long time. But anyway, not too maligned. Too badly, they're wonderful people and they're evolved in the AIA. So I was like, oh yes, but if you could write a nice, simple AT that covers, you know, 70% of the needs across the market, it would be interesting to see what would happen there, running it on Windows and Linux and so on. So maybe that's interesting. It's not something we're interested in, but it's a free software project. It would be quite neat for a lot of people. And so then finally, there's Moonlight, which is implementing Microsoft's little white-tish stuff. Again, on top of Mono, and that needs to be accessible. So we'll be working on that, too. So, what's the next thing to do? So why bother? All right, yeah, great. So, here we have a couple of WinForms apps, that are indispensable, yet another calendar. What the world needs is another calendar application. So I discovered this when I talked to Sonam. They're investing in Sunbird instead of Evolution on WinForms, which is a challenge for me. But anyway, and here's another chess game. Yet another chess game, but all of them written using WinForms. And generally, as I say, it's small, vertical apps, right? It's really irritating if you can't use your silly expense filing system or your silly CMS front-end, or whatever it is that's being written on Windows, and you can't get onto Linux to run, particularly if you're impaired. It's just another level. And we're all used to not having hardware drivers that do proper 3D, right? I mean, that's just small. But it's not such a huge sacrifice if you can't actually use your computer at all to do various things. So hopefully, as we fix the, you know, we're missing Flash, and we're missing this, but we'll also be fixing the, it's also an accessible problem. It's nice, here's another paint.net application. I don't know, I don't know, but that's particularly useful to a stronger impaired person. But anyway, Silverlight is the next thing. So this is, again, Moonlight running Microsoft demo app. You get things, Flashlight, and some other, and, yeah, they work. You need to be able to book your flight in your web browser with the ridiculously pretty graphical thing that someone created. So, I can really finish, which is good. There are no questions so far. We're all sitting there very cowed. So I'm going to heckle. Come on, let me heckle us. Okay, well, here's the timeline that I was drawing in the background and then quite finished. But the punchline is it falls neatly into two equal parts, like this, you know. So the first year we do something, and then the next year we do something else. So first of all, we're starting off with the UI provider and the windforms implementation, as I was saying earlier. And we'll be shipping it first on Slade, but we'll be then moving on to package it and ship it for Ubuntu and roadmap, which is interesting. So it will be everywhere, as it were. And, you know, doing QA and bug fixing in the last quarter. And similarly, in 2009, there were various state milestones and demos for, you know, the various conferences that happened about here and here. One of the interesting things is that moonlight is not actually sort of like 2.0, as I understand it. It's not finished yet, including the accessibility portion of it. So, in fact, you know, we'll be starting to, I don't know when it'll be delivered. Maybe it's delivered somewhere here, but we're not far behind, right? We're starting to really be ahead of the curve there and getting the client on. Actually, another interesting thing that follows from that rather complicated slide I was showing you back in the day is that our recommendation will, of course, be using native managed code for everything. Let me try in there. Let me try and get back to that so I can remember about it for a while. So, our recommendation of this will be broadly, let me try and think which bit is happening. So, this is an improv proxy for user 32 and common control things. So, when we create a WinForms an WinForms application of Windows, I'm sure you don't want to know this, but you will, for a second. What's that? Oh, first one. I hope they're not far enough. So, it turns out that things like the top level window and what you see as a single application is implemented in kernel or normally in kernel and then some of these controls similarly live in, you know, other places but then other more complicated roles maybe, you know, or whatever, doing trees that implemented in a different process again and they all live sort of all over the place. And so, in terms of implementing rich new accessibility APIs for this sort of thing it's kind of a real problem particularly because what you can do in a kernel is somewhat limited and what you can't do and this kind of things. So, behind what looks like a rather simple not very good looking user interface was just this mess of infrastructural compatible legacy stuff that we don't have at all. In fact, our implementation of all of this would be in native C sharp and so it's entirely possible in some corner cases but by the end of 2008 we'll have a better implementation of common controls like WinForms with regard to accessibility than our competitor which would be quite nice. I at least would be pleased. So yeah, so here's the conclusions. accessibility is coming of age. We have been hiring if you are very interested in accessibility and you're more knowledgeable about it or something I don't know if you've got any slots left because Calvin is running the project. And really I think, are there still people that are advertising? Absolutely. So if you have an urge to do all sorts of funky C sharp like you see my ETK tool kit stuff then do ceiling. But I think really accessibility is coming of age and this is going to be really the best platform to do accessibility on. It's going to have beautiful Java integration it's going to have high performance office integration it's going to be wonderful. All the APIs that will be there your application will just work. So yeah, in terms of credits Ken Bracks really owns the space inside SUSE he's been doing all manner of great work for years particularly on the console before any of the graphical stuff happened of writing anything called SUSE or Linux which you know, back in the day allowed you to just boot and install using Braille display and so on for the many years. Someone of course did huge investment here to bootstrap accessibility work across the desktop and the office suite and web browser as well I think similarly IBM, with their cunning employee around the system too and just encouraging all this openness to actually happen and the AII think they have a lot of credit but of course the people in Microsoft are actually doing the right thing starting to open up making their specs available starting to address pattern concerns around them starting to work together with other people and surrender some of the control and things like the AII and Calvin is leading this leading this thing here so you can find out more about it here there's all sorts of nice links and you can talk to people and pretty pictures of Calvin driving through Martin his four wheel drive vehicle so now go and see it online I think I can finish my talk are there any questions? So a hat with a hat How would the ALMY project influence the likes of say Adobe with regards to flash supporting for video over webcamsuring webflash there's currently only support to be correct which most modern webcam drivers don't support well in flash That is an interesting question not one to which I know the answer to flash I don't know it's really more of a moonlight question but I'll just make up an answer I think competition is good in this space it's clear that having AII accessible to and seeing it in a larger place galvanized people into action and started them opening up and creating alternatives similarly moonlight is clearly behind in the rich it's going to be the first free software rich web app infrastructure out there that is amazing if I told you two years ago the Microsoft would be funding the first open source you know web app you would have just laughed it seems ridiculous right still might seem ridiculous to you but it is actually happening so the thing is things change companies do funny things you look at a list of companies here and you can think of instantly some really good things they've done some really stupid things they've done I think it's worth judging what they do by how they do it not judging them simply by a brand if it begins with N if it begins with S it's perfect I think it's a complicated space and we'll see I think the whole world is going ultimately open source we're going to see a you can laugh but I think it's true and I think we're going to see a huge amount of struggle and heartache and pain as big companies try and change their culture and learn to understand what it means to work with other people and to really open your product up and to do that it's going to be a hell of a ride for the next few years as big companies get involved and you know I would love to see this turning into a free software company at least it's more so yes sir a lot of applications nowadays are just normal web apps as in not any kind of silver light moon light flash jizz on the screen the how you mentioned the Firefox 3 accessibility is that a kind of as in the application has accessibility so say you go to the preferences dialogue is that made accessible or is it the pages how does that fit with a bigger picture outside rich that's a really good question is it the content inside or is it thrown around the outside both and the answer is both really it's no good having the documents accessible you can't change locations there are a number of interesting complications with JavaScript applications the creation of very beautiful looking responsive asynchronous Java XML stuff inside a document and making that accessible in a meaningful way and this whole thing of web aria which is trying to standardize this or this group in the W3C working on that and lots of interesting work being on it much of which I don't know anything about so if you search for web aria you'll find out a whole lot of but it's coming it's coming pretty soon that's a good question excellent, any more questions may I welcome some answers for me no more someone was going to help ok well thank you very much you've been very patient