 So thanks everybody for coming. My name is John Florain. I'm going to be talking about Android in short, specifically the way that we replaced the Java layers of Android with an environment of our own. All right, a little bit of a background about me. I am trained as computer engineer at RIT. I work for Sandia National Labs in California. That's a department of energy lab. I work in high performance computing. I've also started doing some research into mobile and security. So as it says there, I like doing open source stuff and most of my coworkers do as well. Is this too loud? Okay. I thought I was hearing a little feedback. Okay. So a lot of my coworkers and I, we really do like working in the open and so projects like this really make us happy where we can take what we do and put it back out to the community. So yeah, smart phones. Probably everybody here has one or damn near. And my first thought when I got one was oh yeah, great. I've got a PC in my pocket now. And I guess it can also make phone calls. They're sort of better than the old PDA model which was you had a computer in your pocket but unless you had the Wi-Fi card then you couldn't talk to the internet or anything like that. And it's got a bunch of nice sensors like the camera, the GPS, the accelerometer. And so you know, there's a lot of people just use it for everything now, right? My girlfriend is, I mean she's a biochemical engineer and she hasn't actually opened her laptop in months. She just uses her phone for everything. I'm not quite at that stage but a lot of us I think are getting closer. You can use it to manage your passwords due to factor authentication, pay for stuff. So they're really becoming very ubiquitous, kind of the ubiquitous computing platform that we've been talking about for so long. But problem with smart phones in terms of you know being a hacker wanting to hack on them is that most of them are closed source to begin with. You look at the BlackBerry, the iPhone, the Windows phone, you're not going to get the source to those. And there's some security issues with these. There's been a few incidents where it's RIM has indicated that they're willing to decrypt users' text messages to each other for the government. I think that happened recently in London after the riots. We've seen the iPhone maintains a record of all the cell towers you come into contact with, meaning that they know your position and store it on the phone. And there's been incidences of say security holes like a DOS attack. You could do a denial service on someone's smart phone by sending SMS. The Windows phone, well nobody really has one. There's probably a few in this audience, but that's it. They also had a similar denial of service via SMS messages. And of course they're closed source as well. And then I don't know how many people heard about the carrier IQ event some time ago. And that was a lot of different phones. I think it was on the iPhone, if I remember right. It was on Android. I don't even know if Windows has enough microchare to actually have been affected by that. But so there's some security issues and privacy issues and all these devices. And Android. I was really excited when I heard about Android. I thought, oh great. So it's Linux based. It's open source. You can download the entire source online. And as we've seen now, there's a huge proliferation of devices all the way from the free things that AT&T gives you up to, you know, $500 tablets and even better. You can write your own applications. Unlike the iPhone, there's no fee to get started on it. You just download the SDK and go from there. You can even hack the OS if you want. It's just Linux with Android, the open source Android on top. And there's a really big community of it. There's a lot of mailing lists. There's a lot of people interested in Android development. So it's very, when I heard about it, I thought, great. This is a really great platform for hacking on. But then I actually started playing with it. And I said, I have to use what language? I can use Java and or C++ with J and I. And I apologize to the Java lovers, but I really want to gouge my eyes out when I use it. And yeah, half of most of my co-workers who are, they're very interested in Android, but they really, they don't know anything about, they don't know Java. There are C programmers, there are C++ programmers, Ruby, Python, all these things. But those aren't really good options. I mean, you can get like Python with scripting language for Android, but it's still kind of a second-class citizen in the Android environment. Besides programming, the vendors don't really have any incentive to keep updating your OS on your old phones. You've bought it. They're not going to get any more money out of you. And they'd rather in a year you go buy one anyway. And then, because we're paranoid people, I mean, how much do you really trust that Joe Random on the XDA forums who just said, here's a, you know, here's a, here's a ROM I just cooked up. Do you really want to trust him? Maybe. He probably didn't put carrier IQ on there. He's probably more trustworthy than AT&T. But, I mean, who knows? Who knows what's in there? And Android, like the other devices, has some security problems. In fact, I think I've seen more issues with Android. Talks here. Stuff on the news. Then with the other phone, phone OSes. Like last year, there was a talk about fake over-the-air updates being pushed via a just a rogue cell tower somebody had in their backpack. I haven't really seen any much confirmation of that. I just saw a post on a mailing list claiming to have done it. But I believe that could be possible. As I mentioned, Android was affected by carrier IQ. There's been a plethora of malicious apps which have kind of showed up. And then from the point of view, you know, you look at the security stuff and you say, well, maybe we should audit it or take a look at it or strip it down to make it a simpler base that we can secure better. But then you realize that there's 15 million lines of code just in Android. This is the stuff that doesn't include the Linux kernel. There's another 15 million lines of code in the Linux kernel. And as we've discovered over the course of this project, I'll talk about it a little bit later. It's just not exceptionally well documented. You got to jump around a lot of places in the source tree. And of course, it's piles and piles of Java which interlink with C++ programs which link back to Java, things like that. It's really quite confusing for one person to all take in and understand or even a small team. It can be a difficult or daunting task. And then my phone, for example, it's a 1.2 gigahertz processor. It's got 512 megs of RAM. It still runs like a dog. And it ran like a dog when I got it. You run a few apps and then pretty quick, your memory's awful. So maybe you don't want to hack on Android as a Java environment. But when we looked at it, we really realized that Android is just this pile of user-level Java and a few C++ demons all running on top of a Linux kernel. And they've got a little busy box environment down there. They've got demons written in C which just run for talking to the radio and things like that. And it's pretty standard down there. There's a few tweaks in the kernel and, of course, they have the hardware support and things like that. But by and large, once you get rid of the Java, you're just back down to Linux. So we said, all right, let's scrape all that off. Kind of like the, I don't know, you don't like mayo on your sandwich, so you just scrape it right off. So we scraped that off and now we've got this Linux platform. And sure it's not driving the screen or anything. We can't get a prompt except through the USB port. But we do have, this means we have a huge variety of hardware sitting there running Linux. It's got support for the radio now. And the drivers have been written. It's just waiting for us to open up and jump in with our own platform. So we thought, okay, what can we do with this Linux platform we've got here? And we looked and some of us at work are a fan of the work of Bell Labs like the Plan 9 and Inferno. And we said Inferno might be a really good fit. So Inferno is this open source operating system written at Bell Labs back in the 90s. It's been maintained since then. Currently it is owned by Vita Nuova, which is a British company. It implements a virtual machine. It's called DIS and it's compiled and interpreted. So Inferno can run natively or hosted on top of Linux, on top of Windows, on top of Mac OS. You can run it natively on hardware directly, but very few people do that because it's so convenient to run it on top of Linux. As it says, it's inspired by Plan 9. It's kind of a sort of an evolution of some of the concepts taken from Plan 9. Sort of with an intention to be used in set top boxes, embedded devices. And this was back in the mid 90s. So it was kind of sort of ahead of its time. Sort of pushed as a competitor to Java as well because it's a virtual machine. The code actually is quite portable unlike Java code, which you might have to tweak to run on Windows and Linux. It's very portable. It's very lightweight. As it says here, it only takes a few megabytes of RAM. I think our environment, you run the Windows system, you run a bunch of demons. You're talking about maybe 16 megabytes of RAM used. And there's about a million lines of code. This includes the kernel, a whole bunch of applications, the windowing system, a bunch of drivers for native, for use on native hardware, which we don't even use. It compiles really fast. You can recompile the kernel in like five seconds on a modern machine. And when you run it, it's started out, it's catching to the windowing system within about a second. So this seems to us as a really ideal thing to put on top of this Linux system that we've got here. So you look at the Android thing. We've got this Linux running on Android. It's very basic, but it does provide drivers for all the hardware, including the binary blobs that the manufacturer shipped for the radio, which would basically be completely out of luck if we didn't have those. So it wasn't really an option to write your own OS from scratch, but building on top of this very thin layer of Linux, it's possible. And the neat thing about running it hosted on Linux is it's actually quite easy to update Inferno, because Inferno just lives in a subdirectory on your memory device. So all we'd have to do to update Inferno would just be to push the latest copy of freshly compiled stuff over to the storage device. So when we were going about this, the first thing we did was, as Shakespeare said, let's kill all the Java. And when you look at it, Android's Linux environment is a little bit different, but when it comes right down to it, you know, it starts the kernel, the kernel loads in a net script and it just goes through the net script processing it. And as we found that every single Java process on the system is launched from one initial process called Zygote. And if you stop, if you stop Zygote from starting in the net script, which is very easy, you just comment it out, then the Java environment doesn't launch and you just have Linux. And so the way to do that, you just take it out of slash init RC, but then you realize slash init, or slash is reset on every boot. They reflash it, they reload it from a, from storage. Well, the way to fix that is either you can build your own custom ROM, which has init.rc the way you want, or you can do what we did, which is write a script, which grabs it currently running ROM, modifies it and then reflashes it. It's very quick, very convenient. Now that we had a Linux system sounds all the Java, we were, we just had to adapt Inferno to actually work on the Android device. There's some hoops you have to jump through in order to actually get it to build, because Inferno's got the, sorry, Android has its own set of libraries like Bionic, which is their libc. And so it's a little bit different from GNU libc, which is what Inferno expects. And there's a script that we found called AGCC, which basically sets up your environment to use the Android GCC compilers and tells it where all the libcs are. So rather than running GCC, you just run AGCC and it wraps it. We found that most of the existing code for Linux in Android was suitable for our users. There were definitely some changes we had to make. I mean, not a lot of people actually run Inferno on ARM, so there were some issues with the scheduling and I think we had to tweak some of the assembly because it didn't work on the specific processor models we were using. And then we had to add support for some of the bits of hardware. We got the frame buffer. I mean, the nice thing about the Android Linux is that it just provides a plain old frame buffer. And I think it's slash dev, slash graphic, slash frame zero. And we had some code left over from a port of Inferno to the OLPC. We simply adapted that. It took some tweaking, but we adapted that to our FR use. And then one of the more complex things was figuring out how to parse the touchscreen inputs. Because it's interesting, they differ from device to device. So you look at slash dev, slash input, slash event something. And which event file it is varies from device to device. But you read it and it gives you a bunch of binary and the format is going to be a little bit different based on which device you're looking at. So basically we just took that and for every device we had code to handle what to parse each touchscreen event and convert it to movements of a mouse, movements of the mouse, clicking of the mouse and things like that. And we also got really sick of dealing with the binary output from these input files. So we just, we wrote code to convert them to text and make that available to all the other Inferno code that saved us a lot of time because then we just dealt with text. Yes? I'm sorry what? Yes. Okay. So I was pointing out that the screen is multi-touch. What we did here, we didn't want to go all the way in and convert it to a full-on multi-touch system. So basically we, let's see if I remember right, there's a way to differ between the first finger that comes down and the second finger that comes down. We just take a single touch out of this touchscreen and treat that as a mouse. For the way that Inferno is built, or the way it comes, that's pretty much useful. That's the way you kind of want it. Although I think there is certainly possibility for adding multi-touch, we didn't, we didn't end up doing that. And then with that done, with the graphic support and the, some of this hardware support done, we then had to hack the window manager to make it actually suitable for use with a phone. Because as I'll show in the next slide, the old Inferno window manager is just not really suitable for using on a phone. We played with it for a little bit, but let's see if I can get my mouse pointer up here. Can you see that? Yeah, okay. So we've got these little tiny buttons up here at the top of the window for managing the windows. We've got this little tiny menu down here for opening up, for opening up programs and things like that. We've got little tiny scroll bars, which we started to do away with, but haven't completely done away with. And the new applications we've written, we have much better scrolling. So, you know, this is something that's very useful if you have a real mouse. But if you're just using a touchscreen, the inputs look like somebody's poking around at the screen with a sausage. It's just not very good. There's no resolution there. So we hacked on a bit and came up with something that's a lot more phone-friendly. We moved the, we moved the, basically the start button up to the top, made it a nice big bar for you to hit. We left the task bar down here at the bottom, but we added a battery indicator. So unlike Android, we actually do, you know, Android, you can run a bunch of programs at once, but it's kind of hard to tell what's still running and all that. We actually explicitly allow you to manage your windows by hand because they'll just show up down here at the bottom. Let's see. So yeah, when you click the thing at the top, you get this much more friendly drop-down menu. A lot of these applications are things that we're already into Inferno. We've just left them like the shell. It's really nice to have a shell, by the way, on your phone, just readily accessible. You can get the same thing in Android, but the Inferno one is quite nice. We, as I'll show in a bit, we wrote a phone dialer and SMS application. It does come with a browser. It's not a great browser, but it does work. We haven't gone through yet and adapted it to actually make it more touch-friendly. So we get little tiny buttons up here and little tiny scroll bars. So yeah, we've got some graphical applications that came with Inferno, which is nice. We have an on-screen keyboard and you access this by pressing the menu key, the menu button on your phone, which toggles the on-screen keyboard. We set up some shortcuts also to manage your windows. If you press the back button, it kills that application. If you press the home button, it minimizes the application down to the little bar down here. We've got shortcuts for setting the screen brightness, things like that, turning the screen on and off. Now, of course, just straight up porting it over. It's porting Inferno over wasn't really enough because it's not really a phone yet. It's just Inferno happening to run on what looks like a PDA now. So we wrote something called dev phone and it speaks to the radio. So on Android, they have something called the RILD, the radio interface layer, I think, demon. And that you connect to via a socket, if I remember right, and you send it a very poorly documented set of commands I think in a binary format. And then it does stuff to the phone for you. So we broke that out. We had one of the guys who did a stellar job at this. He went through all the RILD code and figured out exactly what it expects. And then he wrote dev phone for Inferno, which speaks to RILD for us, so we don't have to deal with it. And what it does is with Inferno, it presents a little file system interface, which we mount at slash phone. And then if you want to make a phone call, you just do what I'm showing here. You can echo slash phone slash phone. And if you want to receive incoming calls, you read from slash phone slash phone. And the read is going to block until a call comes in. And there's a similar interface for SMS and slash phone slash SMS. You just write strings into the SMS file and it sends it out to the appropriate recipient. But of course nobody really wants to make a phone call like that. That would really suck stupid junk every time you want to make a phone call or receive one, have to have the shell open with a cat slash phone slash phone. So we wrote a little application to access the phone to do dialing and an SMS application. We also started working on some Wi-Fi and audio drivers for it. So you can make phone calls. The audio of making a phone call is separate from the audio of playing music, things like that. So that works, but we had not yet in the summer completed and we had to send the interns off and stop working on it. So yeah, we've got a very primitive dialing application. Definitely could be better, but it allows you to do everything that you might want. Make a phone call, receive a phone call, put someone on speakerphone. And we've got an SMS application. We took a very simple approach with this. We just keep the conversation log into a plain text file. And when a message is received, it's put into that text file. The SMS application pulls the modification time on the text file. And if it sees that it's been modified, it re-raids it and displays it here for you. Are we getting ringing through the speakers? I'm certainly hearing it here. Is there? I don't know how to fix that, though. So. Oh, all right. But and this SMS application is really quite easy to write, because all it has to do is just echo or just write strings into the slash phone slash SMS file, and it just takes care of the rest. Oh, and you can see, we started looking at some ideas on how to make applications in Inferno more phone friendly. So we have these nice big scroll bars on the side for your fat fingers, and the text is bigger, and slightly bigger buttons, and things like that. So it's actually not that hard to write a phone friendly application in Inferno. So as I said, we kind of ran out of time. This was a summer project last summer. I forgot to mention that at the beginning. The interns had to leave before we could really get digging into some of the more interesting things we could do with this. But there's a few things we thought of might be interesting to look at. For example, like I said, Inferno runs in about 16 megabytes. It's simple enough, it's small enough that you could run lots and lots and lots of copies of Inferno all at the same time. And so you could just have an instance of the OS per application if you want to try sandboxing them that way, for instance. We also thought since we have such good control, we have total control over this operating system, right? We, it's only about a million lines of code, and that's a lot of that stuff we don't even care about. So each one of us who was working on this pretty much understood how the entire system went together, where the kernel was, what device drivers were being used, all these things. So we had really strong control of the kernel. So we could implement system levels, kind of things like security hacks. Something my boss had tossed at me was the idea, maybe if you think you're going to lose your phone and you don't want somebody else to get access to the data on it, then you can throw it as hard as you can. And when the accelerometer just texts that you've gone over 10 Gs, it just completely wipes the system like that. And something I didn't really mention too much, I did mention that Inferno is based on Plan 9. It's inspired by it in many ways, and it uses the same file system protocol, 9P. And there's really interesting things you can do with 9P because it's when you're working on a system like Inferno because it's network transparent. It doesn't really care where any of these files it's looking at are coming from. So it would be incredibly easy to access the files that you want to store at home. You don't really need Dropbox, for instance. You could easily share files with nearby phones. You would just say export this file tree via, I don't know, maybe write Bluetooth driver, 9P over Bluetooth. You could also use 9P to export your phone's devices very easily. Just say export slash phone slash phone. And then your PC can import those files, mount them, and then with really no extra code, you can make phone calls from your PC controlling the cell phone's radio just from your PC just by echoing stuff into a command file. Anti-theft programs are easy. You don't need all the hassle of there's a variety of these things you've seen, right? Somebody steals your phone, you can access the camera, take pictures of them, figure out where he is. Well, it's really easy when you've got 9P. All you have to do is mount the device, mount the camera device, and then you can just access and just take pictures from home and check out where he is. Besides that, if somebody steals your inferno phone, they're not going to know what the hell. This thing is he'll probably throw it in a ditch activating the 10G security thing. So yeah, the real thing we took away from all this is that it's really easy to strip down inferno and repurpose it for your own whatever you really want to do with it. And I couldn't really find any references to this, but one of my co-workers was talking about was a Boeing project which had just done a very similar thing. Someone else may know more about this, might point it out after we're done, but damn it, we did it first. And so yeah, if you want it on Android, you don't have to put inferno on it, it's quite easy to do whatever you want with it. You get Linux, you can put Python on there, we did put Python on there. As a first-class citizen, you just run Python, and we even played around with some Python graphics libraries which access a frame buffer directly, and we were fooling with writing our own initially before we went with inferno, we were fooling around with writing our own Python environment. You could just whip up your own phone like that. But another thing we took away from it is that actually with a bit of work, inferno could be a pretty viable OS for your smart phone. It's fast, it's light. When we were testing it, when we managed to crash it, which we did a lot because we were playing with the code, we put in a key shortcut on the phone for resetting inferno, restarting just inferno. And you don't even really notice it, except that suddenly all the windows you were running have gone away. It happens in the blink of an eye. It's restarted the phone environment, whereas my Android phone takes 30 seconds a minute to boot up. It's really easy to work on inferno because you can understand the entire thing without too much trouble. We had two interns. We had a high school intern with basically no coding experience. He figured out how to use the programming language limbo, and he was writing good stuff. Within a few weeks, he understood the OS and the programming language was writing good stuff. Inferno already comes with a bunch of software and some infrastructure put in there. You're not starting over from scratch. You don't have to write any device drivers because they're already in Linux. You just take what Linux has given you and abstract it however you like. As I said last, there's no app store, and I don't really trust what's on the app store anyway. If you want to play with it for whatever reason, the code is at BitBucket and you'll probably want to compile it on Linux because that's really the best way to develop for Android and for Inferno. If you want to take a look, go for it. I think I finished quite early. Okay. We have questions. I'm pretty sure the voice input is in the higher levels of Android which I think we killed off. That would probably be a bigger project than the whole OS we just did. I'm sorry, what? Oh, I'm sorry. I thought you were telling me names, but okay, I got it. We didn't get time to implement but the thing is, like I was saying, the nice thing with Linux is that they've already got the drivers implemented so there's these files in slash dev under Linux and all we have to do is write code in Inferno to parse what's coming out of those files and then we can present it in however we want. So it's not much harder than writing a Linux application to access those same resources. Any other questions? The battery life is stellar. It's not really doing anything. Android is constantly checking a whole variety of stuff. Ours just, it just kind of runs forever. I've left it running for a week, picked it up and it's still got good battery life. We're dealing with Nexus S smartphones here for development. If you've got the screen off there's just not a lot of draw there. Any others? Yeah, actually, I haven't tried it with Inferno but the thing is, yeah, you should even be able to export what's on the display, export what's, well, if you're a driver for it, export what's coming in from the camera, export, you could export the touch events coming in on the touch screen. You can export the phone files so you can make phone calls, things like that. It's all really easy to just share all these devices. So Zygote is the first process launched with the Dalvik VM and so it's it's sort of like Java's in it. You launch Zygote which uses the Dalvik and it launches every other Java program on Android. I guess I should have been repeating what questions have been asked before I answered them, I'm sorry. Yes. He was asking, did we change some of the file system permissions to get set up at boot up for Android? We did not. We just left permissions as they were. Was there a specific thing you were thinking about there? I'm sorry, what? Yeah, the file system Android's file system is pretty locked down to begin with. We were working with rooted phones and they were the next SS so we just unlocked them. We installed cyanogen mod but we pretty much built on top of a stock cyanogen mod. I think the only change we had to make was we ran that script which pulled it down, modified init.rc and pushed it back up. Others. The question was how many people were working on this? There were three myself and two interns, a high school and a college student. Can you still get into the Android like the Java environment? Yes, you can. The script we wrote to reflash the device with the customized init script what it actually does instead of calling zygote, it calls a little tiny program we wrote which basically flashes the entire screen and leaves it for five seconds. At that time it will switch, it will start the Android environment and if you leave it alone it will boot into Inferno. We also, it didn't show in there but there's also a menu item in Inferno that lets you switch over to the zygote environment. You can't switch back though because I don't know how to back out of the Java. But yeah, if you want a real environment, not our busted development environment, it's pretty cool. The next question was what made us choose Inferno? Well, we're all kind of Inferno fanboys at work. Those of us who conceived the project and we were looking for something which would run hosted which was important. It's a virtual machine so it's kind of on par with the Java thing. We really like the power of the 9P protocol. The way of controlling things through writing just ASCII text to control files is very convenient for us. It makes it very good for writing applications. We like the programming language. We know personally the people who maintain Inferno right now so we're kind of heading in there. It was really, I think, just a really good fit. We did initially look at, like I mentioned, writing our own Python environment and suddenly have a really good start to an environment already and so we just went with that. Yes. So you're asking about which devices we've had experience with? So we started our work with the Android emulator which ships with the SDK. Then we got, I think the first thing we bought were Nook colors, Barnes & Noble Nook colors. We installed Sanogen mod on there and got ported over to that. We got Nexus S phones and we did it for that too. Those were the only ones we had at work so those are the only ones we ported to but it really, there's not a ton of device specific code for each one. The main thing you're going to be doing is let's see, you'll probably have to make some tweaks for like the frame buffer size and then you're probably going to have to write your own touch screen handling code because the events that come down the pipe are just different for every device. Yes. Well so we did, I did kind of realize that after, when I started talking about the shortcuts when I was writing up my slides and stuff, a lot of Android smart phones are getting rid of all physical buttons. Personally I thought the physical buttons were really great because they just gave us a shortcut to give a command directly to the OS no matter what was on the screen. They're getting rid of them. They're not that bar at the bottom where all the applications go. I just wish that they kept the physical buttons anymore. Am I missing anybody? I think that might be it. Thank you guys.