 Right, so hi. I'm Angus. I should be clear up front. I work for Google, but not on anything at all to do with Android. So for the purposes of this talk, don't mention Google on this talk. That'll only confuse things. I'm just a regular guy who happens to have some Android phones and like hacking on Android in my spare time. So I've got a bunch of my phones here. So this one, I got a G1 two years ago for free. That's always nice. And started looking at it. It was quite interesting. And I wanted to learn more about it. So I decided I'd see how it worked under the hood. And I looked at how the C code ran. The nice thing about Android is you get the full source code for it. So I looked at that and saw how they did things and learned from that. And thought I'd put a program to it. So I'm just waiting for this to start up here. It doesn't really matter. You can't see it anyway. Oh, is there? I might pull out one of my other phones in. Anyway, so I put scum VM to Android. And this is a better one. This is where it runs. Here we go. Start. Let me choose start. Go. There we go. There's even sound. So scum VM is quite a large C++ program. Luckily, there you go. You'll recognize it in true if you've played data tentacle. It is quite a large C++ program. It's luckily designed for portability. It's already been ported to many, many platforms. So it's got quite good abstractions in there already in the code. So it was quite an easy project to port. It really wasn't that hard. And I suspect, I don't know for sure, but I strongly suspect it was probably the first native app on the Android market because I did it before there was an NDK and there were no rules and it was all terrible and I've had to reimplement it since now that there are rules. But I learned a lot along the way. So that's very interesting. So I thought I'd give a talk here about for people who know how to write programs for Linux and kind of the delta that you need to know to be able to write a program for Android. It's a little bit of a scatter talk. I'll just be lightly covering a whole lot of issues that could be entire talks in themselves. So if there are questions, shout out and I can stop and talk more about any particular area. So in case you didn't know, Android is this OS. It's on some phones. It's a little different to a normal Linux desktop in that it assumes a single user. There's no idea of user log-ins and multiple users. It assumes touchscreens, that sort of user interface. It doesn't have double clicking and tabs and those sorts of desktop gooey idioms. It runs on top of the Linux kernel, which is nice. You get an awful lot of functionality, you know, M-Map, file API, networking, all this sort of threading functionality of a modern kernel. It uses a new libc called Bionic, which is based on one of the BSD libcs, I think. It's fairly simple and small. It was designed for... It was designed to keep GPL out of user space because that was scaring some of the phone manufacturers. It is small, very small runtime overhead, unlike glibc, which is quite large. It doesn't have things you don't care about on the phone, like locale support, wide characters. It has p-threads and threading from the beginning. There's none of this need to use the underscore r versions and things of various functions that they're all provided by default in Bionic. But it's still quite a small and simple libc, so some of the fancy features you might expect in glibc aren't there. And most apps, all apps, as far as Android is concerned, are Java apps or whatever the right legal definition is. I don't know what words I should be using in this conversation. And they run on top of a Dalvik virtual machine. So a few other weird differences. It doesn't follow the file system hierarchy standard. Most of the OS is in slash system, which leads to a few amusing things like system bnsh. This is done mostly to make upgrading the OS a bit easier. They just basically... The whole slash system tree is read only. And when you do a firmware upgrade, they just replace that whole tree with a new version. There's some interesting kernel additions. One of them is the out-of-memory handler. No apps in Android actually exit. They start up, they run, they hang around. Threads come and go in those processes. They just hang around. When it runs out of memory on Android, there's out-of-memory custom out-of-memory handler that goes up into a user space process. And it says, right, out of memory, we have to kill something. And there's a defined list of priorities. The thing the user is currently looking at, the foreground application, is the very last thing to be killed. And, you know, idle processes that aren't actually part of any running functionality at the moment are killed first and, as you'd expect, background tasks are killed and then sort of... What are they called? Foreground services. Things like your music player, which might be in the background, but it's something that the user is very much aware is still running, will be killed later, and then eventually the actual foreground app is killed as the last choice. There's a rather extensive IPC patch called iBinder, which is rather black magic to me. I really don't know much about it, but there's a talk later on today by Benno, which I'm hoping to learn more about it. And there's a few other patches, a few other kernel additions like wake locks and things, which changes the power savings, just the way Android does power saving. It's take a lock to keep the screen awake, release the lock when you're prepared for the screen to go back to sleep, those sorts of things. Also differently to a normal Linux system, Android tries to sandbox applications from each other. It assumes applications are fairly malicious and evil and need to be contained. So it gives each application its own user ID. And this is where most of the Android sandboxing comes from. So instead of protecting users from each other as a normal Linux multi-user system is, it's protecting apps from each other, but using the same mechanism. So then there's a bunch of permissions, which I'll talk a little bit about later, and some of these are implemented through normal Linux group IDs. So to be able to write to the SD card, you have to be in a particular group ID and that's implemented just as you would on a normal Linux system. If you declare that you want to be able to write to the SD card, then you get granted that group ID when the package is installed. You'll put in that group. And each application gets its own directory in this slash data slash data and the name of the package. And there's a few common directories in there, like a slash cache, where when you run out of space, there's a cleaner that runs through and just deletes everything from every app slash cache directory. And there's a slash lib where libraries automatically get unpacked into native libraries and a few other ones. But otherwise, that's your application's private directory and it can write whatever it wants in there. And other applications can't even read that, again, via normal fast system permissions. So it's a little different to a regular Linux system. And also, of course, there's no X Windows, which makes it different again to the Amigo platforms you've just seen. The way Android drawing works is most applications render screens and they pass them off to a surface flinger service, which just takes the screens and overlays them with alpha blending and various other effects and shows them to the user. So it's very, very simple compared to X Windows. It's reasonably efficient and clever. There's kind of a double buffering thing where you are rendering one frame while the surface flinger is using the other frame and then you switch. And I don't really know the details of how GL works, but I presume it bypasses some of that to use the hardware acceleration. No X Windows is a point. Permissions. So Android apps declare upfront what permissions is. If you've ever used Android, you've seen this. They declare, I want to be able to write to their SD card, I want to be able to read your contacts. And then you agree to that when you say, yes, okay, install the application. There's no interactive prompting or this application wants to be able to reformat your hard disk. Is that okay? There's none of that sort of stuff. It's all declared upfront. The important thing about this talk is that these are implemented in the OS. They're not implemented in the Java libraries or in the sort of some intermediate layer. So when we're writing native applications, we're bound by the exact same permissions. You can't do anything extra by writing a C program compared to what you could do in the Java land. You have the exact same constraints put around your entire process. People often think otherwise. There's a fairly constant stream of questions on the Android and Decay mailing lists about how I want to write in native code so that I can do something outrageous that a normal app is not allowed to do. It doesn't happen that way. In particular, you don't get root as an application. So you can't reconfigure networking. You can't do any of those things that require root privileges on a traditional kernel. Right. So Android really wants to think that everything is Java. So each application has these entry points. There's no main function that just gets invoked. When you're writing a normal Android application, you implement a Java class that inherits from some of these other ones depending on what exactly you want. Activities are the most common one. And there's various callbacks in there. There's on start, on stop, on touch event. And they get called when various things happen. And you put some code in between those parts, and that's your application. When we're writing a native application, we still have to let Android think that's what's going on. So I'll get a little bit later. But they're the common classes. So you have an activity which is one per screen. Typically, if your application has multiple screens, you have multiple activities, and you'll move between them quite frequently. Services for background tasks. And there's an idea of foreground services, which are the ones that have a little colored icon up the top. And they're things like music players. The little colored icon up the top, the user is well aware that the service is running. And then there's background services, which are for things like sinking your contacts. They just happen in the background. If they happen to get out of memory kill, the user won't notice. You'll try again later, a little while later. And then there's a mechanism called Intense, which are used frequently all through Android. It's sort of a way of bundling up an event. You have a URI, a URL, describing some sort of data, and it's quite common to make up a URL scheme. You'll have some, you know, I don't do this, but I could have a scumvm URL scheme with scumvm colon slash slash the name of a save game or something like that. And then I would fire off that intent. And there'd be a program you can register that you want to receive certain patterns, certain filters of Intense. So you can say, I want to get all of the scumvm colon Intense. And then I'm going to understand them. I'm going to say, well, this means I should start up scumvm with the save game already loaded and start running. Right? The advantage of doing that is that someone else could write, it's a little bit silly in the scumvm case, but in the photo case, you know, I want to take a photo. Someone else could write a different photo taking program. And they could register the same intent filters. And then when your application says, I'd like a photo, fire off that intent. It could run the normal photo app. It could run their custom photo app. It could present a menu to the user saying, there are several choices here. Which one would you like to do? And this is how all of that indirection happens in Android. It's done through Intense. One of the cases where this is used is broadcast receivers. So lots of events that happen on the phone. The power has been plugged in. The SD card has been removed. There's an incoming phone call. All these sorts of events. Fire off Intense. So you can create a broadcast receiver and catch those broadcast intents in order to do things. That's just another common entry point for Android applications. Again, for our purposes in native code, we have to worry about things like processes a little bit more. You have a process. And then each of these activities and broadcast receivers are usually new threads that are created within that process by the Android framework, by the common Java classes. They're all forked off a common parent, but that's for implementation detail. It doesn't matter too much. Really, though, you have one process per application. That's usually the way things work. And if you started forking, exacting other processes, it would work, but you start to confuse a few things like the out-of-memory handler and stuff. It doesn't quite know what to do. That's a rather unexpected thing to do on Android. The general thing is once your activity has... Once on pause has been called, that means you're no longer the foreground application. So the user has pressed back or chosen some other application, or you've declared that the user is finished, they've pressed a done button or something. Your on pause will be called. And then the other callbacks on stop, on destroy may not be killed. If the out-of-memory handler comes in, you could just be killed at any point. So the usual thing to do is to save all of your data. Do whatever you're going to do that commits those changes in your on pause handler. So if you're writing a game or something, you want to save your files at that point. Don't rely on anything after that. So tools for developing, there's two. There's an SDK, the standard Android SDK, which has all the Java environment, lots and lots of documentation, examples, how to add Android apps. For our purposes as well, you also want the NDK, the native developer kit, which has GCC tool chains, GDB server. You'll want both of these, and you'll probably want to be quite familiar with the SDK and how to write a normal Java application before you try and write a native application. So here's the way it works. We have to pretend we're a Java app. So Java has something called JNI, which Android supports quite well, and most of the framework classes are actually implemented using JNI. They're implemented in C++ with a very thin Java wrapper for the compatibility that the public API is Java, but the implementation is almost always C++. So we're going to compile our native library as a JNI library, which is just a shared library, libsomething.so. We install it into the APK. The Android packages are just zip files. So you can install it using the zip command, or you can use the APK builder tool that comes with the SDK. It has to go in this directory inside the APK, lib slash the ABI, which I'll mention in a few slides, and then your library. And when you install that package, those libraries get automatically extracted from the APK and put into data slash data package name slash lib, and they go in there. And so that's in the path. So when you call system load library from Java, it knows to look in that directory to find the libraries, and it'll load it up, deal open it. Everything's happy. And then we invoke them from Java, and we're done. So the runtime to provide it is very minimal. You get libc, bionic, libm. You get zlib for compression. You get a very tiny SDK C++, although with the latest SDK release five, there's a much greater improved one. You can use STL port, or even try your luck with libstd C++ if that works for you. Before then, the provided C++ standard library was very minimal. It was really just stub holders. You also get liblog, which is for printf debugging. So there's a bunch of log macros there, which are right to the Android logs, and that's been my most useful and reliable way of debugging Android programs. The GDB is a little quirky and a little bit hard to get going, so you only use that in extreme cases. You get OpenGL, which is the only real way to draw from native code. And in Gingerbread, which is the most recent release that's now on Nexus S's, only came out whatever, a few months ago, there's a much greater set of libraries for Android applications, but there... I haven't looked at it yet because not on our phones. The problem with writing applications is if you want lots of users, you need to look at the trailing edge of the versions, whereas the Android team guys get all excited about their new features, and they're doing the deleting edge, and you have to worry about the trailing edge of compatibility. So yeah, you have to statically link everything, because if you want libpng, if you want ffmpeg codecs, if you want any of these things, you'll need to compile the library yourself, you need to statically link it into yours in order to get that extra code. There's a very, very minimal runtime environment. You'll find a lot of instructions out there on the internet talking about using various other libraries. If you see anything mentioned, libcutils, or a few other libraries, these are internal Android libraries. They're the ones I used when I did my first port of SCUMVM when there was no official NDK and there were no rules about what was actually the compatible ABI. You don't want to use any of those. They do change subtly in ways that will crash your program in the most minor of point releases on a phone. An example is the Xperia X10. They did a security update, which for some reason involved changing the audio libraries. So SCUMVM before worked just fine. SCUMVM afterwards. There was no sound. The sound initialization failed. It could have been much worse. It could have crashed. So you really, really can't use those internal libraries. They'll change at any point in ways that will hurt you. Defined ABIs. The most common one. That's the usual, that's what all these phones run. The newer phones have hardware floating point, which is V7A. Even the hardware floating point, we still pass, when calling functions, we still pass the use software floating point registers so that it's compatible with the pure software floating point code. So it's not quite as fast as it might be. You can't assume extra things like NEON. NEON is the kind of MMX instructions. You can't assume that. The advice is to test for it at runtime and provide fallback libraries and switch to this version of your library or that version of your library, depending on what you discover is available in the processor, which is a little awkward. And just an example, you need these sorts of sets of fakes. You learn your tool chain quite well. If you use the standard NDK, there's a set of, there's a compile environment in there, which does all this work for you. So it's definitely the way to go if you're starting new. But if you're trying to port an existing code base, like I was, and so you can't use this new make file system, you have to use the existing one. You start to learn all about these flags and what's required. The good news is a lot of this is in GCC45 now. So you can just grab GCC45, compile for that target platform, and if you use minus M Android explicitly or implicitly by using that target triple, then you get most of those flags, I reply it, and that does most of the hard work for you. You need, yeah, minus LGCC placement gets a little bit hairy. You want to avoid, you want to use the libGCC that came with your tool chain, and not whatever might be on your particular Android platform you're using. So you want to be a little bit careful about linking in libGCC explicitly in your line, whatever that says. I can explain much more about it if someone cares. I just wanted to cover quickly JNI. If you're new to JNI, it's not very hard to learn. There's lots of documentation around. Briefly, there's two styles. There's one where you have long symbols like this, and the Java virtual machine will look up, it'll convert the Java method, and it'll look up that symbol in your library and go, that's the one I meant to invoke and do that, and you can see that the top there, there's the Java function, and at the bottom there's C++ implementation of that Java function. The other style is where you explicitly register them. So you say you have a JNI on-load function that's called as soon as you load your library, and in that function you explicitly call register natives, which says this function goes with this symbol, this function is called without symbol, this function has symbol, and there's pluses and minuses with both approaches. The Android guys prefer this latter one, but there's certainly reasons why you might not want that. A common mistake which I should mention is C++ mangling. These symbols need to be nice and visible to DLSIM, and if you're compiling C++ program you tend to end up with extra mangling in there depending on the prototype, which makes it very hard to see. So X-turn C is the answer to you, the question you haven't asked yet, which fixes that up. And that's pretty much it. There's a bunch of debugging tools available. ADB shell gets you a shell. ADB is a tool that comes with the SDK, you plug a USB cable in, ADB gives you commands on the phone so you can install things over the USB cable, you can get a shell. ADB log catch shows you the system log, these are the ones you'll use a lot, and you can use GDB server via the Indike GDB script, which runs GDB server on the phone, and then GDB on your laptop, and you can look at symbols just as if it was a normal embedded thing. That's what I got. So I've very lightly touched on a whole lot of things. If you have questions come and ask me I can talk about these things for quite a while. Each one of those slides could be a half hour talk going all the way down into the details. Yes, you would have to have, so the library goes into your application specific directory, so you would have to explicitly go and look for the library from another program. You wouldn't just find it by accident, and you have to be able to read it. It has normal file permissions limiting it to your user ID. So if you can do things like have a family of packages and if they're signed by the same developer key, you can make sure they all get the same user ID. There's some extra things you can declare in the package metadata, and then you could go and find them and you could load someone else's libraries up. It's rather complicated. I should be repeating questions, you know. Yes, so this comment was there's a copy of the library inside the compressed APK, and there's a copy of the library uncompressed onto the file system. Yes, that's correct. And if you have large libraries, for example, scumvm, this becomes quite a problem. So I ended up doing a whole custom unpacker and complicated things and breaking scumvm up into separate plug-in packages so you only have to install just the ones you want. They have to be M-mapped, so you could possibly store them uncompressed in the APK and then do some hairy M-map with an offset. No, flash is getting bigger. I don't care. Yeah, go back. Yes, you can, but it's... So you put the... Can you target multiple platforms with one package? Yes, you can, but you have to put multiple libraries in the same package. So you have lib slash x86, my library, lib slash army ABI, my library, which can get very large very quickly. So it's obviously not the right thing to do going forward. Luckily, at the moment, there's only ARM ABI. There are no non-ARM Android devices. Let's face it. So the only reason you'd want to do it now is if you had a hell-optimized codec and you really wanted the hardware floating point, but you had to fall back to non-hardware floating point for older devices, and then typically you could isolate just your bit of code and it would be a smaller portion, a small string library, codec library. It's not great at the moment. I expect that to improve. Sometime they'll have to, if they go to more architectures, but at the moment it's that fairly clumsy mechanism. EasyNDK, just see your C++. The NDK is a GCC toolchain, which has GCC and G++. You can compile anything. It could be Objective-C. It could be Fortran, so long as you can produce a JNI on-load symbol in your library. The JNI functions as have C and C++ bindings, and they're similar but slightly different between the two calling styles. Yeah, Go program would be interesting, and quite possible. You just have to JNI bindings, and then the rest would just happen. Yeah, last question. Have I considered clang on LVM? I haven't done anything about it personally. This day would certainly be possible to use. You'd have to add the few, you'd have to add what the minus M Android does to those toolchains, which wouldn't be hard. There's nothing really specific in here. It's using the normal Linux, GNU, ABI, target. So you could certainly do that. You might need a few more command line flags, but it's just native code. Yeah, no worries.