 Hello, I'm Simon Wilkinson. I'm going to present today a developer's-oriented view of Open AFS. This is the marketing blurb. AFS is a secure, global, federated, location-independent, open-source storage system, providing data access from a broad range of devices scaling from handsets to supercomputers. What that basically means, AFS is a file system like NFS or CIFS, which you may be more familiar with, but designed from the ground up to work on a global scale. A typical AFS user has about 25,000 clients distributed around the globe. There's one user who has about 50 worldwide locations, all of which share the same file system. It provides access for devices ranging from the laptops in front of you to a small PDA right the way up to people of written clients for supercomputers. It was originally developed by Carnegie Mellon University as part of their project, Andrew, hence the name. It's the Andrew Filing System. It was commercialised by Transarc, who were later bought by IBM. That's one of the main reasons why lots of people that I talked to just haven't heard of it. During its commercial life, it was available from IBM for a very large amount of money including a very large per-seat licensing fee. So it was only really available to large organisations. And smaller places got NFS for free in their kernel. So it never really caught on in that kind of divide. IBM saw the light in 2000 and released their AFS source code as an open source project. Shortly afterwards in 2002, the end of life to their commercial client because it was a bit embarrassing for them compared to the open AFS client at that stage. The open AFS client was far more featureful than the IBM client. It's had a strong development community since then, mainly based around the American universities who were its main adopters but also spreading out to be a pretty global community. As I said, it's got the advantage that it's designed to work worldwide. To help with that, it provides client-side caching. Unlike things like CacheFS, which you may have seen in various kernels, it doesn't rely on the files in the file system on the server not changing. You're guaranteed that when the files on the server change, the client cache is notified and updated. You'll never get an out-of-date view of a file in open AFS. It provides something that we term location independence, which means that a user doesn't need to know where on the network their file may be residing. In fact, an administrator can move a file from file server A to file server B while the user is accessing that file and without them ever being aware of it, which is a huge win for sites performing system administration where you can evacuate all of the files from a file server and put them somewhere else without having to notify any of your users that there's going to be a system outage. It's very good across wide area networks and at the end we'll see just how wide those wide area networks can be and works well across Nats. It also supports federated authentication in today's world, particularly in academia where I come from but also in commercial clients. Collaborative projects are very common and it's often the case that you want to give two people at different organisations with completely different authentication systems access to the one file. AFS will support that. Just a quick bit on its architecture. Everybody in AFS will talk about cells. A cell is the organisational unit. Within a cell you have a set of volumes. A volume is typically something like a user's home directory. It's volumes that as I mentioned before about things being moved around between file services, volumes that you would move between a file around. One important thing to realise is that everyone's cell is in the default configuration visible from everyone else's cell. If I have an AFS client on my laptop I can see every AFS file system of every organisation in the world. I can't necessarily read them unless they've decided to give me permission but there is the possibility that Carnegie Mellon University want to give me access to a file and I can then just go in and get it on my laptop without having to install any additional software or perform any additional configuration. Cells are stitched together to provide that functionality through doing DNS look-ups. AFS actually has its own resource record in the DNS called the AFSDB record which is kind of a forerunner of today's SRV records. Technology is defeating me. Ah, there we are. So all of this is stitched together by a very intelligent component on the client called the Clash Manager. So the thing at the top where it says file look-up is what a file in AFS looks like. It's just a standard Unix directory path. With the exception that there's a special bit at the beginning slash AFS which is the AFS mount point. The bit after it is the cell name which is then used to perform a DNS look-up to work out who it should be talking to for that cell. Once it's found the information out of the DNS it goes and talks to what AFS calls database servers. These database servers hold all of the information about how AFS fits together. So in the initial look-up it goes I want to find the root of the infedacock cell. Finds the database server for infedacock and the DNS contacts that database server says which file server owns this particular location. It then goes and talks to the file server, gets that directory back and that continues as you traverse down the path. So it would go get the root of the cell, then it would get user, then it would get s and then finally it would get sxw. All of those AFS operations are performed over what we call RX which is another RPC layer. So the client is made up of two components. There's a kernel module which integrates with the kernel's VFS layer. That kernel module provides session-based security which is another quite important feature that AFS brings to the table. If you have shared users like for instance root you have a demon that's running as root that you want to provide access to a particular area of the file system. You want that area, that to be restricted just to the session that that demon is running, not for all processes that are running as root to have those permissions. So AFS actually allows you to subdivide access permissions down into what it terms sessions. The kernel module is supported by a cache manager which provides intelligent caching. As I said there's guarantees of cache coherence through what AFS terms callbacks. That cache manager will also support prefetching. If you've accessed the first 64K of a file it's quite likely that you're going to access a later 64K of the file. So while you're not doing anything the cache manager will go and ask the file server for that file for you. A quick list of all of the platforms we support. Probably spot your favourite open source or closed source platform in there. The Linux, all of this is supported from a single code base apart from Microsoft Windows which is why it's lurking at the bottom and we're not going to talk about that much. But all of the unixes including Darwin in Mac or Sex are all supported from a single tree that Open AFS maintain. Which means that when we fix a bug or put in a feature improvement for one particular operating system all of them get it. It does have all of the joys that tracking out of tree kernel modules in the Linux ends up having where the API is constantly shifting under your feet new ways of doing things are often only available to GPL modules which is another excitement for us. So that's first of our challenges is maintaining this single source tree. It's all done with Autoconf. Autoconf, we build from every released kernel of Linux from 2.4 right the way up to 2.6.24 we can still build with our current code base working kernel modules for all of those through some very clever use of Autoconf. Another of the challenges it's a very large code base. It's unfortunately also a very old code base. As soon as you ask GCC to provide you with warnings you get a very large number of them. One of the things I'm working on at the moment is just tidying up the code so we can work out what warnings are just because somebody hasn't prototyped a particular function which of them are because we're actually calling something in the wrong way. It's got a very large user... Well, it's got a sizable user community which is generally made up of large enterprise organisations who have very high stability expectations which makes it very challenging to integrate new features into the code base because your stable tree has to be very stable. People tend to be more paranoid with their files than with really any other part of their computing infrastructure. Or however, some low-hanging flutes to find. One of the big problems that we hit is application developers don't really think that much about network file systems when they write their applications. In particular, file browsers which have an approach which goes, let's stat every directory. Then let's look at a file below every directory, KDE's dot directory. That causes a real problem if you have something like slash AFS which has a directory for every single AFS cell in the world which are located at the end of some very slow bits of string from wherever you happen to be sitting. So the AFS cash manager has optimisations to con these file managers into thinking that these files exist when they don't. Only for particular locations that will be slow to access. But we do keep finding these problems generally GUI applications that have probably only really been tested on a machine where the author's home directory is on the local disk. Another classic and I'm actually wearing the t-shirt from the project, Thunderbird had an interesting feature where if a file became unavailable because your access permissions for that file had run out, it would just seg fault. Lots of people don't check whether close succeeds. Actually, if you've got a file system where things happen on a close like AFS, that becomes an issue. As I said, APIs for kernels are constantly changing. Much slower pace of change with things like Open Solaris, much quicker pace of change with Linux. So there's always a need for developers to keep AFS up with the newest versions of every APIs. We're still finding bugs that IBM put in there. Big plans for the future. There's going to be a major overhaul of the cryptography, putting in GSS API-based encryption mechanisms. AFS at the moment uses a UDP-based transport layer. That's going to get thrown out and replaced with TCP. We're finding that most kernels are far better at dealing with TCP applications quickly than they are with UDP ones because that's where all the optimization work's been going. We've got a quite interesting thing called object storage, which is getting the big computing laboratories very excited. And my personal project, which is implementing disconnected operation, so you'll be able to pull the network out the back of your laptop and still read and write to all of the files that are on your laptop, reconnect the network and it'll update all of them back to the server. Developing all of these things does have issues. File systems generally aren't seen as being sexy. It's quite difficult to get people who are interested in working on something that's as low level as a file system. Developing and testing file systems requires quite a lot of infrastructure. You need to be able to bring up a client and a server and with AFS database servers. It's a large, complicated codebase, which can be quite a challenge to pick up. And as I said, stability is very important. You need to make sure that your code works properly before you try and put it in the tree. But that said, AFS, with its relatively small development community, is having a really big impact just to run through these images once a time. This at the top here is BABAR which is an experiment at the Stanford Linear Accelerator Centre which is using AFS for all of its code storage needs. I think they're generating data streams off that collider of petabytes per second. The company over on the left is an American company called Pictage who are a wedding photography company who use AFS for all of their data storage for all of the wedding photos that get uploaded to them. They've got, I think, the figure is around about 500 terabytes worth of storage under AFS management. This little beast next to me here is called K9. It's a rover that's been developed by NASA Ames which uses AFS for image data transmission between the rover and its base station. And over on the right is on the far side is United Airlines who used AFS for their document management system storing all of their flight maintenance manuals within AFS and then distributing them worldwide. They had the situation that if AFS wasn't available on a particular gate that a plane had to fly out of, that plane was not allowed to take off. We also have other organisations such as Morgan Stanley, the investment bank who use AFS for all of their windows needs. If AFS, sorry, all of their UNIX needs if AFS goes down they have no UNIX. If they have no UNIX they have difficulty trading. You can imagine the premium they put on stability. So that's kind of a quick overview of AFS. Hopefully it's got some people interested in taking a look at it and possibly in thinking about contributing code. There's lots of bits that could be done do with being done and more developers are always welcome. Anybody got any questions? Ah!