 So, oh, there goes our seat belt. So there's a few things that we'd like to present today, but more than anything else, this is going to be a technical presentation on the technology that we're using to support federation and also the technology that we're using to support desktops and identity. But before we do that, we just wanted to define the characterization virtual laboratory for those who don't have a sterile background. Fundamentally, it is a program of work for characterization virtual laboratory, which is a program of work to connect Australian characterization instruments with data management environments, tools and analysis platforms of Australian research in general. The achievements of the program today are that we've connected over 100 Australian instruments, many of them by interest, of a significant value to Australian researchers, both financial and in the data that they've produced. And we've had over 2,700 users of IPHCPL or the software that we've developed on IPHCPL, because a lot of the software that we developed is pre-used across HPC facilities in Australian international. From the researcher's point of view, it's really just a place where a researcher can access their characterization data and many of the softwares and tools that they might need to process and analyze them. The C Devil itself is doing a number of things. It's connecting instruments, it's continuing to create rich online environments, and it's undertaking a training and fair data program as well. The part that we're going to talk about specifically today is the rich online environments, and it's really just one segment of what we do, but we can't necessarily present the entire technology stack of all the program. And one of the things that we're focused on is the federation of the characterization BL across multiple nodes. So the characterization BL desktop is the desktop environment that we provide to our research user community. Largely, it's been run across two nodes, and it's insomniatic cloud, and it's still so massive. And we're now federating that out so that it's run across a significantly larger cohort of nodes. And so the nodes that we're developing and deploying under this sort of prototype fashion at UWA on the pausy research firm at UQ as well, and a prototype in Sydney of a Windows CDL instance, and also deploying some software at University of Wollongong, which is part of the CDL as well. It's important to actually understand that many of the instances here are actually quite different, and different partners in the project require different things. So we have gone through a bit of a process to try to work out what federation and what CDL nodes across Australia might look like. As an example, UQ wants to integrate with an existing partner from computer, whereas UWA, the instance there, will be an instance very similar to what we run at Monash, and it will run basically from source. They're very different instances. But what we have done is we've developed a set of federation principles that are consistent across the entire network of CDL nodes, or that will be consistent across the entire network. And basically this is what it means to be a CDL node. It's a single user portal, a single signup and registration mechanism through AAF, a single help desk, a single shared software step in as much as possible, considering the technical and licensing challenges, and a single desktop user interface and UI branding menus, background, software module, these sort of things. In a sense, what we're saying is the consistent user experience across all of the deployments that we're undertaking. There's a number of things that aren't in scope just yet, such as the ability to capture data at a number of instruments that's inconsistent, and it probably should be because all instruments are different. And so there's different technology stacks that we're deploying, and it's largely these two architectures here, one is to deploy from source on OpenStack using really our scripts, our Ansible scripts that we use to deploy Gloucester and Crystal talk about that. And the other one is to integrate with an existing high-performance computing system and basically provide a new layer over the top which makes that system consistent with the CDF model. I won't go into this in too much detail because we don't have a lot of time, but those are the two models, the two technical models that we're deploying. From there, over to Crystal. All right. So please stop me if I get too excited. I'm actually quite pleased with the technology stack we've developed for doing all this stuff, because for me it's actually really simple, basic bits of technology, but we've hung together to make something really useful. So as I'm presenting this, you might find that I'm a little bit disabove as not much to see here, but on the other hand, it's actually a really cool way these bits hang together. Now as Wojtek mentioned, part of what we've got is a technology stack deploying what amounts to a cluster, and it might not be a really high-performance cluster, but it's a cluster nonetheless. You have a head node, you have a set of batch nodes that you can share or two, you have a shared file system. And we've placed all of this around OpenStack that's free and that did a wonderful job of putting it in front of us. Ansible and good. Now I'm going to talk about that first and a little bit after that I'll talk about the desktop delivery technology stuff that actually faces the user. Now Ansible, if you don't know, it's a configuration management tool, much like Puppet and Chef, but it's in Python. You define your configuration and you let Ansible get it there. So you define the end state that you want to be in. We end up using this for almost everything. You can see that Ansible is a lovely example here of probing all the nodes in a class to find out the uptime. So it actually replaces tools like PDSH as well if you wanted to. And it's done as bricks. It's basically wrapping SSH. Anything you can do with SSH, you can do with Ansible, but it lays on a bit of Python glue and it means you can write it out writing a shell script for it. You can write a YAML file defining what you want. It's got loads of support around the world so you can hang together ideas from other people. Now Solve would have been just as good. Honestly, there's nothing special about Ansible. Move along. If you want to learn Ansible, I think that's great. And I think it's one of those things that you learn by doing the last one. The other really important part is Git. Now we've got a slightly weird Git setup with our definition of a cluster because we've actually found that there's loads of bits that are common to every cluster I've ever touched because I think they're a good idea. And I think most people are familiar with these. Most people know that if you're going to run a cluster, you're probably going to put an LDAP on it. If you're going to run a cluster, you're probably going to make an NSF line to connect to a user local to get your software off. So we've tried to separate all of our Ansible components into stuff which can get reused and stuff which is the discrete variables which vary between the clusters and most importantly, passwords. As a result, we've got effectively two Git repositories, one of which we're completely open and anyone can go and have a look at. The other which has all our passwords in it and we would be horrified if they'd ever got out to anybody else. And we actually have a different password-based repository, a different repository for each of the clusters we run. Now we do use Git sub-modules and this gets confusing as heck for most people because Git defines this as a mechanism for bringing in dependencies. So it's actually really useful to say we've got this set of roles that are common things we do on all of our clusters and each cluster depends on that and it allows you to do this sort of QA where the upstream roles and most recent version might change but you might not have QA values being ready for your cluster. But for anybody who's used Git, they know that Git's confusing enough on its own and now I'm telling you that you've got a second Git commit in there and you have to make sure which one you're committing an update and to keep everything in set. So it takes a little bit of a learning curve even for highly technical people to get over it but it is the exact solution that you want for this scenario where you've got a shared piece of code and a non-shared piece of code. A really important component is also how we make changes. We follow a very standard software developer principle in that one person is going to actually do the work and submit a request and another person is going to review it. At that point we base where a little bit real easy in that anyone can go ahead and deploy it. We don't have a big CI button for our clusters because it turns out it's really hard to figure out what on Earth a unit test is for a compute service. But at that point we can go ahead and break it. And we have one rule. If you make a mistake that the users notice, you bring in a cake to apologise for your mistake. And I actually have a little bit of documentation up there at isocake.redocs. I encourage everyone to support this as a principle and self-improvement. It really leads to trying hard not to break things when you know the penalty is apologising to everyone by making it. So that's how we build our clusters really is that the other really interesting bit of in my opinion of what we do is this trick of federating across Linux-based post-existence. For anyone who's been around the traps for a few years will know that there's been multiple attempts within the Australian community to say, let's all have one link to username. Let's have one password even. So we're all about Rajan accounts and the same as our Pausier accounts and the same as our Massive accounts. You can log in anywhere you don't have to remember about both passwords. This is really nice. It would be lovely to be able to authenticate once and access more clusters or even just have the identical authentication keys. But it turns out nobody wants to really sign up on this. There's a lot of technical work necessary to make that happen. So we actually decided to work around it. Now we have gone... I've got the point here that our clusters we try to give them identical environments to try to make sure our software is in store everywhere and one of the things we're leveraging for that at the moment is singularity containers. We're finding this is a really awesome technology that will effectively non-root privileges on the same things which Docker does. The nuts and bolts end for our Federation. When you access a desktop through the web browser what happens is the application which is all actually this tests Java and AngularJS the application goes and asks for access to a cluster. You've been logged in by AAF and because all our clusters do have different usernames we've configured so the AAF service has to look up your username for a given cluster. You then in your web browser still provide an OAuth2 permission to allow the screwed-up web web service to access the cluster on your behalf. And that web service never sees your password for the cluster and only gets a short bit of credential which is going to expire after a while. So this is my way of saying really if you can trust this guy you haven't told him she can't script and I'm going to lose access in an hour This is really one of the ways we're trying to get buy-in from everyone for a really low barrier to entry from the security perspective to say look it's a secure service you don't have to do a full year-long order to figure out this is going to be okay. We really felt that was important for getting buy-in for building up the Federation is to allow each cluster to keep control of their usernames and groups and to guarantee that just because you signed up to the CDL and we've perhaps given you resources on M3 to be able to do some basic CDL work you don't necessarily have resources to do work in any other part of the country on any other cluster that's up to you talking to those cluster administrators to ask for resources. Now in the future we might get enough people buy-in to be able to say yes you signed up to CDL you are clearly an Australian characterization researcher to get to you anyway and our next generation of Federation I'm actually thinking about developing this right now we're actually only to have our web application asked for access to all of the clusters at once and we'll poll around and see which clusters you're able to access. The result of this is hopefully that you will have a little option to copy between Massive and CDL at EWA or copy between Massive and Rajan there won't be any extra passwords to enter into it we've already got a short list of credentials we'll just make that happen for you so that's our technology staff Thanks sir Thank you