 Yn ymddych chi'n gwybod, mae'n gofyn, yn cael ei gweithio. Mae'r ffordd yma, mae'n gweithio ar y slot yn ddod ar y ddod, ac mae'n ddod yn bwysigol, ac mae'n wneud i'r ffordd ar y ddod yn gweithio, ac mae'r ddod yn gweithio'r ddod yn ddod, ac mae'n gweithio'r ddod yn ddod. Yn ymddych chi'n gweithio ar y ddod, mae'n gweithio ar y ddod, mae'n gweithio ar y ddod yn 2016, rwy'n gofyn i'n argyflwydu sgwr, ysgol Ffederal Rhigor. Mae'n gweithio ar ein gweithio ar y ddod, Palau Gwlad.gov, ac mae'n gofyn yn芦redig llawer o ddod yn gweithio gofyn a llunio, ac mae'n gweithio y ddod yn gweithio ar y ddod yn llunio, a followersa wasgwyllur i'r edrych i'n gweithio ar y ddod, a'r gweithio ar y ddod yn gweithio ar gweithio ar y ddod. Byddwch chi wedi fylech y troac oherwydd yddych chi'n gweithio, ac whatever is it, it's about making releases boring so that you can do them at any time during normal business hours in a... just at the push of a button. So we can get all kinds of changes into the hands of users safely, quickly, sustainably. Why do we care about doing such a thing? Well the main reason is we want releases to be painless boring events we can do at any time. You want to make it quicker to get functionality to users. We also want to improve quality and stability. Continuous delivery also, it turns out, reduces the ongoing cost of software development. So there's investment to do things like test automation and continuous integration and all these kinds of things. But that cost is amortised over the back end as your product evolves. And then finally, if you're doing it right, continuous delivery will make your customers happy because they're getting the things they want faster. And it will make your employees happier because they're getting to engage with their users and build stuff and get it out more quickly. So who here works in a regulated environment? Okay, lots of you. So you probably had the same experience that I have where you go into a regulated environment and you're like, well, let's do continuous delivery and people are basically like, ha ha ha, that's hilarious. You can't do that here. And so basically, most of my professional life since 2010 has been saying, let's try this and people are going ha ha ha, that wouldn't work here. So I thought, where's the place where I can go, where if we do that, it'll stop people saying that. And it turns out a really good answer to that question is the US federal government. Because it turns out it's extremely hard to get anything done in the US federal government. You know, shipping software is something that we all have to do at some point. I mean, you can only code for so long until you eventually have to release it. US federal government is one of these places where it's just incredibly difficult to do that. Shipping software should not be rocket science. But we can certainly learn lessons from rocket science. Any time you're going to launch something, you need a checklist, right? So before we release, we're going to make sure that a whole bunch of things are in place and working properly before we can launch. So if you go and actually do some research, which is what my boss did, my boss, when I was working in the government, had a spreadsheet where he listed all the different regulations you had to care about if you were going to launch something and how many pages they were. And so it turns out there's a lot of things. There's a bunch of laws and other documents that you have to follow, such as records management and associated bits and pieces. Privacy acts are very important. We want to keep our citizens' data safe. There's the very ironically named paperwork reduction act that we have in the USA. A whole bunch of accessibility standards that are very important. A huge document called the Federal Acquisition Regulation, which governs how the government is allowed to procurement. There's something called the Anti-Deficiency Act, which means they are not allowed to spend money that hasn't been appropriated by Congress, because back in the day, 100-something years ago, what agencies used to do is spend all their money on things and then say, oh, we haven't got any money left. And then Congress would basically appropriate more money because they had to do something for the next six months of the year, so that's not allowed. Can't do that anymore. There's something called the Economy Act. Can't remember what that is. Have an e-government act to make it possible to e-govern. A whole bunch of other things, various provenances. And then there's a bunch of guidelines, as well, that we have to follow agency use of third-party technologies, a whole bunch of other policies that we have to care about. And then we have to follow rules that the White House sets out. The kind of thing that the White House deems that agencies have to do is it's created by the Office of Management and Budget. This is a whole bunch of circulars that they produce. Something very important in the US government is the Federal Information Security and Management Act, which determines the risk of the services that you produce and what the appropriate, and directs the National Institute of Standards and Technology to produce a set of policies that we have to follow, a couple of which are here, the Information Processing Standards, which determine what the risk is of different government information systems and what the various controls that you have to put in place when you launch those are. 140-2, which governs cryptographic modules in government services and hilariously was actually referred to in an episode of Silicon Valley, I nearly fell off my chair when I saw that. I was not expecting that at all. A bunch of special publications that FIPS produces, which list all the various parts of the risk management process and the controls you have to implement. 853-4 is this enormous document which lists all the controls that you have to implement in government information systems. And there's 800-53A, which lists how you test those controls and document the testing of those controls, and so forth. Lots of special publications. What else? I'm trying to move forward from some of these. The hilariously named Einstein system, which is about packet detection and making sure that bad things aren't going through packets, basically preventing attacks by using packet inspection. FedRAM, which allows us to reuse approvals for information systems between agencies and then a bunch more OMB memos that we have to care about that come out of the White House. How many pages roughly do you think we're talking about here? Any guesses? Between 2,000 and 10,000. It's less than 10,000. Are we pleased to hear? 8,000. It's about half that. It's 4,006 pages. This is the complete set of things that you have to care about when you launch a government information system. And not all of it is necessarily very clearly written or unambiguous. So actually working out how to interpret those regulations is also somewhat problematic. I wasn't quite sure how a Jewish reference was going to go down here. Apparently it's okay. 800-53, this is the one I was talking about which lists all the different controls. So these are all the controls that you have to care about implementing everything from, and you can't really read it, but basically everything from making sure you've got fire extinguishers in your data centres and making sure that everything has timestamps to making sure that your telecommunications services are set up properly. And it also lists, based on whether your system is low, medium or high impact, which controls you have to care about. And for each of those controls, there's a sub-control. So AC2 account management, there's things like, do we remove temporary accounts in a particular time? Do we disable inactive accounts? So basically, before you can even think about launching a government service, an information system in the federal government, you have to know which controls you're going to implement, and then you have to implement them, and then you have to test your implementation, and then you have to document the testing of the implementation, and then you create this enormous packet that gets signed off by someone called an authorising official, which is someone in an agency who's authorised to launch it. And the problem is that takes a really long time. We worked out it takes between six to 14 months to go from dev complete to actually being in a position to launch something during which time you're going through this process. So that's very troubling in terms of your ability to implement continuous delivery. You want to implement continuous delivery, every significant change you make is going to take six to 14 months. That's a bit of a problem, particularly because actually what we're finding now is that pretty much any significant system is the target of advanced persistent threats. People are scanning your systems all the time, people are attacking your systems all the time, your system will get broken into. So the question is how quickly can we detect and remediate problems? Prevention isn't going to stop attacks. You've got to be able to remediate really quickly. The first question when the vulnerability is discovered in a library is which of my production systems has its impact and how can I remediate those production systems but the new version of the library without the floor and launch as soon as possible? So we wanted to address this problem. One of the ways that we wanted to address it was by building a platform. It turns out that most of these controls are actually implemented at the ops level rather than the dev level. There's a lot of those controls that are about how you run your data centre and those are actually things you can build into cloud platforms. In fact, this thing called FedRAMP that I mentioned earlier, what that allows you to do, according to the way FISMA is written, every agency that wants to use, for example Gmail, has to independently test and approve Gmail. There's a lot of agencies in the US federal government, like a lot. If the Justice Ministry wants to use Gmail, they have to go through this whole six to 14 month process and then say people at GSA where I worked want to use Gmail, they have to do it all again. Now that's clearly absurd. What was invented was this thing called FedRAMP which allows you to basically reuse other people's authorisations. Or in fact, FedRAMP will do what's called a pre-approval. They'll go through the whole process for you and have it signed off by the CIOs of the Department of Defence, Department of Homeland Security and GSA, and then other agencies are like, well that seems reasonable, we'll just say, well they've already done it, we're just going to approve it. That's really good. It allows you to only have to approve a particular cloud-based system once. And you can do this for infrastructure providers as well. So there are FedRAMP approvals for AWS, for Azure, for Google Clouds, Salesforce and so on. And what they do is they say, well we've shown that this set of controls are implemented in a way that's satisfactory. And so if you're building on top of that, you only have to worry about the residual controls, the ones that we haven't already shown that this service implements. And that makes things a lot easier. So if you're building on Amazon or any one of these or Google Cloud or any of these platforms, you only have to take care of the residual controls. We worked out that actually you could implement a lot of these controls also at the platform let. So by building a platform that took care of a whole bunch of controls, what that meant is if you launched a new application on the platform, you only have to take care of those controls that aren't implemented by the platform and the infrastructure. So FedRAMP website, we built this thing called Cloud.gov, which took about a year to build and then about another year to go through the process to get the provisional approval. And it's actually in use by a whole bunch of different agencies right now. So it's built on AWS, that takes care of a bunch of controls, and then Cloud.gov takes care of another bunch of controls for you. And the coolest thing about Cloud.gov is the fact that the domain Cloud.gov was available. We were extremely excited when we actually saw that Cloud.gov was available and we bought it straight away. And it takes care of the 325 security controls that you have to implement for a moderate impact information system, 269 are handled by Cloud.gov. And apps that you deploy on top of Cloud.gov, we only have to worry about between 50 and 56 of the residual controls depending on what exactly it is that you're building. So that's huge. Now, instead of having to document and test and document the testing of 325 security controls, you only have to care about between 50 and 56. That's a massive accelerator and your ability to take stuff and put it like. So when people talk about platform as a service, often they talk about it as a way to deliver applications faster, but for us it was actually about risk management and about building secure, robust systems and about our ability to meet our compliance concerns. And that was actually the primary driver. And that was really interesting for me because I never thought of a platform as a service as a way of actually meeting compliance and risk management concerns. So that was really great. The whole inspiration behind Cloud.gov really was Heroku. And my first experience with this kind of technology was Cloud Foundry. So Cloud.gov is Cloud Foundry running on Amazon Web Services. Cloud Foundry is designed to be Heroku that you can run in your own infrastructure basically. And my first experience running that was taking a little app that I'd written and deploying it by typing CF deploy and it just running in production. And I'm like, that's amazing. Just being able to take code on your laptop and type CF deploy and then have it run as a one liner. Just completely blew my mind. But now there's a whole generation of people who are growing up who have only experienced that way of deploying things. I once was trying to, someone who was new, and I just got a role as a DevOps person. And I said, what does DevOps mean to you? And she said, it's all the things I never had to worry about when I was using Heroku, which I thought was a great definition of DevOps. But the great thing about a platform as a service is you can just deploy into a production-like environment from day one. So if you saw Dave Farley's talk earlier, he was talking a lot about production-like environments. Well, if you've got a platform as a service, you've got a production-like environment that you can just deploy to and develop on right from the beginning. It's amazing. The reason that a lot of people don't like platforms as a service is that they constrain what you can do. There are certain things that platforms as a service make it hard for you to do. It turns out that many of those limitations are actually good limitations when you're building distributed systems. So, for example, in a platform as a service you can't actually write data to the file system. That's not something that you're normally allowed to do. That's because it's a terrible idea and you shouldn't do it when you're building distributed systems. So people complain about the limitations of them. I think in many cases those limitations are actually good things and they'll stop you from shooting yourself in the foot and making poor decisions that actually won't be very good things to do when you're building distributed systems. It means that you can do push-button deployments. Enterprises who are adopting continuous delivery often ask, should we have a centralized toolchain or should we let teams use whatever toolchain they want? When you have a platform as a service there's a really nice boundary which is CF deploy or whatever command you use to deploy your software. So what we did in the GSA General Services Administration which is where I was working was we let teams use whatever CI technology they wanted. They could use CircleCI or Travis or whatever, but there was this really nice boundary which was that at some point in your deployment pipeline you'd press a button and it would run CF deploy with the credentials that were required to deploy and off it would go into production. So that nice boundary is a really nice way to think of what's the responsibility of a central team that's managing the platform versus what are we going to let individual teams do and what flexibility and autonomy we're going to give them in terms of their ability to build and deploy stuff. The most important thing obviously is that most of the controls are taken care of at the platform level. And then finally, what we were able to do is create templates for all the compliance documentation that you had to have in order to put a government information system live. So you wanted to deploy something on cloud.gov we could give you a bunch of templates all those in, that's your compliance work done. And as a result of this we could get systems from DevComplete to live in a matter of weeks, which was a huge improvement over the many months that it took to take information systems live if you're not deploying on a platform as a service. So platform as a service is a very powerful way of accelerating your ability to meet your compliance goals. So what are the principles for building such a thing? Number one, you want everything to be self-service. I talked about this this morning briefly but I just do want to reiterate this because this is one of the things that I so commonly see people getting wrong when they're using or when they're deploying an enterprise cloud. So quick show of hands. Who here is working at an organisation where you have a cloud that you deploy to? Keep your hands up, keep your hands up. So if you have to raise a ticket or send an email in order to get the resources you need if you can't just self-service them through an API put your hands down, who's still got their hands up? Anyone? Like maybe four of you. So NIST, the National Institute of Standards and Technology in their definition of cloud computing which is NIST special publication 800-145 which is I think NIST's shortest publication. It's like five pages which is just amazing. They list these five characteristics of a cloud. Number one is on-demand self-service. A consumer can unilaterally provision computing capabilities such as server time and network storage as needed automatically without requiring human interaction with each service provider. And right there, most enterprise clouds implementations I've seen like don't meet that. So you haven't got a cloud. What you've done is you've replaced your data centre with something else and not changed any of your processes and thus the outcome in terms of your development process have not changed at all. If I'm a developer I still have to raise a ticket and wait for someone in order to get the resources I need. That's indistinguishable from having a data centre. So what have we actually done to improve our ability to deliver software? Practically nothing. So there may be other benefits, but time to market is not one of them. That's usually problematic because the whole point of a cloud is to enable things like faster development. What we found when we surveyed people last year in the state of DevOps report is that only 22% of teams who thought they were doing cloud actually met the five criteria set out by NIST. So this is hugely problematic. One of the things that a well-designed platform as a service does is it has a nice simple ACL model that enables you to deploy stuff to the paths without interfering with other people's stuff. This is a huge problem if you've deployed to infrastructure as a service. How do you manage multi-tenancy? How do you manage a situation where there's multiple services or teams deploying to your infrastructure? So this is something that we face a lot. Actually when I joined ATNF, which is the team I was working on in the GSA, we basically had a single Amazon account and we just gave a whole bunch of people administrative access to the Amazon account and let them go nuts, which is what developers want. Developers basically want to have root access to their infrastructure account and to go mad. So we let them do that. We're like, okay, let's try that out. So one year later, when I joined the GSA, I was put in charge of this cloud environment or this cloud account and I logged in and I literally nearly threw up in my mouth because it was just full of just incredible amounts of crap. And I had no idea what I could safely delete. There's just huge amounts of boxes and databases and load balancers and security groups and very few people had bothered to label any of that stuff. So it's like, well, can I delete this load balancers? Will that do nothing or will it bring downvote.gov? That's the kind of question that I don't want to have to answer for every single configuration in my infrastructure account. That's terrifying. So that's a huge problem. How do we deal with multi-tenancy? What you need to be able to do basically is garbage collect. You need to be able to garbage collect in your infrastructure as a service account and know that I can delete these things safely because guess what, you're going to be paying for it all. Actually, in the government, we're not allowed to pay for things that haven't been funded with appropriated funds. You're potentially breaking the law even. So, you know, if you have this kind of one infrastructure account that you put everything in all these different services, what you're going to find is there's going to be very large maintenance costs to maintaining that. The reaction that most people have is, well, we're not going to give developers access anymore because developers are just going to create a bunch of crap in here and then we're going to have to clear up after them. Then the moment you put a barrier in front of it, you don't have a cloud anymore because you're asking people to send emails or raise tickets to create resources and then we don't meet the definition of a cloud. So, you end up with all this sprawl. That's really problematic. The other thing you can do is create multiple accounts. So, multiple infrastructure as a service accounts, one for every service, and then you can give a team their own account and give them complete control over it, but then what do we do about shared services? You don't want to have to access 20 different logging services to work out what's happening across your whole enterprise service portfolio. So, there's a whole bunch of things that you want to be shared across all your products and services like logging and monitoring. You don't want to have to create and configure those independently for every single account. So, that's a problem. You also don't want different teams doing things in different ways. It becomes very hard to... As we do in the government all the time, you have a vendor build something and hand it over to you and then another vendor builds something and hands it over to you and they're built in two completely different ways. That, again, is a huge ongoing maintenance cost. So, that's problematic. So, this is where a platform as a service can really help you. You also would need to ACO or you would have to go through that whole risk management process for every single service if you deploy to infrastructure as a service, whether that's for one account for everything or one account per service. So, platform as a service, you only need to get that authority to operate, go through that risk management process once for the platform. It also deals with role-based access control really, really well. So, this is the thing that I really love about Cloud Foundry. It has this amazing role-based access control model where you create an organisation and then I can give the keys to that organisation to a team building a service and then they can create spaces and apps within that organisation and deploy stuff to those services and apps and that doesn't interfere with anyone else's stuff. But then, if an app they're building, they want to end of life it, they can just destroy the app and it garbage collects all the associated services, like the database or any authorisation stuff they've built or the TLS endpoints or whatever, it'll garbage collect everything. So, what that means is A, the different tenants can't interfere with each other's stuff, but also we have a great solution for clearing things up and making sure that people aren't getting bills for things that they don't need anymore or that we have those things that we don't need anymore still lying around in our account. So, the role-based access control is fabulous. I can't overemphasise the benefits of that. Again, platform as a service imposes a lot of limitations. In many cases, those are good limitations that you actually want to have in place for distributed systems. Significantly reduces the maintenance and operational costs. I spent much less time worrying about how to maintain applications over time with a platform as a service. It was great. The only thing I cared about was where did this code to run that was used to build this service come from? So, where in GitHub does this source code come from? Once I had that, I could, if I needed to patch something, I could just patch it in version control and then press a button to deploy the new version. The maintenance was super easy for me. It was really great. The third principle for building a platform as a service that I think is really important is to use native cloud primitives. What I mean by this is a lot of people move to the clouds and then they start buying commercial firewalls and deploying them to the cloud because we've been using Cisco in our data centres, so we're going to use Cisco in the cloud. Don't do that. Use security groups or whatever the equivalent is in the cloud provider you're using. It's much cheaper and there's a much lower maintenance cost over time. People roll their own databases instead of using the database that the cloud provider provides. That's a terrible idea as well. Don't install MySQL to your infrastructure account. Just use whatever native databases the cloud provider provides because it will substantially reduce your ongoing maintenance cost. People worry a lot about vendor locking with clouds and having multi-cloud provider solutions. I actually think in most cases that's just not a real worry. All the cloud providers... No cloud provider I know has ever increased costs. They all drive down costs all the time. The cost of actually doing multi-cloud is extremely high and it incurs a substantially higher ongoing maintenance cost. Just use whatever primitives the cloud provider provides to maintain those over time and patch them and do all those things that you care about. Sorry about that. The next thing is everything must be reproducible from version control. What that means is not just the apps that you build but also your platform as a service. The code for that must be reproducible from version control as well. Obviously you're keeping all your application code in version control so you can redeploy your apps, but we should be able to redeploy our platform as a service from version control as well. All the codes for cloud.gov is in version control. It's in GitHub. You can go to github.com. You can run your own cloud.gov. You can actually just go and install your own cloud.gov. Finally, and for us the really big thing was making sure you can take care of compliance at the platform layer. This for me, for a really long time, I was a grumpy old man about containers and I thought it was massively over-hyped. Then what I realised is in cloud.gov you can run virus scanners and intrusion detection software on your hosts and then deploy your apps on top of that in containers and then app developers don't have to worry about installing Nagios and intrusion detection and virus scanning and all that stuff. That completely changed the game for me. I'm like, yes, containers are amazing. I love them. That's fabulous because you can put all that stuff in at the platform layer and just deploy your apps on top of it. It takes care of so much stuff. The only other thing I want to talk about is the importance of managed services. When you're deploying to a cloud, you have to have things provided for you like databases, for example. Cloud Foundry and other platforms as a service make this super easy. What you can do with Cloud Foundry in particular is it provides extension points which allow you to just self-service a database instance or some other external service for your app with a one-liner. We actually built a plug-in so that you could do TLS termination. What it would do is you would self-service TLS termination of your app and it would spin up an instance of Cloud Front to do CDN and it would self-service a SSL certificate using Let's Encrypt and deploy it and it would rotate that certificate every three months so you never had to care about your SSL certificates again. Given that one of my biggest jobs or, I guess, most failure-prone jobs was making sure that everything had valid SSL certificates, you do not want to go to your federal government service and find that the SSL certificate isn't valid anymore. It's a really bad luck if you're a government for your SSL, for your services to have expired SSL certificates. It's a real pain in the ass keeping track of them all and people would use different providers. Having a service which would just automatically rotate those SSL certificates every three months, that was amazing. It was really, really huge. Being able to have these managed services that are associated with apps and that have the same lifecycle as the app so if I kill a particular app, it automatically garbage collects all the managed services and that's huge. I think the important thing also to point out here is the amount of stuff you have to build that's not just containers. A lot of people think that building a platform as a service is just Kubernetes. That's not the case. There's a huge amount of other stuff that you have to build alongside that. The container scheduler is just one piece. Having log aggregation, making those logs searchable in a multi-tenant way, having alerting, having managed services, routing, all these kinds of things. There's a whole bunch of other stuff that you have to worry about in addition to your container scheduler. So don't think you can just install Kubernetes and have a platform as a service. You have to care about all this other stuff as well. If you want to know more about what we did, we wrote up a white paper. You can go to devop-research.com and there's a white paper on what we did and the principles that I talked about today and a lot more length. Hopefully, if you're building cloud infrastructure in a highly-regulated environment, that will be useful to you because it's got a lot of the lessons that we learned doing that in the federal government. Next time you meet someone and they're like, oh, that sounds great, but it won't work here. Well, we can do it in the US federal government. We can do it anywhere. That's my main lesson. If you haven't already, please take the State of DevOps survey this year. That's open for the next month or so. It takes about 25 minutes. Really appreciate your time. It helps us learn what works and what doesn't work and how to get better as an industry. So please do that and get all your friends to do it if you possibly can. We have a bunch of time for questions. One thing I wanted to understand was what challenges did you face from federal government or a pushback given that it's a complete mindset shift, right? Losing control, everything to cloud now. So what challenges or pushbacks did the team face while trying to implement this? There's a huge amount of fear, uncertainty and doubt when moving to the clouds. There's obvious things like the best practices are very different. The way you need to work is different. The processes change. There's just a lot of... You've just got to learn a whole bunch of new stuff when you're deploying to a cloud infrastructure. Fundamentally, a cloud infrastructure, the biggest paradigm shift is this idea of disposable infrastructure and disposable resources. The idea that rather than having servers that sit around for a really long time and have really high uptime when we care about uptime, you don't care about that at all in the cloud. You should be able to just kill something and have it instantly replaced in a way that's completely invisible to users. So that's just one example of a paradigm shift that goes all the way from how app developers think about stuff and the idea of disposability of resources and being able to autoscale and being able to... One of the things that's really cool about it is if we found a library with a vulnerability, and this actually happens pretty soon after we had Cloud.gov working, it was really simple to patch it. If it was on one of the stacks that was used, you could just push out a new version of the stack with a one-liner with a fixed library and it was fixed straight away. That really helped people understand the benefit that came along with that. So people think disposable resources, that's a complete paradigm shift when we're like, okay, but we can roll out a new version of a vulnerable library so the entire application, our entire set of services in less than 24 hours from when the new version is available, people are like, oh, that's really great, and it was. That's like a really huge benefit. The really weird one we faced was procurement. So in the federal government everything has to be procured or the easiest way to procure things is fixed cost. Because they used to buying like fighter planes and printer toner and stuff like that where you pay a certain amount and you get something in return. Now obviously, Cloud doesn't work like that. You pay for what you use and it's not just that you pay for what you use, you're also paying based on the end you use the usage of the service as well. So there's this double source of variability. Like, you know, I might build something and scale it up and scale it down and that might incur costs, but then if I get 10 times the end user usage of it, that's going to increase the cost as well. So procurement people really hated that because of the unpredictability. And again, the Anti-Deficiency Act means if you spend more than you're supposed to spend you have to write a letter to the president and my boss once had to do that because we exceeded our expected costs on the Cloud. So the procurement people were very, very nervous about that and didn't like it at all. I actually had to make this very complicated spreadsheet where we listed our predicted usage of every single Amazon service and then I did this thing where it's like well, what's the standard deviation and then adding a crazy spreadsheet and at the end of it I'm like, well, I've basically added this huge buffer for the expected costs to take care of potential variability and what that meant is we were spending much or we appropriated much more money than we could possibly have needed to take care of these extreme end cases but our assumptions were totally invalid because we used a normal distribution to predict what the usage would be and of course that's not how Cloud works. It's basically more like a power law curve which you can't model statistically very easily. So it's all bullshit basically but you've got to do it so you can get the procurement through. So that was really hard and I think what you've got to do in a regulated environment is have cost control so basically say well you know and again a platform as a service will let you do this very easily and say well you're only going to be able to use this many resources and if you go above that we'll just limit your ability to use further resources so you can manage the costs. So that was definitely the weirdest one but a lot of it was about education and about training and about having people incrementally move to the cloud rather than trying to move everyone all at once just to get rid of a lot of the nervousness about actually moving to this completely new paradigm. Yeah. Yeah. Thank you. Hi. Hi Jess. Thank you for wonderful session and providing cloud details about the insights on cloud deployment and delivery. My question is, as of now in our organization we are using cloud foundry along with some different half pipe and mechanisms. Would you like to give us more insight about the common platform which we can utilize for different languages like Java, .NET and the most the problem area is the SQL databases, Oracle database has to be under again a cloud and all. So would you like to specify one common platform where we can have it? Yeah. So Heroku has this concept and a build pack is basically like a stack. So you can see here this middle layer is the application stack. So we provided and basically what that is is that's like a container image which has like a whole bunch of stuff. So we had one for Java which had like some version of the JPM and some associated bits and pieces. There was one for .NET. There was one for PHP. There was one for Node. So there's a whole, I think there was like seven standards build packs that we provided. And so your first stop was to use those. And if you used one of the supported build packs again that we took care of a lot more controls for you because we configured those build packs in such a way that they met the controls. If you wanted to roll your own thing we would let you do that but then you had to take care of a whole bunch more controls because you were basically responsible for making sure that stuff was patched up to date. Whereas for the build packs that we provided, if a CVE came out we would say okay well what build packs does that impact? Okay now we'll push out a new build pack with the fixed version of that library or middleware piece and again you can redeploy all the applications there completely transparently with zero down time with the new version of the build pack which again it's just amazing, amazingly powerful facility of a platform as a service is this ability to transparently upgrade your middleware with no impact on users, it's amazing. So that's what we did for that. For databases we provided access to RDS basically, Amazon RDS via managed services so you could self service certain pre-configured RDS instances we provided I think Postgres and MySQL but RDS and the equivalent in the other cloud providers that you can get pretty much anything you want through those so that's what we used for databases was what the native platform provided with a thin wrapper so that you could access it as a managed service from your app and then it would provide the credentials to connect via environment variables to the apps. Over here. So the question is do you face data retention constraints for government services? Yes, that was one of the things that the application developers had to determine.