 Michael has a, yeah, Chao kind of gave most of the talk away so in fact I can probably just sit down now and you know talking about Terraform and all the other things but so you know we've started this migration to the cloud and thank you so you know we started this journey earlier this year and I'm sure a number of you might be familiar with it you know what we call the GCC around here is really starting to look at providing access to AWS as your and Google cloud and you know the adoption to this in coming on board from you know systems that are you know either legacy or moving into an environment that's not familiar there needs to be a you know a number of different things we need to get in place you know going into the cloud there's a number of concerns that we've seen I mean you know inconsistent deployments are a big issue time consuming to migrate large numbers of instances into the cloud potentially human error that's going in there and you know it changes that these sort of you know may get lost may not get tracked or not getting documented so these are some of the things that we sort of had a number of concerns about as we started this migration and so you know ideally we're looking at automation to kind of help us out you know taking on board the benefits of the cloud you know API driven infrastructure automated pipelines and you know building remote systems leveraging on infrastructure as code so you know what is infrastructure as code just a quick hands how many people are actually familiar with what IAC is okay how many people have absolutely no idea okay how many people are in the wrong talk okay so I mean infrastructure is really sort of codifying the representation of infrastructure and it's something that is really enabled and beneficial for the cloud providers that we're moving into it's not necessarily restricted to the cloud and so you know these sorts of tools describe the desired end state that we're looking at and you know really let us understand the changes between things going into the cloud to under you know get that delta of what's going to be changed and what's going to be updated and so what are the benefits that we're looking at in terms of IAC so you know consistency across multiple environments now we may not necessarily be deploying across all three cloud providers at once but we want to be able to enable at least consistency when we are deploying in there and being able to share a lot of these resources that we're working with it needs to be able to be version controlled being able to be stored in Git or one of the tools that we're you know typically used to for handling changes inspectable and auditable you know being the government we have a lot of strong requirements for both security policy and you know you know being able to actually take these sorts of things and have it visible documented and stored safely and correctly so you know these are the number of benefits that we're looking at to try and enable with IAC and you know for those who are familiar these are a number that are fairly popular IAC tool so you know each of the cloud providers has their own set of tools each is one of them is different Terraform is another one HashiCorp if you notice is coming and speaking at stack next year a bit of a plug if anyone wants to come and join us but there are other things around Pulumi which uses actual source code to be able to deploy these things and Ansible which you know is a tool that's typically seen for configuration management but also stretches its boundaries across into things like infrastructure as code now as Chao Ho had alluded to in the introduction you know we decided upon Terraform you know one of the reasons was that we chose it as a common tool that targets all three cloud providers it uses the same language across all and you know express architectures which will tailored specifically for the benefits of those different cloud providers you know it can target different parts of the cloud providers not just say virtual machines it can handle account management it can handle provisioning it can handle you know all of these sorts of cutting-edge things serverless and containers and all of those cloud native tools that we look at and you know one strong consideration was a number of the teams inside of GovTech and some of the agencies were already familiar with Terraform it works on on-prem and it works on cloud so this is something we can use in existing knowledge and it was very interesting seeing as we we started developing an understanding Terraform is that all of these teams would sort of come out of the woodwork and say yes we've been using Terraform you know can we help contribute so we're built around that sort of level of knowledge there so with infrastructure as code there's an important concept that really needs to come as part and that's our immutable infrastructure so this is an approach for deployment where servers are never modified after they're deployed these sorts of changes are made to a golden image if you want to call it a common base image and then the servers relaunch so you know it takes away the you know that sort of idea of you going in there and manually patching something or deploying a few updates to your check to your servers brings everything back to a repeatable process to be able to relaunch your infrastructure and it changes the model quite significantly so you know it reduces the patch management scope don't need to necessarily go in there and patch manually what you typically do is rebuild the image and redeploy you know these pristine golden images you know if something's going wrong you can replace it without necessarily having to go into the weeds to debug something or you can you know put it aside go and investigate it for potential security vulnerabilities and then have a fresh image put into its place and you know it's also ready for auto scaling so you know if you need to scale up depending on demand for your cloud infrastructure golden images are the way to go and do it so how do we typically do that well we'd use another hashi corp tool called packer and that will let you build these golden images based on a set of scripts deploying a set of common configurations that will be applied and then bundle them up so middleware applications you know operating system hardening all of these sorts of middleware to make sure that we're building images that are compliant and repeatable for these sorts of workloads that we're deploying so i'm actually now going to get off this stage and hand over to my colleague samuel who can go into a little bit more detail of these things oh yeah we're recording this testing 1212 okay we're good okay so as hunter was sharing we've been trying to advocate the use of the use of infrastructure as code and the use of invisible infrastructure we've been going to many agencies we've been going to many guvtech teams and we've also been speaking with the vendors that support these guvtech teams and agencies we've been doing a lot of advocacy on how to take advantage of infrastructure as code how to build in multiple infrastructure how to build automation to spin up one two three different environments with exactly the same architecture the important thing about guvtech architecture is that there's a whole series of security policies that every environment needs to comply with and having that having that expresses infrastructure as code uh and being able to build many consistent environments is is really a time saver you know for my especially for migrating on-premise work workloads onto the cloud this this is a this degree of automation is a great time saver okay now the thing is everyone has a different interpretation of what what their architecture should be uh how the policy should be interpreted specific to their infrastructure so we've been building a community we've been building out the ic working group where guvtech agency staff guvtech staff and agency staff and the vendors have been discussing with us how to build out their architecture using infrastructure as code so we've been discussing the various engineering challenges towards this we've also been trying to do a standardized terraform code we've been taking the inputs of all these lovely security folks and engineering folks and we're trying to distill them into a set of consistent scripts that can be redistributed back to these users you know having a central set of scripts to work off from to start your the development of your custom environment and being to able to reuse that across all your deployments that's that's something you don't have to start from scratch that's that's the that's the point of the standard templates okay so again compliance with guvtech policies and also targeting the various csp's cloud service providers gc on gc like such as adus azio and gsp in progress okay what we hope to accomplish after sharing the benefits of ic and distributing the common templates if we all have standardized architecture we can start practicing this thing called policy as code we have automation deploy architecture we can actually start developing automation to check your architecture every time you make a change to your architecture you can check check for compliance against security policies you can even pre validate your changes before they're deployed to see if they comply with policy so there's this thing called policy as code you can expect your ic or you can expect your the existing environment against your ic to see if there are any security policy that security policy violations okay the whole idea is assurance we want to prevent security incidents as much as possible like i said we can audit existing environments we're looking at such to such a cloud custodian that open policy agent and expect to achieve this degree of automation okay i will also advocating the idea of jid ops who here loves the change adversary board procedure are you familiar with that procedure it's a this great process where everyone gets together in a meeting freezes your code you know do discussion is this going to change your system adversely that is a very lengthy process and it's a very manual process what we like to advocate is move it closer to the developers the ones maintaining the system and the ones building out infrastructure as code so making use of the developers jid platforms such as jitlab and jithub they have the ability to record discussions around pull requests any change to your code you can record discussions around it and these jid platforms can also trigger automation uh specifically jid ops triggers testing automation right testing and planning automation you can run all the automation tests you can security scan build your artifacts and security scan them and in case of ic you can create deployment plans you can create a plan uh and look at it manually and check whether it's going to affect your system adversely that's important all these plans and reports and everything all captured on the jid platform and uh basically using this jid platform you can do all your discussions okay all the results are recorded in the pull request discussion thread uh whoever's it uh responsible for maintaining your jid repository you can trigger the same automation to apply the deployment so all of this is automated and all of this is recorded everything is tracked to your jid platform okay so some of the benefits of jid ops it changes the it moves the change management decision closer to the developers okay all relevant documentation test results discussion and deployment decisions are recorded in the discussion threads and most importantly all versions of infrastructure and its configuration past and present is version in jid that's the uh that's the benefit of using jid offset and ic so this is a quick example jid lab in current uh because we're using terraform there's this thing called Atlantis this software listens to your jid lab for any pull request and any discussion commands that you might have okay uh when you create a pull request you'll do a terraform plan it will Atlantis will update the pull request discussion with what's going to be changed on your infrastructure if i don't know if you can see the text here but basically it's going to say uh i'm going to create one more instance of ec2 instance okay uh when a reviewer says okay that looks okay to me uh runs Atlantis supply okay uh the same Atlantis software runs the deployment and uh creates the applies to change and once the change to apply is recorded into the pull request the pull request also records who who did the apply who who gave the apply command and finally when everything is done we can merge the request yes yeah so that's jid ops uh any questions yep sorry let me grab okay sorry can you thank you uh so you mentioned that this works in uh git lab and git hub yep uh what about uh big pocket yeah yeah yeah any major uh now most of the jid software that we use they all have feature parity they all have roughly the same uh functionality webhooks uh jid ops uh pull request uh discussions so yes uh big bucket is also a viable option for jid ops any other questions yep give me a minute hello testing one two so the details of all the public clouds are very different yes how do you account for the differences in the terraform though are you doing like the most common denominator or you go you got like specific extensions for each uh each public form to be clear about uh terraform uh you have to use the cloud specific primitives so uh what that means is if you're going to write your infrastructure as code for uh i for aws you have to use aws terraform code if you're targeting gcp you have to use gcp terraform code means you cannot deploy that or another you will have to do a porting but basically if you have existing modules and templates you can quickly translate your architecture from uh one cloud provider to another provider we are trying to build the modules so that uh yeah it's done by hand but we're trying to build the module so that it's easier and more standardized uh if it's code right you're familiar that uh you can actually build uh code modules for reuse so you have a very complex architecture that is compliant with uh a graphic security policy uh for aws and gcp the changes in details is what's in the workload what are the images what are the middleware that you're going to install on your infrastructure so you can actually uh take your middleware and apply it to your gcp architecture as long as you have the automation to set up on both gcp and aws it's just porting the workload over at a little high level okay you can talk about this more about this sure of course because what happened uh if you use jamming on the time code aws code deploy right uh they can do that code changes so what you want to do with uh ccd is that you do continuous testing you do continuous deployment your all your artifacts check out correct uh they are green they're okay to deploy now when you have iac and use gdOS with iac you take those build artifacts and apply it to in your ic you take the images built in your in your crcd personally i for our project right we use code for machine as ic okay then we use a code pipeline to set code change right so so what i i what what i don't understand how uh how it how get out come in the future because in a the rest of that they know they don't just need more give give up so you need something new or or and for way for where i i i see right uh because initially when we do the initial deployment right we do iac you measure the server policy those are the changes you the ccd different philosophy yes there is all automation to continuous deploy but why do you want a gated deploy why do you want to say stop this change is not approved how do you stop that you don't approve the pull request you don't uh so with ccd you can simply say i don't merge it into the particular branch that i use for ccd depending on how you define the workload so so in uh in the code pipeline you know you can specify whether you want to be automatically approved or you can go to uh so with bitbucket ccd is considered uh okay it's important as ccd we it's important that you retain ccd for testing your uh code artifacts end to end correct but when it comes to production environment we are proposed proposing jit ops because we want that additional uh capturing of information yes you can use jenkins as an alternative to alantis but i think uh what's the specific difference jit ops is an additional method methodology you uh in addition to your ic testing the telephone plan you also use the discussion the discussion thread as a means to trigger test automation and your deployments so there's additional interaction in your gdops you can possibly build it into your cib ccd pipeline by taking also to do the same job if you look at what you know the the philosophy behind gdops is is it separates the ci and the cd portions so where you typically will have a um where you typically have a single end to end process that says i've run on my test successfully then i'm either going to stop and i'm going to gate it and then i'm going to push it into staging production or whichever environment that you have so that's a single end to end process there might be a stop in there to wait for somebody to manually approve it but the change is actually i guess shifting left to samuel explained it so the philosophy around gdops is that you take um all of those changes the review process that goes in there and you put it into your ci process not your cd process so that once everything is been tested and once the discussion has been had about are these changes potentially going to impact production um and in the example that samuel is with atlantis which is uh terraform specific but in the case of something like cloud formation you can run a test to see how much of an impact your cloud formation changes will have on your infrastructure so that is moved into your cd portion that could be Jenkins it could be code deploy it could be any number of ci tools once that's agreed it gets merged into your git repository then you have your triggered build out of fact that then gets picked up by another tool in the example we had with atlantis it'll get picked up there and deploy it to whichever cloud provider that you have whereas if it's say something like kubernetes kubernetes will have something that runs inside the kubernetes cluster that monitors your git repository and says ah okay this change is approved and this change is merged into production therefore i'm going to pull it across the why so it changes the security model a little bit to make it so that instead of being a push where potentially you could have malware thrown into your environment this way actually looks at it in the reverse and pulls it across and therefore you have the ability to uh detect changes and importantly in the kubernetes example monitor for changes so that if somebody comes into your your environment a hacker for example and makes a change to your containers or your deployments he can pick that up and revert it for you immediately so it changes the security context in that model now git ops is a philosophy so it doesn't sort of specify this fits in with aws or is your or anything else it's a concept that runs across any of these sorts of tools and technologies um it's fairly new so you probably don't see a lot of discussion i know if you're gonna have a look at any of this blog i know it has been a few uh articles posted by various different authors so something that's relatively new but it is showing a lot of benefit to change the deployment model but also change the process for both approvals discussions and uh tracking changes into your environment so it's flexible enough that it'll work over a number of different technologies over time it'll work and it'll fit in with uh a surplus workflow for example so you know you can really look at how does this fit into my particular deployment scenario and the fundamental pieces as Samuel alluded to is that you're shifting everything that everything is stored in your git repository that's your source of truth therefore everything that gets pulled out of your git repository is then deployed to your live infrastructure so it becomes a very very important separation of your ci and your cd and so you know you know the example of having a change advisory board means that it gets moved shifted left if you want to take a kind of hit lingo at the moment of of cicd um and you know let's both the developers have a lot more control but it also lets security teams have a lot more visibility everyone can discuss have a common discussion around this i have that explains a little bit more in terms of your question okay probably that's one more question before we hand over to my good friend my good friend big man um just uh if you precise lines uh took about the template management which is a different application have a different kind of templates have a knowledge my question is how many different types of templates we have and how do you allow your user or your it management team to customize templates that's a really great question so big one will talk about that okay we get a lot of feedback from it a lot of feedback so i'm going to hand over to big one big one has been i will try to share big one has been working really hard he actually goes to the agencies and tells them what to do yeah yeah he goes to the agencies and tells them what to do he also uh coaxes coaxes works with the vendors works with the vendors to uh also achieve IC yeah quite a lot of them oh okay okay okay here you go all right