 Hello everybody, thanks for such a great introduction. Some of these things will actually affect the rest was also same effect So I'm here to talk about Terraform. This is one of Project and one of technology which I was using quite actively for the last couple years First of all, let me talk about who am I and what I'm passionate about when I'm not windsurfing or sailing I actually do coding as well. So AWS Technologies, that's what's driving me crazy for the last five years. I decided to organize user group Meetups in Norway Yeah, several years ago and since then we had lots of people who dive into Amazon web services and try to Use them on a daily basis. So it's going pretty actively in Norway now I'm also one of organizer of DevOps Days Oslo event where I still push people towards DevOps and try to Try to show what is it and try to apply it So today is my first unemployment day because my contract and but I still like to solve problems So I hope that DevOps will be everywhere and eventually I will find jobs so for the last as I said two years I was using Terraform and like I Was technical reviewer on one of not one, but it was the first book about Terraform and I'm very active in Terraform community. So I'm Administrator and maintainer of Probably more than half of Terraform community modules. So I will talk about it later on I'm also seeing Terraform as a big and massive amounts of Technologies and ecosystem and community. So I try to stay active within Open source and being active contributor on different projects. So one of these is Terraform module generator Which I have just released Recently and there will be much more In pipeline so stay tuned. You can find me on LinkedIn and github so also this talk I wrote them here you can see them but Half hour is not enough to talk about Terraform that's certainly I mean It's big and it's interesting technology and it solves lots of problems but I'd like to talk about Terraform as being part of delivery pipeline Sorry that I me maybe Mixing words like CD and CI and continuous delivery continuous deployment Continuous integration and I often call it as CI CD Because it's hard to figure out. Oh, are we talking about CD now or I'll be talking about CD version 2 There are two CDs, right? so Let's get started. So first of all Who of you are actually? Knowing what is let's say infrastructure is called just raise your hands. So one two three four five Oh, actually the full cool. So right place to be so what about deployment pipelines? Almost same amount of people. So yeah, that's good I was afraid that people will be not raising hands and then I have to delete the whole presentation and Talking about basics. Cool. So then what about pipeline as code? Exactly same amount of people because you eventually write this as code, right? So in this talk, there will be two technologies both of them are Created and sports sponsored by Hashi Corp one is Terraform Which is the main one which is tool to write plan and apply Infrastructure changes and another one is pecker pecker is a tool to build automated machine images like AMI in Amazon or virtual box images and so on both of these tools apply Infrastructure as code as core principles as well as few other things like immutability So what we want to do probably deploy infra how we do it You get it right deployment pipeline, but why? so first of all how typical is Continuous integration and continuous delivery pipeline looks like you check in some code or developers check in some code You execute some very easy tests on it like Unit tests you package some artifact. Yeah, you promote it to another Stage or another environment you run more For all tests like functional tests and you verify that yes It makes sense to promote it even further and you pass it through different environments So eventually if you want to do continuous deployment then this code which developers just wrote will land in production That's a end goal. Otherwise. We don't have to write any code at all. So but City pipelines they give you clear Idea about potential bottlenecks which will occur in the runtime or in With these code so as often and as fast you can get these feedback is better So you get more Certainly that code is actually what you want because it passed all these tests and yes, let's release it So this is how typical CI CD pipeline looks like infrastructure changes is If the deployment of new version of application or changing your infrastructure resources They can happen along the way they can happen literally on any of these phase and There are many different ways to To implement them on different stages, but on this talk I will be Talking just about terraform and how it can be used here so in a nutshell Kind of shameless plug for circle CI to zero here, but it can be Jenkins or Travis or anything else This is just my preferred tool of choice, which is free from many use cases and it also allows infrastructure As well as pipeline to be in the code so that this Pipeline is Literally a single YAML file. So as you can see here, I run Some sort of unit tests. I validate infrastructure. I built a my I plan it to prove and Once it's what I want then I applied this into production or into other environment So that's basic things Structuring about Terraform code is something what lots of people don't pay attention to or they think that I Can treat my code Later, I don't care. I mean that's my application code. That's my source I need easy to instance for those who don't know this is The most basic way of giving virtual servers in Amazon web services So they will think about this after they actually write code That's fine for small projects, but as your project grows there will be more need to either Separate your application code from the infrastructure code or make some sort of decisions so It's important to take into account that yes, you will have to make these decisions But at the same time you don't have to over complicate it and create 20 different resources as we can see here You can do this later. So one of common Pattern I would say is it Infrastructure Repository can be split it into several let's say folders. It can be even several repositories. That's not That's not the biggest challenge. What's important is that you are you treat your Your infrastructure and later will see pipeline as code so that everything is documented. Everything is repeatable Most of these things are immutable and so on So to do this As shown on the right side It's pretty good pattern to To structure your code like this so I saw lots of hands of people who know what is infrastructure is called terminology means So here I'm just showing how the basic terraform template looks like so this template is There are basic it just creates bucket and it puts object into this bucket and output As I show on the right column, there are three comments reform in it plan and apply Essentially, this is all what you need to know if you get started So if there are people who are curious after this talk like hey, it sounds cool But I know there is magic souls which you don't show in this presentation like you literally have to Set several environment variables install terraform It's all straightforward and it's all done for you so installation brew install or any other way of installation and Then you plan you see what's going to be changed and apply means you apply this code so this works just like that for Situations where you have one file or small file and no need to think about how to Change this code over time Let's say how to promote this between different environments development stage introduction and so on So let's just get it started and get basic impression most sophisticated examples People tend to use Tools like packer which is building complete image One of good practice is to build Immutable machine images every commit or when your code is merged into let's say Master branch or so or when you are about to perform more complicated test so and Packer and terraform they both follow They are both products from the same company. So it's pretty easy to get started with two of them In particular in this example, I'm showing that configuration file app.json is creating AMI with specific name, which is then discovered by terraform using data data source Called AWS AMI So it was it was probably Not so revolutionary yet. Let's look into FTP. FTP is You all know what is FTP, right? So no FTP is frequent terraform problems That's what kind of new definition of FTP because nobody use FTP anymore. No, but FTP is still there. So one of common problems which I see especially being as a maintainer of terraform community modules. I see That it's very easy to get started with right and terraform code. That's very good Starting point for many people they just get started in one hour and think that they are done But in fact they put problems into the code. So one of this is that They tend to hard code values Just just because it's the easiest way because they go to console and they see this ID and they put it back And that's problem number one So really using data sources or at least TFRs file for discovery These values is very good practice and second as we Go into manage more and more resources into the clouds and create more ephemeral resources over time It's very important to keep track of these resources. So tagging is Significantly a significant step to improve and to make code Easier traceable, I would say so if you have Many resources and you have to put all of this into Into your code yourself. It's tedious work And it's very easy to make errors because you forget to tag something or you forget to use data sources To change values So as a solution to reusability code is terraform modules. So terraform module is essentially the block of It's a logical block of resources which you can Use to increase code reusability and improve code structure We do this a lot when we write code ourselves. So why don't do this for infrastructure? So terraform modules can be versioned and There They can be private or public so you can when you start working on your internal project Only you know what kind of architecture you're going to use. So if it's going to be specific requirements for your services, then you can make it as an individual module and Keep it closed for yourself. You don't have to publish it officially You can what's even more important is that you have to version it and you have to Tell your application team that They have to stick to specific versioning. So semantic versioning is still a pliable for infrastructure. That's So that's a cool part of Infrastructure is called is that lots of things are not new. I mean we will write in code for Not for one year, right? So we will write in it for many many different situations So apply the same for infrastructure is cool and it helps as I said code reusing and encapsulating groups of resources and eventually it will be easier to perform testing of it So I have some demo and I also have internet Let's see if internet is working well So I'd like to demo two things. First of all Do you see this well? Like this, okay So this is how workflow looks like Yeah, so there are several steps and all of them are configured in the code as As I can show That's it. So the pipeline configuration is in single YAML file and it has definition of different jobs where I Just describe what kind of commands I want to run and At the end there is workflow where I title all of them together into chain or it can be parallel execution or it can be sequential execution So at this pipeline I'm Performing some Terraform related things like I'm validating Terraform configurations file on validate infrastructure phase and my Configuration for the Terraform looks looks like Looks like this So here I'm using module security group which is hosted on Terraform community modules Organization and I want to use specific version 0 to 3 and This means that all of my infrastructure is now using just this version and when developers will release new version And they think I should be using so I will change this figure here. That's that's an easy way of Following specific versions and then there is app.json which looks Pretty pretty similar to what I showed on the slides where I create AMI with this name and run several provisioners and That's That's it So as the next step step I run plan infrastructure and I can show output of it So the most interesting step here is Terraform plan me being as Developer I want to know what kind of changes are going to happen. So I see this Pretty nice Terraform plan output where I can see that new AMI is going to be Is going to be used so it forces to create new resource and Since this is exactly what I wanted to do I click approve infrastructure. So this is mental step which comes here and Once I approve it then it applies just this Just this change which I talked about So it destroy previous resource and it launched new one with correct AMI and At the end it outputs this IP address so I can go to this IP if I didn't kill this instance then it should be here. Yes so This means that build number two four eight was a result of pecker build Yeah Think I think that's it. I Want to go back to the presentation and actually Talk about some more thoughts about about this thing before I Can accept questions So let's see So it's loading Okay, that's very promising Can you see this? Yeah, you can see this Let's pretend that I click present and it works So, yeah Terraform is as I said really really really biggest and really growing projects. So there are lots of github tools Created by Pretty cool people and pretty cool companies outside of hush a corp. So it's very easy to get started and people just tend to fill in missing parts So one of this project which People created or at least one guy created and some other contribute was tf lint which allows you to Validate your code and make sure that you put actually same results the same values that like security group ports and IP addresses of vpc and so on so I encourage you to use tf lint as as a starting point to make sure that Since it's the fastest and very easy to To execute you have to execute it as early as possible into your pipeline Also paying attention to codes coding styles and apply terraform fmt is is pretty good Practice as well Just so that the code which is written by you is not different By code written by someone else in your company or even by other People in the community. So that's pretty important as well And Yeah There are several projects which Even on this conference and before when I talk to people and hear them. Yeah, but we can use cloud formation Or we've been using cloud formation for many years. Why should we jump into this one? And we are not going to to jump away from AWS in the near future. So why should we invest time into using this? so Like one of my favorite answer one of my favorite answer which was my favorite Was that hey, you can run terraform plan and see what's going to be changed Now that's not relevant because you can see the same in cloud formation but still my main main Kind of sales pitch why you should be using terraform. Is it it's except It's not forcing you to write some strange code like yaml or like Jason or use another development concepts to generate this code It's asking you to write plain terraform templates, which are very easy to understand They have building validation building formatting and not type checking but They allow to get started much easier And second thing why people prefer to not use or not even look into terraform because it doesn't have graphical user interface Yeah, of course it doesn't have because it's all in the code and the infrastructure is code is actually against you clicking in console and Trying to be smart and Not put everything into the code. So There is tool called atlantis by Hoot suit. So this is one tool which Which is getting popular and it's getting more attention now and it's One of that tool which is missing in tool chain when it comes to orchestration and executing terraform in delivery pipeline So I really encourage you to look into that one terraform itself provides some built-in concepts for working with complex environments. So there is concept of terraform workspaces But I honestly couldn't figure out why it's so important and why it's cool So to be honest, I don't recommend this one It's it's there and it's possible to think about your code as separate environments But it's pretty hard to get started and very easy to break things So if you want to break something then it's very easy to get it's wrong And terragram is another one is a product which started to fulfill missing parts of terraform back in the days and just Just after One year or so they found the specific niche into Making it easier for you to write Code and following don't repeat yourself pattern So that terragram is forcing you or is giving you tools giving you ways to write terraform code which you can apply for different regions for different environments Using different conditions But Yeah It still doesn't Doesn't give you the full power of what programming language can do for you So second not second but one more thing which people often Expect terraform to give to them is that I want to create List of instances in a loop. So I want to have real programming Concepts implemented in terraform. So no, it's not possible to do use a native htl But there are tools like json net or you can use your favorite programming language to generate terraform code and so on so Look into terraform as not Full-featured programming language. That's important to understand. It's tool which allows you to plan apply infrastructure and using community tools you can Easily fulfill the Romanian beats and if you are all into terraform already and wants to promote your Your usage of terraform in the company and See you that it's not so easy going because people are reluctant to accept this change I would suggest to just Read official documentation where they have very nice comparison between AWS and terraform for example, it it's very hard to explain in this talk, but That's it from me. So if you have any questions, I would really really love mc to come Thank you, we have quite a few questions. Um So let's start at the beginning The first one just a simple one someone was asking what the ui was that was in there. I think it was circle ci Yeah, uh, do you want to just tell people a bit about circle ci is for those that haven't seen it before? Yeah, so circle ci is uh SARS solution for continuous integration and continuous deployment so This what I showed was a second version of circle ci which was released I think like couple months ago, maybe three months ago. I don't remember and it allows to put everything into the code and You don't have to manage your own ci system so I see lots of people struggling with traditional DevOps toolset, but Uh It's not for me. So I use something what I don't have to manage Next talk will be about something what we don't have to manage Um, and I guess similar to that is uh, why is tepon better than something like uh salt stack or puppet? Yeah, that's pretty pretty often comparison between these technologies. So, uh Uh How to say so first of all is easy to get started with syntax being able to see what's going to be implemented and large amount of modules and potentially high quality of modules in terraform than in puppet or chef or I don't know about salt stack, but Quality that's something what terraform community is try for So if you are using terraform you potentially should have Uh predictable quality Unfortunately with puppets and on my own experience. That's not the case. So you go to puppet forge and which of Hundreds of I don't know ubuntu modules you have to use It's hard to find the best one with terraform. There are fewer options but uh The biggest difference between these two two or not Two groups of tools like terraform versus everything else is that Everything else was not written for infrastructure in first place. It was written for configuration management and infrastructure management and deployment and doing everything at once and terraform is Just for managing your infrastructure as code in a repeatable way and getting predictable output Someone asked is this an aws only solution or does it work on things like azure or giggle or other clouds? yeah, so terraform doesn't have any affiliation with aws Same as me. I don't I think you mentioned that I work with terraform with aws But in fact I use aws service. I'm not affiliated with aws. So yes terraform can be used to manage Around 50 different types of resources including cloud providers dns management systems Docker or kubernetes and all these kind of platforms are available Once you figure out the basic of uh, how to create terraform code how to structure it effectively You should be able to pick up and apply the same for azure or google cloud The difference of implementation is sometimes significant. So it's it's Usually following the demand that As more people are asking about AWS specific features it gets more attention and it will be implemented faster. So if you have any difficulties go into github Open an issue and it will definitely sparkle some attention some discussion and Potentially your problem will be implemented and released not at the next Major version, but it can be released literally the next day So an aws specific question about that. Have you had any success managing? I am users and I am roles on aws. Yeah. Yeah, so with terraform it's possible to manage I would say 90 percent of resources which you Which you have access through console or through api and I am group policy password policies All these access keys and all security stuff is very well covered there. So it's part of Offering since I don't know at least one and a half year. So it's it's how it's using very actively Okay, a couple more technical questions. Um, where do you keep the Terraform state file? What type of locking do you use? Do you set it up so circle ci can only Have one deployment than at a time questions like that Yeah, starting from the from the last question is actually easier. So at lunch is by hootshoot takes responsibility for executing single Single Single how to say yeah, so executing single copy of it. So if you have multiple pull requests With all of them changing terraform configurations You you really have to merge them one after another And uh, atlantis by hootshoot will prevent you from doing that So atlantis will not allow you to merge all at once and then try to solve the problem So atlantis is one of solution for For that one when it comes to managing state between If you have complex infrastructure You you have to manage state outside of your Laptop and most common choice on the easiest to implement is s3 Where file is uploaded at the end of each execution And it's versioned so you can roll back to previous version. You can see what exactly has changed this period of time Natively, there are no tools which can show you what exactly has changed from this commit to that commit But uh, there are some other tools which you can use Uh, so right now terraform provides lots of Lots of features which you need or may need. I don't think I can imagine what I'm missing right now From terraform Point of view or maybe I know where to look for it So that's that's great. There's a couple more questions in slack that maybe you can answer later that I think we're a bit too technical for now But uh, yeah, if you've got any more questions, I know some people have used this and played and got some questions Please do come and find Anton in outside in one of the breaks and have a chat. Let's give a round of applause to Anton