 Tell us all the things you should never do with terraform. Thank you Okay, now you can hear me Anyway, thank you Walter for the introduction and yes In fact, there will be some things which you should not do But at least you should be aware that they exist. So maybe you have this actually situation when you should be doing it so Couple words. Oh my god. Of course, it never works in this room So this time without clicker again What I do I'm staying through the active in open source community and I'm writing and maintaining a lot of open source tools related to terraform AWS and I became AWS community hero because I actually Do something in the community giving talks and trying to figure out how to make usage of AWS and terraform First question to all of you, please raise your hand if you are using terraform AWS modules Excellent. Oh my god. That's actually much more So it means that some people went to previous year talk as well because in previous year I talked about terraform best practices and how to use it properly and honestly I am tired of using it a correct way. Okay, so this time we will look into how to to not use or how you can use and this should give you some information about possibilities and Yeah, what I do I actually do everything related to terraform consulting and providing different services running workshops Trainings and I also like mentorship. I mean I'm exposed to a lot of different code and solutions Which I try to aggregate and sometimes talk about them You can see my blog where I write exclusively about terraform and AWS on Anton the gunka.com and Yeah, this is stuff which most of you as I understand are using which is pretty cool There were seven million downloads so far and I'm sure that amount will be just increasing because Come on so far just one Thousand pull requests and issues resolved right so we need more issues more pull request more users and more sponsors So there is a link Another project which I'm involved is cloud craft which is Opinionated and pretty good way of drawing AWS diagrams in the browser. I Have 25 minutes and I have 66 slides left. So we have to go fast. Okay What I'm working on is that once you visualize something in the browser You don't have to repeat yourself when you actually implement this as code So modules TF is an open source project, which I work on You can go and use it for free and see what kind of code and structure and the tools are actually used So infrastructure infrastructure is all what is necessary to run A Application it's all kind of computing networking and other requirements which we have to maintain So this is my favorite definition of infrastructure despite the fact that this is not coming from Wikipedia And Terraform I'm not going to ask who use Terraform because I mean otherwise you would go to another talk Plan disclaimer Everything what I'm going to talk will probably Look, oh my god. Why do you even show this? I don't really care because this is disclaimer. Okay, and There are a lot of things a lot of features of Terraform which exist in documentation in users code But it doesn't mean that they are good. For example, Terraform workspaces And there will be some useful parts as well in the talk I put them in the beginning so that you actually get something useful first and then you Figure out that oh my god Terraform can be used as a hammer because everything except I don't know is a nail, right? So let's use Terraform for absolutely crazy situations And then we'll look into how Terraform 012 actually helps and what does it do and how cool is it? So first add of course and let's make Terraform with AWS faster for free with SMS and registrations Right who doesn't like to have it So please use this one You'll be surprised that if you specify in the provider that you can skip credential validation check Which region you are using meta data API check and all of this you will have in most cases you will have at least One no, so you will have one instead of ten API calls and if you are working on slow network Like an air airplane this helps a lot you can work with Terraform offline using this hex and this is document So this is not hack. This is useful. Okay a Little bit more practical stuff versioning and secrets. I mean everybody likes to deal with secrets I know so we'll talk about that but first versioning so versioning in Terraform exists in many places and many flavors We have Terraform core which Obviously has to be versioned 012.sansin providers many from the community now you can get them from the registry and a lot of them are in private because nobody wants to publish them and Also, there are modules which are more Obvious way and more obvious things to be versioned most of them are in registry some of them are private and so on So when we are talking about versioning in Terraform of Terraform core itself There are not many tools which are useful and helpful For example, if you need to run Terraform on your local machine Dfm is one of these tools which you run And switch similar as you do with Ruby and for example If you have to specify version of Terraform as well as version of providers That's really good thing that you are starting to Look at your code or something what will actually leave longer than you quit, right? so you're worried about future and The point of versioning of Terraform providers is that when you specify your dependencies and They will be combined later on and you cannot override them from the parent modules which you use That's why good Terraform modules are never using provider block inside of module itself always instantiated at the highest level and There are many tools which try to do similar stuff for modules Honestly, all of them failed in my opinion even the one which I'm talking about now because it worked on October Now it doesn't work The point is that it is really hard to make versioning correct For example as it says here support ranges for modules from Terraform registry Seems to be the case if the latest digit is less than Then nine ten is not more than nine according to this tool so I was surprised and Overall the flow looks like that is that we first set up version for the Terraform Then we figure out which version of provider of modules We need to do and then we use Terraform in it to download providers Which satisfy modules requirements which we have and then we use Terraform as normally Terraform plan apply and destroy If you are into automation and CI CD things you may look into Dependable which still has ongoing issue related to Terraform 012 syntax From what I understand it will be resolved sooner or later because amount of Disappointed people is growing very fast. You can go there and upvote it yourself Secrets in Terraform, it's still a problem with Terraform 012 not a big surprise Terraform core Has not done everything what is necessary, but let's see what we can do So first of all there are two types of secrets which we're talking about the first one is Inside of plan apply and output and then the second one which is still this No changes at all is Terraform state. So whatever you put will be in state first of all when we specify that we want to Output password we may specify that we want to use sensitive through then it will be written as sensitive Pretty good and pretty helpful sometimes If we want to use actually a random string and we want to generate this random password and then we use it somewhere else Quite recently. There was a new resource which is called random password And the cool part of it is that it does not output ID of the password itself as console output which means that we can run it in CI and It will not be available for anyone else. So check out random password resource And overall the solutions are very specific to the provider and if we are talking about AWS There are a lot of options Related to the provider implementation of how to deal with secrets and how to encrypt them somewhere It is it is pretty big topic in general So if you have some money, let's say from zero to seventy dollars You may get benefit from telephone cloud and you will be able to have secrets managed using telephone cloud and Your state file will be also stored on telephone cloud. So you actually Will not have so much problem with secrets, but this is only if you have some money If you don't have money, you can use to like secret hub I Think it is one of the best tool out there. I'm surprised that not many people are talking about it including Employees, I don't know why but I'm telling you that this is cool. So check it out It has support for TF ours TF state. It's another resource It's not a provider in Terraform, which you have to include and it will be dealing with secrets So secret hub is Documentation So overall versioning is doable secrets still problem secret hub looks good. Okay That was the end of useful information Okay, now let's look about what is actually hammering right. I mean hammer looks cool, right? We we know the Terraform is powerful. Let's use for pretty much anything. What means anything? so let's manage resources, but don't care about state and What does infrastructure glue Terraform mean and how to manage any resource on AWS? Using Terraform and how to do application deployment and how to actually use Dynamic Terraform modules and what does actually she means So let's look in the first problem how to create resources and don't worry about their state if you are coming from infrastructure you like infrastructure management you get used to There is resource. There is configuration or definition There is state and we have to take care about this TF state file They're careful it because we don't want to lose it. It's really a sensitive and it's all these scary things in most cases It's right. I worry about if we want to use Terraform for just right only operations Or for example, we have some tooling which allow us to generate Terraform configuration files and we want to manage data which we populate console key value Or we just want to upload and overwrite any amount of files on S3 or Any other reason that there are a lot of different reasons I'm putting this as an example. Yes, there is a backend type called in mem Unfortunately, I cannot put link to documentation because documentation doesn't exist and nobody wants you to use it. So I Encourage you to check this out. It's really helpful if you are creating something what you don't want to worry about For example, it's your test environment, which you know Certainly will never touch anymore. So that's great example of using in mem. It will make your Work with Terraform faster. Of course, so there is no state. So every time there will be Creation of resources. There is no way to change them. But in many cases it helps Infrastructure glue Terraform. Well Sometime ago we figured out that a lot of APIs exist Terraform has a unified way to manage all of them Like if you can manage resources on your file system to create them to read Create archives or call different scripts see allies rest API make HTTP calls and many many different things And I see not many people are actually taking it Seriously because what what it gives you is you have a way to interact with this API is over this Providers in very standardized way. For example, this code shows that we can connect different type of resources and Create files make an archive for them and upload this archive into S3 bucket. So first of all, we use Features of Terraform 012 to iterate through list and make list of users for example And then we make files in specific location on our file system Then we make an archive from that directory and we upload this archive into S3 bucket So all of this is 24 lines, which of course can be Updated and you can use CLI or AWS CLI or you can use Any other tool or you can write shell script in one line But the point is that of this talk is not to show that hey the same you can do in better way No, Terraform can do this so How to manage any type of AWS resource if you are coming from AWS, you know cloud formation, right? Many people know cloud formation Okay, cloud formation does not support a lot like it doesn't support everything and Terraform does not support everything either, but in total they support pretty much Everything if not everything then you have two options You have to open pull request to AWS provider and then wait and wait and wait Meanwhile, you have to use your own fork And I'm quite sure that there are people who open pull request to Terraform AWS provider and You are still waiting for pull request to be merged, right? Yeah, I can see some faces. So yes, it takes some time. It takes some time and meanwhile you have to use your own fork But even worse situation is when you have some stuff developed in cloud formation because that's defaulting in AWS back then even still now and You want to move to something else, but you was told that all you cannot to delete this database or S3 bucket So what can you do so that you have to use better tools? But you at the same time cannot recreate resources. This is where the requirement is if you could create and delete That's easy so the solution for that one is that you first create cloud formation stack and you specify Deletion policy retain so you have cloud formation stack where deletion policy retain means that this resource will not be deleted After cloud formation stack is deleted. This means that it's not managed by anything else literally Then you delete cloud formation stack and you import this retained resource to Terraform It looks it looks like this is that you have to write code described in these resources like in these cases three buckets and you have to import it and you have to See that plan is happier because you Customize the resource configuration and there is nothing else to change. So what you have now is that? resource is managed using Terraform and it does not have any relation with cloud formation anymore and if there are some situations which you cannot manage Using cloud formation or using Terraform. You can still use no resource and AWS CLI AWS CLI still support significant amount of resources Even registration domains if you want And yeah, of course, it's never too late So application deployment I have a little bit time and I want to ask you what was before I mean on this conference you've heard about immutable infrastructure more than once you all know what is pecker and docker schmocker There are many different tools out there But what was before like who tell like how did you do before all of this? Let's say Maybe in some places you still do it like I did and I still do it for some of projects So what is like any way is how you do this don't mention anything fancy like immutable of course No, no Kubernetes, please no pecker or docker What? Yeah, SCP SCP is pretty cool. Yeah, SCP is supported Quite successfully. That's too boring. I would say What about this one? Terraform and git pool isn't it cool that you can use Terraform to actually use Remote execution to connect to the instant to an instance which is not managed, of course anywhere and you run git pool on that machine Isn't that future? Hopefully not but still you can describe this resource you are kind of one way into to into 21st century and Then you say oh no now I need to redeploy this so you run Terraform tamed you tamed that resource and you run the play Apply again and then you run it again I think it's brilliant because you are almost there you start to understand that oh, yeah There is a gap between this process, but Terraform can bridge them So and one of the most complicated feature or not even feature but Scenario I would say how to use Terraform for things which kind of was not designed for and This is called like in web browser. It's called polyfill Where polyfill is code that implements a feature of web browser that do not support the feature In terms of Terraform We all wanted to have something what Terraform if we go to Terraform open issues and we see that there are hundreds of upvotes or even some Disappointed customers say like oh this feature is not supported. We cannot we cannot use Terraform because it's it's too bad What what can we do for them? For example for each with modules like many people wanted to use it I'm sure because I go to github issues and they see so many upvotes or even more Like historically people wanted to have reusable module, but they wanted to have different Values inside of ignore changes for example an auto scaling group I still have debate between people who say like oh, we don't have to specify a minimum maximum desired capacity as Required values because we wanted to have up and down like scale in and scale out But at the same time other group of people say that this has to be fixed And there is no way for me as a maintainer to specify that ignore changes can be parameterized based on what you need Because inside of life cycle block. I cannot use variables Actually, I cannot use variables in many other places, but in life cycle ignore changes This is one of most obvious second example is prevent destroy inside of life cycle We wanted to have this optional like now. I'm entering pseudo mode. This means that prevent destroy should be false Okay, fine. So this feature is not supported in Terraform. I think it will be supported in the near future to be honest, but We'll see and another example is when we have So-called self-service. I this is a feature which I have hardest time to explain and even figure out this myself to be honest But I want to have module content to be based on the input to the module itself Not just values of the of resources has to be parameterized, but the content of the module itself If we look into this module block where we can see that HTTPS Mitreform.tf is actually a website where the source code of the module is generated based on Results submitted from the form. This means that inside of ZIP archive, which is going to be produced by Mitreform.tf There will be content which is very unique. For example, I can embed some secret stuff based on request I Receive or I can generate infinite amount of combinations. I can use full features of Programming language. I can even use Pulumi to generate Terraform code inside of that HTTP response if I want but the point is that This is a place where we can add So-called sheen to the Terraform if this was not enough Sean from HashiCorp is a good guy. He published workshop puzzles There are I think five of them which I didn't list here You can go to that repository and try to solve them yourself and you will see that Terraform is a pretty cool tool to do Random stuff. When I say random, I actually mean random. For example, you you may Want to orchestrate printers remotely using Terraform. I mean yes, it has HTTP interface so Terraform can talk HTTP. So you see it works And I was told by one guy who actually do this for work because he works as a system administrator remotely and He says that sometimes he has to do like most of his job is with Terraform and proper usage of it But also he has to do some IT stuff without going to to work Which is good So Terraform 012 what does it mean for us? Please raise your hand if you are still using 011 and want to move to 012 like immediately Why this part of the room really? Okay, I'll ask this part of the room. Do you guys know what is 012? Okay, good. So yeah, I'll talk to them No, but What does it mean for us? So there are at least two groups of people Terraform developers and Terraform users and the reason for this separation is quite Strict, I would say is that we don't have to expand the term of full stack developer even more like you knew? HTML Java script PHP now Java or there is DevOps or DevOps as well and Terraform as well That's too much. There will be no quality. So if we would we will be constantly expanding this term That's that's too bad. The quality will be not so good. What I believe into is that there are a group of people Typically, we call them as DevOps engineers. They know AWS. They know provider. They know some Company start them standards. They know security to some degree They know what is encryption and they want to figure out how to connect different components together so the end result from these guys should be reference architecture which they publish and maintain inside of organization and They actually use happily all features of Terraform 012 While Terraform users are on another hand just go into Viki page and see what kind of solutions Do we have in our organization? like how do we deploy this stuff and they pick existing module from the reference architecture published in organization and they use it just to build whatever they need They don't have to know all these features about Terraform Sheen and crazy other things which exist They just need to get their job done and for that. They just need to know basics of HD HCL 2 Which is Terraform 012. They already know how to build stuff and they know the the main area So my last slide here is that Terraform 012 is great and there is no doubt that that's what we need and we need this for for very many reasons, but I don't want To have even more people who are not dealing with Terraform on daily basis To be so excited that yes now I will go to work and now I will make my code work in Terraform 012 and everything will be perfect. No, it will be crap Honestly, it will be crap because you need to understand a lot of internal Implementations like you need to know how to connect these things together so that you can actually live with them when Terraform 013 comes out and Yeah, Terraform is a universal tool. I hope you got this Impression but honestly, please evaluate and pick more carefully than just listening in my talk Thank you No questions One question I'd like to know why you don't like workspaces. Sorry. I'd like to know why you don't like workspaces. It's so loud Yeah It is so loud This is this is for you. Yeah, I knew this is common question which people asked to We've just changed Sorry, we've just we've just refactored our counter to use workspaces. Oh, right