 We have very clear instructions to start exactly on time and we're not doing that, so we're already being bad people Welcome to the configurative also the info management of room We renamed this year because we wanted to make it more inclusive for also things like terraform and kubernetes Our first speaker is Alessandro and he will talk about 10 years of puppet installations. So without further ado Good morning, can you hear me? So I'm going to share some of my experiences in puppet in particular, but I'm surprised to any coefficient management tool after 10 years of Working with it. My name is Alessandro Franceschi. I am working as a CISOP Operations for more than 20 years. I started to work with puppet something like 12 years ago and Since that time my change my life changes because I started to configure and manage my systems in an automated way Now I am CTO in example 42 I am the author of tools like puppy, tiny puppet, tp for short, psik and other some kind of unorthodox puppet projects on social media so you find me as an albagante So who are we? I expect I hope I suppose that we are more or less people who work with IT operations We who are interested in IT automation and in particular Classic configuration management tools like puppets old chef and C-ball and so on What I'm going to say here is based on my experience on puppet but applies I suppose to any tool So if I suppose this is a common concept for everybody here But let's do a quick review with the infrastructure code we mean the code and the data we use to describe the configurations of our infrastructure and Since this is called the way version it We tools like to get and since this is a version code we can test it We can deploy it And apply the changes in our infrastructure according to the changes in our code Let's go quick. So what are the things typical of people working with infrastructure code and tools? So first of all you are working with code you work with the variables Which have to have some values somewhere you have to define what configurations go in what servers in what nodes You have to define also To ship the right code and the right data for your systems When this doesn't happen that the things happen things like you are changing the configuration of System and these are chain configuration is broken And so maybe you restart a service and the service was down or You try to do some changes which are you expect to do in a server and these changes happen somewhere else or Things that you try to do changes and actually this is not a really the important changes So you keep on trying to do the same things These are quite more or less common fear famous which may or may not Drive to big failure. So something bad happening on our infrastructure Depending on the extent of the change effect So we do a change in a little file in some configuration somewhere and these restarts network Interfaces of all infrared infrastructure and maybe with wrong settings. This is a big failure which has bad effects Sometimes it's just a small arrow, which is enough to create a big Problems and most of the times is also due to lack of knowledge. This is quite Obvious if you don't know exactly what you are doing. You do something wrong and this wrong thing as a consequences The good news is that we can prevent a big change bigger failures from happening at least Even if they happen There are some ways to prevent them and to mitigate them preventing and those testing of course testing things before we roll out the changes to production and But the preventing means also peer review for example discussing what is the change that we are going to introduce in infrastructure with our colleagues in order to Prevent or at least have a better Feel better what could happen It's always important when we work with completion management tools to understand what's the extent and the risk of our change So if we change a file for a specific server, we know that That server only that server is applies involved if we change a file in a more general Configurations me may expect that this change applies to all the systems Now how do we realize that this change is going to be destructive and or not We test it and we test it and we should test it in all the different conditions Also, it makes sense since he all the coefficient management as code is based on a sort of centralized the management of the infrastructure This means big power and the big risk we can roll out a change which applies All at once or not the servers and this could be dangerous So maybe for very critical systems we may have Situations where we don't actually apply the changes to for real But we do running so-called not more than poppet reward, but there are similar Concepts in other tools. So basically we may have a service which keep on having some keep on being Configured and managed but that don't have the change that applied Automatically immediately without some kind of human control Again, it makes sense also to change to roll out the changes to our servers in a phased way so that we can catch early Warnings and if something goes reg it goes bad the breaker and interrupted a roll out Also, I think my very personal opinion is always good to have this red button I mean it depends on what tools are you using say in tools like puppet You have a central server and clients which always can keep on connected to the server to get their configuration and apply them locally In such a place the red button could be You stop the server you disable the puppet service on the server. This doesn't affect clients. I mean this doesn't interrupt the ability of clients to keep on Servicing their own applications or whatever, but this prevents them from getting in this case have wrong configurations and applying them So when we work with the tools like these are what are the typical projects and typical things that we have to do Well, the nicest of things is when you have a greenfield implementation new projects to do we start from scratch everything new brand cutting edge and the funny and it is actually the best Situation we can work with because we don't have to deal with existing servers. We have we can have an homogenous setup We can say okay this time we do things in the right way then after one year It will be the usual mess of spaghetti code, but at the beginning it can be made in the Same way When we have to cope and face a project like this my very simple and personal opinion But the common sense I would say is started to do what is more common They so called the baselines of your servers. You have some configurations that probably you apply to all your systems So these are the first things to automate because once you do it once so you do it for all the systems Then you can start to automate your own most common applications. If you are Java shop over you use Tom cut all the time and your applications use Tom cut It makes sense to write your code to manage to cut Tom cut installations It always makes sense to of course prioritize the time you think it may it takes it to do the automation and the benefit Doesn't make sense to automate what we do once you set up a server maybe with a way based installation where you have to Click around you know that this ever is probably going to live for several years Automating the installation and the setup Can be done only when you have already done all the rest So leave this at the end because doesn't make sense to try to automate something which is out to automate maybe and you will automate once When we write and we start to write to our the code of our Infrastructure has to be done. We have to face some decisions some architectural decisions and Here is where we can make the biggest mistakes because if you don't know well enough the tool We can start with some assumptions and some ideas Build all our code base on top of this and then after the end after some month or years We find out that how we organize the things that doesn't fit doesn't work So request a lot of effort is a proof or prune and so on so my very personal opinion in such cases ask for to people who knows more the tool and so that they cannot trace drive you in the right direction and When writing code that to manage Things in our systems. It always makes sense to do it in singles unit You have the puppet terms is a module to manage Apache you have this model which manages only Apache and takes care of managing Apache well And these should be a single unit which does only this specific activity and can be composed According to your needs on your servers with other models so now that to apply and do what you need Here is also useful to keep every single unit separated individual independent so that you don't have a losing Loops and connections between units which can be harder to manage in the long term Now I'm not a big fan of documentation sorry, but I Think that in terms of infrastructure as code the code itself is the documentation of the infrastructure if you can read the code You should be able to Understand what is needed to configure your systems in the way you need so is somehow self Telling the code about how to and documenting how to do things But still what is important is to document the logic the the world Principles behind your code set so that a new person coming in I can from this basic principle have the global idea of how things work and then when it has to Find how to really configure some specific settings. He can go through the data to the code that you have Kind of typical application Projects we have to deal with that which actually is the most common is when we have to migrate from an unmanaged or manually managed infrastructure tour to a Infrastructure managed really tools and like the ones we are talking about So these are kind of my migrations from a brownfield an existing setup to are now a controlled and centralized Infrastructure now this is much worse to deal with because we work with the leading systems where there is production leaving and we have to like Sargery on the real on the leading person So it we have to be much more careful every changes basically is a change on the production So has to be tested has to be applied with a car and so on so compared to a green feed migrate a setup But this is much more Let's say time That's much more time and attention Also the point is we don't need really to automate everything what we have inside I mean if you have very old service, which are going to die whichever are using a very old operating system We don't really have to Need to the migrate them or maybe we can migrate them Migrate in terms of start to manage them with a confession management tool when we have done all the rest Here humble opinion the Pareto rule always apply look for your 80% of the things that is done in the 20% of the time then when you ever made most of the things you can keep on starting Start to do the particular and most specific settings that the net is and so on And always also consider that migration of an existing faster to implies a sort of two axis work You work on single service to migrate according to their operating system or kind of service and so on and the Applications and the configurations you mentioned on each server and you can do this gradually So you can start to migrate or for example start to install your agent of your confession management tool on all the servers And then at the beginning this does nothing and then you gradually start to do more and more things on each Server in a guy in a gradual way when you do these things typically The skeletons from the wardrobe starts to come out. You see comfy you will see understand that your configurations on your servers are much Not diverse are different according they're not organized or more generous and your introduction of a confession management tool Will start to introduce some more genetic We also have to manage Applications our own applications in this case is always better to talk with the domain experts They are developers who write your application discuss with them. What are the interfaces what all the data to configure their application has to be Designed It's important to understand how the things change in your opinion your applications according to different services in production in Development in different data centers What are the parameters which change in the configurations of your application so that you can? Parameterize them in your code so that they can come easy to handle that Also one typical thing is when you have to upgrade your code base This is a very true and has been very true for puppet over the years now is much more stable But there were times where from one major release of puppet to another some code changes had to be introduced because there were The pre-created features and so on and this is important to do it is important to be Possibly on time with a tool, but it's also important to do it in a tested way basically following the same principle of a Migration thing one step by step the most important stuff things first what works keep it If you it doesn't have some added the problems and so on and Basically here is where unit tests are according to me Makes the difference here is where they are really useful because you can test everything before actually doing any change and Verify that your changes not change anything so to say in terms of a refactor in your code Other operations that we do all the day they by day is this kind of operation at the end We find out working with young files most of the times And Here ideally whenever you have a change request so other this user and created is virtual awesome Do this configuration? Either we don't have to write any code because we already have written all the code needed It's just a matter of inserted a right to that in the right place And the nice thing here is it will achieve the situation where we can have the user itself who makes the The request Provide the configuration that he needs because we have expanded them how to do that You have a clear and easy to use interfaces in terms of how that has to be structured and so it does the work for you basically you just Approve They change the request the commit Merge commit and so they did the merge comment or whatever The big idea then is I have a task to do I pick a ticket I start to write my code I test it somewhere I Deploy the code on the server which manage your systems your centralized application management Tool server if any You apply them on your systems, and then you check all these activities can take some time So it makes sense to automate as much as possible Every single step or whatever at least the ones with who are more time-consuming and a reason for to meet I have to go fast because there is not much time Testing everybody the life everybody likes tests nobody likes to write them. I think The point is really this according to me I don't like to write tests the point is let's Test at least the basic essential things the need to to have so you need to Configure an application on system you what is the test? There may be a simple script which just checks that application is running correctly is serving the right thing single tests can be then grouped together and organized in any with any tool with any Monitoring orchestration to do a bigger Test suite But still the point is a single test can be used since the beginning you write the code You write this script that you run it and you are immediately know it And then now you will write use and execute this heat in the future. It's just a matter of choice of tools Let's go on What is important? Basically a tool is not important I talk about popular because I work with puppet but hopefully this applies to any tool and according to me Is the specific implementation of the tool which makes the difference if a successful automation is done or not? and the implementation depends of mostly are matching the tool so what what is the knowledge of the tool so that and You can do better or worse things according to the tool and And Basically if you don't know enough about the tool feel free to ask to someone who knows more Also, it's very critical when you Introduce tools like this to involve people explain to people sometimes in automation infrastructure management means for some People or if it's automated. I don't have to do the work anymore. And so this is still in my job That's not true. The point is automation makes automated. What is a something that the computers can do well and we can have Spare and more time to do things which are more interesting for us to work with and so familiar with a tool makes things easier to handle makes people more happy to work with a tool because they know they understand how he's walking and so they feel more comfortable using it and And and basically is always that more of the people know and more they are effective in things Tools are mature now. I mean it's 12 12 years and working with puppet Compared to 12 years ago. Now puppies are totally different kind of babies that it solves for real a real car and operations needs Managing not only servers a network devices storage networks devices cloud Elements containers related infrastructure may be containing themselves if maybe they are not the right tools for the scope and so on And they are both over the years puppets. It was good in doing managing configurations. Now. It's good also in doing orchestration is puppet was only the agent based the tool now can be used also without agents, so Over the years the tool can evolve and that would in order to do more and more So what's the future now considering that? The future is present. We have really seen it in some way. We are really seeing the container Explosions so we have really seen the new tools which are replacing somehow or can replace the old Classic tools point is anyway that I think we are still having Mainframes around we are still in using comments invented 40 years ago see PLS and so on I think in 10 years 20 years we will keep on adding tools Technology that we have now we still will have the machines physical service and so on maybe more less. Maybe there will be much more orchestrated Containerized applications. Maybe everything will most applications will be moved to the cloud which we don't manage anymore Still, there will be a lot of I think things to manage So the point is in the future. We will have more and more things to manage that's for sure and Someone or something has to do it So I don't know if we will be as a physical person or some here some other thing I more or less Human that does that but in the case since more and more things are going to be Present automation will be unavoidable Last thing I've been a quick faster, but just one last tip which applies to whatever I said up to now Whenever you have to face a problem ask the usual questions. What is the problem? Why I had to do this who is involved there who has to do what? Where has to be done the finger in terms of computers of what files what servers what configurations? when and especially When all these as well questions around that and shared about the people they how the specific implementation is the easiest thing And that's it