 Welcome. We'll get started. So this is, how are the people, how using containers can make your life easier? But this is not a technical talk. So if you're looking for details, gory details of actually using containers and Docker and all those things, we're not going into that right now. So if you are expecting technical and want to find another session right now, that's perfectly fine. Yeah, it's cool. Welcome to our talk. I'm Bastion. I'm a system engineer at Maze.io. You can find me on Twitter at Dasrecht. I'm one of the people who have way too many side projects. So I also am part of TEDxBurn, like the locally run TEDx community. I do DevOps Days at Zurich. So if you want to speak at DevOps Days at Zurich next year, I'm here. I just started doing an open air cinema because it's nice in summer and you can do stuff outside. And I at least forgot a lot of things. I'm Tyler Ward. I am also a system engineer at Maze.io. I'm a former Drupal developer for various agencies like Bastion. Too many hobbies to list. But at home, I am an organizer for the Pacific Northwest Drupal Summit. So if any of you would like to come and speak, Portland would love to see you there. Cool. Yeah, then let's get into the agenda, what we will be talking today. First, we will cover what containers are and what they are not because what they are not part is also important. Then we cover benefits and pitfalls because there are a lot of ways of doing Docker the wrong way and we will share our experience we had along the way. We will go into the journey we had with Docker in production. We will talk and touch briefly on the containers in the future, like where we see containers going in the next few years. We talk then about specific Drupal container tools and in the end we will have ample time to have a little bit of a discussion and cover your questions. So starting off with what containers are not. Containers are not a virtual machine or server, though the end result sometimes is presented that way. When you build containers the correct way, you have a container doing a specific task for a specific amount of time, not a long running virtual machine or server like we are so used to seeing. Containers are not Docker, though Docker is containers. There are other containerization systems, though you will hear us use containers Docker very interchangeably. There's a lot of other ways to do it out there as well as some coming in the future as well. Just using containers is not going to solve problems for you. It's not magic. It's a lot more work than magic. So you can't just start using containers on your systems or on your local workstation and expect it to fix all kinds of things for you. It's a lot more complicated than that. And having containers or Docker or any of those things is not going to magically get you to being DevOps or not that type of situation. In fact, it requires a lot more effort than just installing something and suddenly it helps with enabling developers and operations working together, but it's not going to solve all of those problems. Containers are, they're very small, they're very fast, they're lightweight because of how we build them. They share resources. So with Docker in a VM you have all of these things inside of one server. We bring in a Docker engine there, container engine, again using those interchangeably. And we share all of these things like the system kernel, firewalls, networking, file sharing. All those things are brought into each container so you no longer have to manage those. Your service can be your service and nothing more. Because they are small lightweight, they're just running one thing, it makes it more secure. When you're running containers the right way, the process that's running in there only has access to the bare minimum that it needs to function. So you can't go into that container and see everything else that's running on the system that's hosting that. It can only see maybe a socket file that it can connect to to get access to another service. And then you have orchestration systems, Docker swarm, OpenShift that help enforce security there and make sure the containers have what they need to do their job when they die and come back. Another thing that helps with security is that you build the containers off of images and images are mutable. If you change something inside of one of these containers, you can destroy it and bring it back. It's back to the same state, nothing's changed. Somebody finds a security vulnerability in Drupal and you're running it inside of a container and they manage to change something in what appears to them to be PHP. You can fix your problem in Drupal, replace the container and you've solved a security problem there. For those in the back, there is still a row up front if you want to come forward. We'll take our friends. All our friends. Cool. Yeah, I'm now focusing a little bit on the benefits and pitfalls when it comes to Docker. And that's one really good quote from Richard which speaks a lot in the DevOps community. Using containers will not fix your broken culture. Take that in mind. It will not just magically land somewhere where like, wow, everything works now and we just switch on containers. The tech stuff is rather easy. The human part is hard. Humans resist to change and if you just come and say, hey, we have containers now, not everybody will embrace that or let that happen just easily. Just put a lot of effort into making sure that you make it as easy as possible for people to get started. Another thing is because Docker images basically are a layered system, you need to put a lot of effort into thinking how the hierarchy should work because if you have bad base images which do a lot and change all the time, you will have a lot of time waiting for your builds to finish. So try to map out first what you want to achieve and then start to basically find the way how to make your images as small as possible. Another thing is you should really not reinvent the wheel. Try to base off from well-known and well-maintained images like Alpine or CentOS because they come with most of the things we need on a daily base. You can build everything your own but you will spend a lot of time maintaining that stuff. Of course, it's a little bit creepy, let's say it that way, just using work from somebody else but hey, that's basically how we do it with ThruPaul and just using Composer and pulling all the things. If you have a properly set up CI system, you should not be worried about just pulling in an external dependency but when that would break, your CI system would inform you and say, hey, there's something broken that it would just not deploy. Another thing is if you just have your monolithic system and you roll that up in a container, it's still a monolithic system. There's a way to do that. You can do that with ThruPaul but you should really not do that. And when you start to use containers, let's say you have 100 servers and then you start building everything in microservices and you go crazy and you end up with 700 microservices and containers and it floats around, you need to be very good at the operational part because your problems will scale with the amount of containers you start. If something goes wrong, it just goes wrong in a lot of different places and it just starts to bubble up everywhere. So a lot of things that currently work and we like monitoring, it's just not holding up when you start to use containers. The thing is I mentioned, I just mentioned this, you need to monitor correctly. What does that mean? When we had a server back in the day, we knew that there is an NGNIC on top of it so we just monitored that NGNIC somehow and you really knew what to do and now you just have containers starting and the systems are currently not that sophisticated to just go in and see oh, there is an NGNIC and I just monitored that because that container might just fade away and you need to know when an error is really narrow because it would leave you with a lot of false errors if you approach monitoring the old way. Another thing is storing locks is really hard because as I mentioned, a container might just disappear or die and be rescheduled by your system and you might not have catch the last error which would lead you to the solution of the problem. The whole lock storage and how to deal with locks is also really complicated and KISS, the really simple thing is like the operational things like oh yeah, just go in and restart the service, get really hard if you have the service catered around in dozens of containers but there are also benefits. The problem that you push something and it worked perfectly and beautifully on your local machine and it hits production and everything falls apart is basically fading away because the builds you run on your local machine are exactly the same as you run in the cloud so there is no configuration difference between those. You might have flipped a few things why environment variables but the software and your code is exactly the same and another cool thing is it's platform independent so you build it on a Mac, you run it on a Linux server it doesn't really matter. Sharing is caring so if you have your developers adding a lot of features like changing or adding Redis to your Drupal site and they commit that and somebody else pulls they just run Docker Compose up and it will start the container and also add Redis so you gain from the work which is put into it a lot more than if you would need to say oh, if you want to use that branch please also install Redis on your local machine so you just have a container and it will automatically start. Speed is also very important things back in the days when we developed on vagrant machines like to start a vagrant machine it took like, okay, you need to download an image there goes my first copy and then you do the provisioning and it takes like 5 to 10, 15 minutes until you have something to work with and with Docker you can just say I want to work now and it downloads the image that takes probably 30 seconds and then it just starts it and after you've downloaded the image you have it locally and you just spawn it in seconds and you can also remove it in seconds it means also more flexibility as I mentioned you can add Redis to update the PHP version and change the things on your project and it's just working and you give a lot of power to your developers whereas in the old days or like on all the infrastructure you would need to talk to the IT department and say okay we want to upgrade to varnish 4 and we might want to have PHP 7.2 and things like that you can basically just build that into your Docker images and yeah, operations is becoming a software development skill because basically everything you build in that Docker image is basically what developers define so you need to hold the developers a little bit accountable on what they bake into the images yeah, but you can also have a lot of fun with Docker so I set out at some point and was like ah, I went through my week and I look at my bash history what I usually run, like I run Ansible and it was like I could run Ansible in a Docker container because I don't want to have Ansible installed locally or if I travel by train I also want to have a YouTube video to watch some times and I just wrap the YouTube download binary into a Docker containers because it's easier to just share it and I was like, Tyler ask me hey how do you download those YouTube videos and say, here's the Docker container just run it this way and it downloads it some of you might know tinycur terminal it's basically you just throw a lot of git repositories at it and it shows you it's perfect for stand-ups it shows what you committed today it pulls the weather from Twitter and it's like a small dashboard you can use and it's a NoJS thing and I was like I don't want to install NoJS on my machine so I just wrap this in like 5-10 minutes into a Docker container and it runs beautifully you can go crazy and install Firefox into a Docker container and run it so you're not bound to just having a server there you can also have applications which need a user interface spun up in a Docker container or if you want to go really crazy you can run Docker in Docker so if you want to build crazy stuff that we sometimes do you want to run Docker in Docker and you can basically find it on Docker Hub that it's totally crazy Alright, so our journey, the amazing IO journey into containers not an easy journey it's taken about eight about eight years to get to where we are through four different generations of our hosting platform the first one was built it was like a lot of companies completely in-house why should we trust someone else's work they don't do it the same way we want it done it works but it also means a lot of maintenance and upkeep and you have to have people to build it for you on your staff we went from there to having an external company to then take that over and then improve upon it, replace portions and so on which is good and bad we now have a team we're working with to take care of all those things but still built by us and them not relying or not using a lot of the work there's large communities that can share their work which is what the third one was is we started using Puppet Nance that will bring in all sorts of modules and plugins and tools from those communities to build and manage the hosting infrastructure which we're still operating right now though we're not adding to it where everything we're doing is now going into the fourth generation of our hosting platform which is entirely based on all sorts of containers and microservices we recently open sourced that it's called Lagoon you can find it on our github as well so you can take those things and actually build your own you can skip the first three steps and build your own hosting company at step number four so what is actually going from generation three to generation four look like we had to start somewhere so the first problem we wanted to solve was that local development problem so we took all of our Puppet and Ansible code and we built Docker images off of it now we were doing it the wrong way because it was a monolith all of the services that the Drupal site needed to run were in a single Docker image they weigh in at like 1.2 gigs but everything was there everything's running so it was a great step in once we had that going automation around that so we can build and push new images so all of the developers we work with can get those get the updates right away so we had a lot of less reservices into Docker so we put solar in we needed to be able to easily update to new versions of solar so we added Docker to our servers to be able to run the solar service of whichever version we need on those servers and give our clients easy access to the latest and greatest stuff and we brought back PHP 5.3 we had a few clients who either didn't have the team or the budget or whatever reason they had these Drupal 6 and 7 sites that needed PHP 5.3 so we were able to take build them an image and then put it in an isolated environment so bringing back the security it's an old unsupported version of PHP it's not receiving updates anymore but we're able to run it isolated they're not risking any of our clients or infrastructure and it's in Docker easy to take care of now from there it got really messy we started with the latest version of Jenkins which adds pipeline support to the Jenkins files using those tools to start replacing our deployment system building small microservices that would run as part of that and then we had to look at orchestration of all these systems we settled on OpenShift and Lagoon works with and Lagoon if you come by our booth and see the demo it's amazing but it's close to a dozen and a half different microservices that work together each one does a very specific thing they're all, primarily all Node.js applications that run and can be replaced on the fly to do a very specific job for a time that they're needed so we're starting them up as part of the deployment I'll talk a little bit about the future of containers and as you can see it's all about microservices so when we talk about containers the approach as we go today where you define everything basically in YAML files in your code that's basically the cloud native approach to system engineering or the application development to say and this will become more and more the standard that we move away from basically having the systems on one side and the code on the other that basically everything is one thing and if you want to change something you just change it in code you push it and you will have your new changes like having more backend systems and adding to those projects you gain a lot of flexibility through those things another thing is that if you saw Ranger OS it's basically an operating system built around Docker so every service is running in a Docker container so it's not like SystemD where everything on the Linux world is eaten up by SystemD everything is eaten up by Docker so that's really cool another thing is containers in a car are fairly complicated systems nowadays if you think about the Tesla they are currently thinking about having a way to basically update their autopilot or update the navigation system and how cool would it be when they can just ship a container via the cell network to your car and spin it up again I mean they did something similar with the software update lately in the States when people needed to be evacuated when the hurricanes came they just added more battery capacity on the models which had a lower capacity so they rolled out just an update and if you think about this it's basically we can also just deploy a container to that a lot of work is also done on the Linux and on the Windows side so we can now run Linux containers on Windows and we will soon be able to run the opposite running Windows containers on the Linux because Microsoft is really heavily investing into containers and also into the interoperability between those two worlds so I never thought that I will say that Microsoft is a cool open-source company but they are doing a lot another thing is that we will probably even in five years time not talk about the Docker anymore it will be just another technology but the concept of a container will just stay and as I mentioned earlier operations will be a crucial part of everyday's software development life so just quickly some of the companies that are using containers and doing cool things aside from what we've mentioned Netflix when they get new content they are entirely using individual containers to do all the transcoding now so they need a stream for your Roku they need a mobile version they need 4k each one of those they start a container as service to do that task it finishes it it sends it back to their file system and then the container goes away so the capacity is now freed up to do the next job they're not having long running services that consume resources when they don't need them in fact instead of watching for uptime they watch for how short the uptime is and if it exceeds a length then they look at a problem instead of seeing it running too short and thinking there's something wrong they make sure they don't run for too long Visa is actually moving massive portions of their transaction processing into containerized systems now hundreds of thousands of transactions being processed in various services but it gains in the ability to scale that's because they can take an image and just start adding capacity across a large cluster of servers they have an easy way to scale to process all of their transactions two years ago now Alibaba it's a Chinese kind of equivalent to Amazon large shopping service they have a large sale day called Singles Day $12 billion in transactions they received they'd never seen traffic like this before but because they had all of the services they need to process the traffic to handle that built around containers and microservices they were able to scale out the capacity in real time as they needed it to handle this impressive load and then in the Drupal world there are a few of us that are running containers in production Pantheon they're not using Docker believe it's LXC based they've been doing that since the beginning very solid system service breaks they can replace the container site comes back to life Platform is also and they give you an amazing amount of flexibility using their containers that you get to choose from a wide variety of services to run inside of those and then of course Maze.io we're also doing containers for Drupal hosting and Node.js and all of those services as well a few of the Drupal specific tools won't go deep into them but to give you an idea of what's out there the hardest one to use but some people do is just straight up Docker and Docker compose commands it gets you going it provides a good base but you do have to there's a steep learning curve there Calibox has been around for a while and they just release the newest version called Lando they've worked with Pantheon for a long time on that I believe the new version has a plugin for platform now so they're expanding that to work with those providers to make it easy to develop their systems locally you have easy access there Docker for Drupal DDEV from Drude Outruggers done by Phase 2 in the US and then ours is called Pygmy or we have another version called Cashlot depending on what your needs the thing that all of these provide is they make it easy to bring this to your computer so it takes care of the problems of DNS and a proxy so that you need to find out what container where your Drupal site is running locally these tools all take care of connecting those dots together to make it easy for you to work locally start up a container work on your site save edit reload and then tear it down and stop using those resources when you're done working on that project or need to switch to a different version at the same time it's basically the run through so we have time for your questions about things you're doing or what we're doing or where you guys are easy so it wouldn't be a Drupal con session without mentioning that we are hiring we have two openings for system engineers who want to work around the UTC business time zone hours you don't have to live there we're a fully remote company so if you're interested let us know thanks thanks thanks