 Welcome, everyone. Thank you for coming. I'm just curious how many of you are from Seattle? I'm not, but okay, we have one Where where else are you coming from? Did you fly in? All right, I want to know who came the furthest You came where did you come from? Georgia. Do we have any other one anyone from Georgia? Rhode Island, there's a good one. Yeah, I think If I were a betting woman, I think he might have beat you From Georgia Right anyone else anyone else from anywhere further like overseas or anything crazy like that No, okay. Yeah, I think we have a winner with Rhode Island that's funny because I Went to deliver my son to college this fall and we had that contest to figure out You know which parents came from the furthest and I think we had people from Australia show up. So that was pretty incredible to talk with them Anyway, this is my first conference physically after all this time So it's been what almost a year and a half since we've been allowed out of our homes and stuff So there's been a lot of virtual conferences that we've done Probably more virtual conferences than I've ever done physically for sure and that was a little exhausting But you get a different audience. That's for sure. So I appreciate the opportunity to be able to come in person and be able to do the virtual experience as well for those that have other obligations or whatever So I know there's a slack channel for questions or anything So if things come up or whatever, I'll keep an eye on that slack channel If you don't get a chance to ask questions afterward or you want to know more We can we can go that route I'm this talk containers standards explained I've given this talk several times in various places, but usually with a different focus and It's pretty docker heavy because I'm my background is as a developer So that is what most developers are introduced to in the beginning when they start working with containers at least in my experience and This particular talk has just a focus on how we got here some of the history behind containers Some of the organizations involved in developing all of this new stuff that everyone is is using now For their cloud native development and all of that all of the tools and the components stuff like that and then I'll talk a little bit about the the components of Docker what Docker actually is so it will Bring into perspective what the product itself actually provides as well as what the back-end open source stuff is all about Alright, I am Melissa McKay. I came from Denver. So not terribly far away. Whoo-hoo another Denver person Oh, yes, I see your your Go buffs shirt there. Whoo-hoo And I work with J frog right now as a developer advocate. I've been there for a little over a year and a half So I'm relatively new to the DevRel atmosphere and stuff But prior to that I have about 20 years of development experience I've been an engineer at various different companies both large and small been on several different teams and It's it's been a lot of fun. I've really liked this but that part of my career now It's time to to teach To get more involved with community that kind of stuff. Those are my goals I am a Java champion and I did recently become a doctor captain. So I'm in involved in training and stuff And contributing to the community in that way You can find me and on Twitter and on LinkedIn Those are the best ways to reach out to me if you want to get a hold of me or ask a question All right. So like I said today, we're going to talk about containers but more about the business behind them and how we got here and I have a thing about That is not a container. You're in the right place. I promise. I have a thing about Shipping containers. I just refuse to put any shipping containers in my talks or anything I'm just tired of seeing those every Talk about containers or Docker. They all have shipping containers. So I'm done with that. So this is what you get bananas and The reason behind that is actually When I was putting this presentation together, you know My mind was kind of wandering and I I remember this story that my grandfather told me when I was young In Christmas, so this was around the like the 20s and 30s and for Christmas He and his brothers would get a banana one banana and That was it for the year and they would take a fork and they'd scrape that banana peel and it was a really special treat for them so The analogy I guess is limited resources limiting limited compute resources and so I think of bananas in that way Because of that story so our industry has adapted to dealing with this over time and You know in the 60s and 70s it very limited very expensive But we'll talk more about you know, what happened since then because clearly today It seems pretty easy just to bring up a container run a container develop your stuff package it ship it off All of that's pretty pretty straightforward today And there are reasons for that and we'll talk about that more Since I am a developer I always come from that perspective and my background with Docker is the first time I experienced it or used it was on a team that We were hired to maintain and ship out a product that had been developed by a third-party contractor So the decision to use Docker and containers certainly wasn't ours at the time It was just part of the project and I didn't know it. It was brand new to me So I saw a Docker file. So of course this begins this whole, you know Line of research to figure out what I'm dealing with and how to even build this project Luckily the documentation for this particular project was really good So it was pretty simple just to get it up and running But I really didn't know what was happening under the covers So right away ran into problems where like for example, I didn't realize that if you don't remove your containers, they're just gonna keep accruing on your system and the back-end Volumes and stuff stuff just kept accruing. So I did run into problems like that and had to learn through trial and error Now I'm kind of lazy when it comes to like really diving down deep into something which is both good and bad So the good part is I can get something up and running quickly But the bad part about that is I run into problems that I never Anticipated didn't think about and it's you know, sometimes this can be pretty big Especially if you find out something in production, which did happen to me a few times However in this particular case a docker was really easy to install on my machine I have a Mac love my Mac So I just used docker desktop and I was able to get stuff up and running just fine All my questions here, you know, we're just answered to the bare minimum so that I could get my project out the door All right so if you're doing the same and You are starting to wrap up all of your applications and containers You're starting to go cloud native and and consider this if you haven't done it already Definitely think about the problems that are specific to your project that you would like to solve and Put these down and writing somewhere because I see so much documentation out there that goes into depth on how to do something without explaining why The result of that is you get a lot of times where something gets implemented and then it's implemented a different way And then someone comes along and re-implements it the original way and you just go back and forth because no one ever said Why it was done that way to begin with That was something that happened with our project We did not know why it was developed this way. It just was so we went back and forth on Whether that was the best decision or not It would have been would have been good to have that commitment written down somewhere so this list is just a Short list certainly not all you know it doesn't have everything in here, but these are things that we considered Well, we were working with these new containers The first one it's kind of a joke Java 11 was released in September of 2019 But Java 16 and 17 are you know came out and are coming out There's so many projects are still way back on Java 8 for various reasons just because of slow moving organizations or they're just you know, there may be some Some legitimate reasons Why a business would stick to Java 8 for a while until they get everything moved up But I also know how frustrating it is to deploy a nice new Java 11 application For example to a prod machine and then realize that only Java 8 is installed so that That happens Containerization helps with that a bit. So no longer. Do you have to track what is actually installed on your production servers? I also had a situation where it was decided, you know Particular libraries couldn't be installed and our security went through and removed all of them turns out we needed them For our project so mysteriously our project just quit working If we had been using containers in the beginning that wouldn't have happened All right, too My workstation is a Mac Bacro How many of you have worked in code where you have found Like if statements around a particular OS that you're using Yeah, and I'm thinking now about like the browser wars especially, you know If windows if this then you know do something different there was one time when We had some file manipulation going on and and that really messed up when we were trying to run unit tests and stuff If you were on Windows versus Mac it was kind of silly Those are symptoms of bigger problems But I'm just saying it is really nice to be able to run your applications without having to worry about the OS so much Okay number three There is a bug in production So as a developer you you find out about a bug and you want to reproduce the problem But that is pretty difficult when you're already developing the latest and greatest Let's say you've just been pulled off of a feature that you're working on and now you need to go fix this High-priority bug, but it's on an older version and you have made changes to your environment that are not backward-compatible And I'm thinking, you know various versions of Python installed and things like that can cause you some problems and Consistency and build reproducibility is key when you're chasing bugs. So being able to You know launch your Your project or your product at a specific version without needing to mess with any of that is definitely another pro for using containers Number four it works on my machine, but only on my machine I'm sure we've all been there I've spent a day or two fighting something that come to find out. It's just my environment Again, that's consistency and software versions and libraries and that makes your life a lot less complicated when you are using containers Number five we just hired three new developers This one I have to say was a really this was really good I just gave the example at the beginning where I was given this project You know we were our team was supposed to maintain it and and everything and we were able to just get it up and running Right away because it was all Dockerized and it we just were able to launch the product and it was fine Six my service is super popular So this is probably the biggest reason why a lot of people move to containers one advantage is being able to You know push your your service or application out there and you can scale it now dynamically and You don't have to You can utilize, you know what exists now for cloud native deployments in order to scale your application as you need all right, so Consider the problem you're trying to solve before jumping into containers and don't just do it by default or because Everyone else is doing it The use cases for them even if you're not using them in production Which there's still a lot of folks that are not using containers in production for various reasons. There are other scenarios that are worth considering Deb environments is a big one Don't suffer from that disappointment of spending that entire day debugging a problem that only happens in your machine We already talked about that Test environments having the ability to easily launch other external services so some Some services that you are developing it may just be impossible to be able to have a test environment That is exactly like prod that is the ideal situation That's something that you you want to get as close to as possible But you may not be able to being able to utilize things like test containers Database containers things like that can really help you avoid those problems of You know like an API that doesn't work quite correctly Even though you don't have maybe the load of production you do have the actual communication going on that's in production and then of course your your prod environments in Places I've worked in the past it really just depends on the the company size really but There was a point when you know our operations was a completely different team and being able to get another production server up and running with some kind of some kind, you know a hassle and We had a ticketing system. We would fill in a ticket and request our resource We may or may not get it and might be a long time that kind of thing so being able to Run our applications anywhere being able to use the existing, you know bare metal Resources that we have or the existing VMs that we have set up Different Linux distributions. They didn't really even need to be consistent because we could you know Deploy our containers and be able to run them and make use of our resources a lot better All right, I want to move on to the market and you know Why do we care and get to get a little evidence of what is actually happening out there and why everyone is moving? This direction we'll get to that afterward I did some hunting and This is just one report. There are several out there, but this one is by Cystig if you don't know what Cystig is It's a company that provides monitoring and troubleshooting for Linux systems They put this container report out every year and it's based on the analysis of their users So, you know, obviously we're missing a large part of the population but it's interesting just to see how things moved even just from their perspective and Part of this report includes the data on the container run times that are in use So the first one here in 2017 There's 45,000 containers. That's not really, you know That's not a lot really in the great scheme of things and they didn't have a graph available or anything of Different run times because 99% of them were Docker. So right away out of the gate. There's a bunch of people using Docker in 2018 they doubled the size. Okay, 90,000 things are things are starting to move a little bit At this point we start seeing other run times. So you can see that percentage is there We've got 83% Docker and then the rest are our other run times that are available We can see, you know, something's going on with Docker here All right, 2019 now we jumped to 2 million containers. Now either Cystig just really did good in their business that year or There are a lot of more customers using containers than they had in the past Notice that Docker here is still really high is 79% but 18% is container D and as we'll learn in a few slides later Container D is actually a runtime that Docker builds on top of now and We need to understand what the difference is between a runtime and What we're installing on our on our machine when we're running and building and all of that the last percent that 4% there is cryo and Notice before like in the previous slide There was a little little section for rocket. This is kind of a sad story So notice this graph has no rocket in there Coros was acquired by Red Hat in the beginning of 2018 and Prior to that rocket was accepted to the CNCF That's the cloud native computing foundation as an incubating project and it really looked like a good competitor to Docker's container D However, since that acquisition the development of that project went dormant and in mid-19 2019 rocket was actually archived by the CNCF and in February of 2020 that project was ended It's still there. You can still get it. It's just not being maintained or developed anymore So that was kind of kind of sad To see that one go. They were a really good competitor All right, the last slide here about the market in January Sistig came out with their 2021 container security and usage report This one it's hard to know if we're comparing apples to apples here considering that little qualifier statement of a subset of Customer containers not really sure what that means because they're still still at that two million mark But the report includes a ton of detailed information about their demographics and their data sources Actually much better this time around than in the past even and there's other interesting information around what Services customers are running as well as what they're using for orchestration. So I do have links on here So if you get a chance take a look at this report, it's pretty interesting But in that shell what I pulled from this was the increase in usage of container D I went from 18 percent in that previous slide to 33 percent now And it'll be interesting to see if that trend continues because Especially since Kubernetes announced that they're deprecating support for Docker as a runtime and I'll talk about that more later as well All right now I want to get into some history because clearly the market has been moving and more and more services and applications are getting Wrapped up into containers and that's how they're being deployed. So understanding how we got here is the best way to Make the best use of this tool You need to understand what you're committing to for your project and it'll help you avoid some nasty pitfalls and Help you use this tool in the most effective way just knowing I Have a good example of this. I had a team once that just really jumped on the bandwagon using angular, right and Then angular was a problem the next time around So Understanding where things are moving and what businesses are actually doing behind the scenes can help you avoid going the wrong direction And having to you know backtrack quite a bit this particular project had to rewrite the entire thing, right? So You know that popular cognitive bias as well if you if all you have the hammer everything looks like a nail That is also applicable here containers are useful, but they aren't intended to solve everything not yet anyway All right, so this is our history lesson. This is a lot of information. So sit tight, but it's really important in the 60s and 70s Computing resources were a lot more limited of course than they aren't today They were dedicated for long periods of time for a single task for a single user and of course this You know caused a lot of bottlenecks just general inefficiency Efforts were started to develop a method of sharing resources without getting in each other's way or having one person Inadvertently cause an entire system to crash for everyone Both hardware and software That advanced virtualization virtualization technology started to trickle in and one development was the software Chirrut, which is where we'll start so in 1979 During the development of the seventh edition of Unix Chirrut was developed and then later added to the Berkeley software distribution in 1982 the system Command allowed for the ability to change the apparent route directory for a process and its children Which results in a limited view of the file system in order to provide an environment for testing a different distribution For example, so this is you can see where we're going here This is what we're used to in a container when you you know exact into a container for example You just have this view of a file system that is all contained Chirrut was a move in the right direction for providing isolation that we need and several years later in 2000 free BSD expanded that concept and introduced the more sophisticated jail command in the free BSD 4.0 release those features were later improved in 5.1 and 7.2 There's some a period of time there that helped to further isolate file systems users and networks and that included the ability to sign an IP address to each jail in 2004 we Start with Solaris containers and zones bring us ahead further They give us an application full user process and file systems space and access to system hardware Now there are a lot of other things that are happening in between here I'm just pointing out the things that are obvious. You can see the move to the containerization that we have today So Google jumped in with their process containers. This is when things start moving that happened in 2006 Those were later renamed to C groups, which we've worked with containers. That sounds familiar Those that censored around isolating and limiting the resource usage of a process and in 2008 C groups were merged into the Linux kernel which along with Linux namespaces led to IBM's development of Linux containers LXC containers and Now things get interesting Docker was open sourced in 2013 and that same year Google open sourced their project called let me contain that for you Which provided applications the ability to create and manage their own subcontainers and from here This is when we see the use of containers explode and Docker containers specifically Initially Docker used LXC as its default execution environment but in 2014 Docker chose to swap that out that tool set out for Lib container, which is a native solution written in go So when I first started with containers, I thought that was really an interesting bit of information that this is written in go I'm you know, primarily a Java developer by the way, so I Just find that interesting that languages is is Encroaching Soon after that let me contain that for you project ceased active development with the intention of joining forces and Migrating those core concepts to docker's lib container So then we start this is where we start seeing these companies start working together and moving in the same direction All right a lot more happened, but because I'm a Java developer and I care Java 7 was released in July of 2011 and Java 8 was released in March of 2014 And as you can see there's there's a bit of a problem here because given the development that was going on in Containeration containerization at the same time It shouldn't come as a huge surprise that older versions of Java are not fully container aware Except it is a surprise if you don't know this and you wrap wrap your Java 8 application Only to suffer from out-of-memory killer activity and the reason for that is The mechanisms that the JVM uses in those older versions Retrieve the resources available from the actual host machine and not from the C groups Like you'd expect so they are not isolated like you would expect Later versions of Java 8 fix this problem But it's definitely recommended to get to Java 11 if you want to utilize containers with Java That is the first version that is fully container aware Where you can enjoy all of the benefits. All right a lot more happened during that period of time I'm intentionally skipping over some stuff, but I want to get to what happened in 2015 And this is important to know about because it'll give you more insight on why we saw activity so heavy in the market and other shifts, especially with Docker on June 22nd 2015 the establishment of the open container initiative was announced and this is an organization under the Linux Foundation That has the goal of creating open standards for container run times and image specs Docker was a heavy contributor to that organization, but in their announcement of that new Organization it was said that over 20 were involved in this This is a long list and you can see a bunch bunch of companies have shared interests and common goals and You know, we want to come to some agreement on how Things are going to run so that we don't have just one company taking the whole market share like we've seen we want more Opportunities for all and better agreement on what's needed in the in our ecosystem and stuff so I have mixed feelings about this because I know that You know coming up with you know regulations for example or specifications or just coming up with you know Common components and things. This is all a good idea. I Know when I take my car in to get it fixed I appreciate that I can just you know get I can replace pieces and parts as I like But that's not always the case sometimes I get some you know random part that can only be found in one place Had that happened to me recently The other thing is you know when I when you have standardized Building materials, you know and you end up going back and forth to The hardware store because you don't have the right size of screw or you bought the wrong size of door or window You know that kind of stuff So that's the goal here for software is to make those processes a little bit easier for the rest of us so not everything is so customized and We are we can figure out what these commonalities are that everyone cares about what we can take and run with Without reinventing the wheel and then still get the customizations that we want to put in All right, the OCI accomplished some pretty big things They in 2015 in June They established the runtime specification the image specification the distribution specification in 2017 those runtime and image specs were released the first version of those and in 2021 fairly recently distribution was also released and One month after the OCI was established the cloud native computing foundation the CNCF was established They do a lot. There's a ton of projects that are either incubating or graduated under its umbrella But there's just a few activities that you need to know for for this for our care today they are involved in a lot of Rapid development of orchestration. So now that we have all of our containers and stuff out there and now we're concerned with how to deploy them and Orchestration frameworks like Kubernetes Took on the responsibilities of the coordination the configuration the management of containers in deployment environments and Kubernetes version one Dotto was released at the same time the CNCF was announced that is not a coincidence Kubernetes was contributed by Google and was the first project that was brought into the CNCF Right on the heels of that release Version 1.5 came out which included the container runtime interface That's another term that you may have heard again and again that term at runtime I'll go over some more details on that in a minute In fact right now so what exactly is the CRI the container runtime interface that is a level of abstraction that allows Kubernetes to support Alternative low-level container run times. So now we now we have other run times that we can use and the container runtime interface Will adapt you can you can plug in so that you're not stuck with Docker for example and Docker the company is Also a member of CNCF and it's they're well on their way to breaking up that huge tech stack that we know is Docker They contributed their CRI compatible runtime, which is container D So container D came from Docker even though it's considered an alternative runtime Docker actually now uses it in their product Container D was developed in order to integrate run C, which is a low level Runtime and that was that was Docker version 1.11, which has been out for quite some time now So this is old news. All right, that's enough with history That was a lot of data to go through but my hope is that now you have a better understanding of how we got here Why everyone's excited about this and how many companies and organizations are actually involved in You know going forward with us But there are still some terms that you might be confused about I know I was as a developer starting out with this like when I install Docker and I'm running Docker on my machine What exactly is going on there? What what exactly am I using? What exactly is open source? What is the product? How what how is that relationship? And can I use something else besides Docker for example? So what we need and why Docker just really made a Took a lot of the market share was that Docker does all of these things You know, we needed a way to define the container, which is the image format We needed a way to build the image of a container We need a way to manage those container images We need to distribute and share those container images. That's Docker hub We need to create a container environment That's you know part of your runtime launching and running that container and then manage the life cycle of those container Instances all of this happens with Docker. So you can see Why they were able to just make it simple and easy to do but as a consequence We have a hard time separating these pieces and parts and understanding the differences between them and what what you can eliminate from that tech Stack and be completely open source without using a Product all right container runtimes. So this we need a little more explanation here there are Low-level and high-level runtimes the Docker tech stack contains more than just the runtime But the runtime is the big thing people talk about Especially with the kubernetes announcement Docker did quite a bit of reorganizing their code base over the last several years. They developed abstractions pulled out discreet functionality Open source a lot of it donated some projects and one of the results of that effort was a project called run C Which was contributed to the OCI organization and This was the first and for some time the only Implementation of a low-level container runtime that implemented the OCI runtime specification There are other runtimes out there and as of this Talk this is a very active space So you need to be sure to reference the current list that's maintained by the OCI for more up-to-date information And I have a link to their organization out there other notable low-level runtime projects include crun C run however, you want to pronounce that an implementation in C that was led by a red hat and Rail car an implementation in rust led by Oracle, which is now archived So there just isn't a lot, you know run C's great a lot of people use it Not a lot of reason to go out and write your own low-level runtime Developing a spec is really challenging and collaborating on that OCI runtime spec wasn't any less challenging So figuring out where the boundaries are and what should and should not be included in the specs It took a lot of time before the release of that first version of the spec But it's clear that just implementing the spec to get that low-level runtime isn't enough to get adoption You need more you need stuff on top of it So additional features that you need to make it usable for developers since we're concerned with much more than just The creation and the running of a container. We're concerned with all of those other things distributing building All of that so this leads us to the higher level run times Which are container D and cryo right now These run times include solutions for many of the other concerns that we talked about before around container orchestration and that includes image management and distribution and both of these run times implement the CRI Which eases that path to kubernetes deployment as well Both delegate to the low-level container activities. They delegate to the low-level run times like run C So I've brought kubernetes up a few times This is a really good blog about this subject But in short the deck deprecation of the Docker runtime does not in any way shape or form Mean that Docker is going anywhere or losing support You can still use Docker to develop and produce Docker images and containers as you do now This is just relevant for the runtime that kubernetes is and if they're the default is now going to be Container D, which Docker already uses that as well. This is nothing to be concerned about All right, that said something that happened in 2017, you know, we talked about shared components. I told you the story about cars and going to the the hardware store over and over again Moby came out. So this is basically the upstream of the Docker product So there are you can think of it in four levels You've got the top level, which is all the way upstream components Then you have Moby then you come out with Docker community addition and then with support you come out with Docker Enterprise Edition So this is your opportunity to get Custom bananas if you really want to go crazy go check out the Moby documentation There's all kinds of tool sets in there components where you can build run use your own stuff It's pretty cool. And that's something that I'm working on now as a Docker captain Figuring out, you know, where I want to insert myself in that world. And that's it Basically, well, I mean, there's some helpful links here You can't go wrong with the the Docker documentation. They've done a lot of work there and improvement there But in addition to that also check out those organizations the CNCF the open containers org Which has all of the OCI specs available for you to look at Also, they have a community slack channel the Docker community slack channel Which you can also use to talk about Moby things like that They have just under 28,000 users So it is a huge resource all of the Docker captains are on there as well watching, you know Conversations and stuff and jumping in when something is important. I know there's a lot of discussion right now About the latest licensing stuff for a Docker that's happening for Docker desktop And that's that Thank you very much. All right, I don't know if there's any I think we have a little bit of time for questions Anyone yes Very good. Well, there you go. You need the isolation Yes, okay, so remember this slide where I gave you all the description of what Docker does for you Podman does a few of those things, but not everything so building Building images Can get a little complicated Especially if you want really want to work on your Mac or you want to work on your Windows development Laptop you do need a Linux machine really to get the most benefit from stuff that you can get from Moby Which includes, you know, their Linux kit stuff the build kit stuff All of that for example if you wanted to build images without using Docker desktop or you don't want to install the Docker daemon on your machine for whatever reason Maybe you don't want to pay for it Then that would be the That would be where you would go. All right, there's a couple other projects out there, too. There's build a There is well, there's jib as well, which is the way to make Java containers But you and you don't need to install Docker to do that There are some limitations there like how you deal with layering you have to consider But it's not enough of a problem to prevent someone from using it as long as you understand what's going on Any other questions? All right, so I just want to say definitely go check out the specs They are available on the open containers website. They have links to the github links for the specs Those are just interesting to keep track of Often a lot of questions I get are around like is my Docker container the same as an OCI container and They are for the most part when you are talking about storing your images for example Being able to push an OCI image versus a Docker image to any of your Your Docker registries that's something to pay attention to some only support Docker However, Docker there there are some minor differences and how How they're labeled but they're essentially the same thing They should run the same way. All right. Well, you have my Twitter information LinkedIn whatever if you come up with anything that you want to ask later There's also the Slack channel that this event has provided and I'll be keeping an eye on that as well