 Hello everyone. My name is Lucas and I'm thrilled to be here to be presenting you on this awesome event Well, let's get started to it. So we don't lose any more time Okay, so today we're gonna be talking about integrating containers into JavaScript in several ways I've been doing this and How did I get to this and how we can leverage this power? Okay, so basically These are my social networks. This is my email. My name is Lucas Santos I am a developer advocate from Sao Paulo, Brazil here and Microsoft And if you want to get in touch with me in any way, just put my social network dot L from Lucas Alsandos dot dev Okay So if you want to know anything about me at all if you want to know any other social networks Just go to info dot Alsandos dot dev. Okay, then let's get started, right? So containers, they are a thing now, right? They are containers containers are widely used for a lot of things They used in several areas with several technologies kubernetes is here Dockers is dockers here and there's like a lot of stuff that actually uses containers and it's Something that's been really active and really hot in the technology area But containers in a new the excess like since the 80s or something like that they started in Linux and you're not actually telling the whole story of containers here today What I'm going to show you is just how we can make this work with JavaScript And why did I choose to do that, right? It's easy to control containers, right? It's pretty easy dockers here to show you to show you that We can actually control it via like a CLI or something And basically docker does most of the work for us You don't need to understand how the underlying infrastructure actually works because this is basically you know Not important for you to get your work done So controlling containers mostly have been done by using CLIs or other Graph code user interfaces or something like that, but it's easy if you're human, right? Most of the CLIs and most of everything we use now, they're made for human beings They're made for people, right? So docker CLI is awesome It allows you to do a lot of things and there are other toolings that are actually pretty good In what they do to allow us to spin up containers kubernetes is here to show you that we can have a good CLI We can control and a lot of other things, but the point of all that is that we We need to create things as if they didn't work They didn't were created for humans, right? So All this tooling has been created to be used by humans in the command line interfaces instead of programmatically through an API or something like that But what happens if you're not a human if you're like a program on an API or an operator or something like that You might face a hard time By you know Doing that for a lot of reasons that i'm going to show you and now we are going to just understand What is what we're actually working with right so to present this to you We're going to use a technology called continuity is actually a tool that was built by oci is a the open container initiative It's built up by like a lot of companies and including docker and microsoft and a lot of other companies and the The basic idea the goal of the oci is to create a standard interface for all containers and all images that we use today so Continuity is a high-level runtime. Actually, it's more like a runtime manager because it controls something called called renzi Inside a linux machine. So what we're going to do today is that you're going to spin up a linux machine, right? So i'm going to use a linux machine With container installed and run seen stalled so we can run our demos and our examples here So basically this is why containers are actually widely used in linux and not so much in other os's right It powers docker. So docker actually uses continuity Underline its implementation and infrastructure It was part of docker in the beginning and then was split up into several other stuff and continuity differently from docker or other Dilling It was created to be manipulated through an api or an sdk client or whatever, right? Despite continuity also has a coi interface called ctr Because it's actually pretty easy to manipulate containers using the cli and basically container d actually does that For you if you have like a cli or something you don't need docker installed in your machine And you don't need anything else installed. So it has a cli have you know despite being created to be used programmatically But this is not important actually having the cli is one of the parts that Will allow us to control it a bit better, but that was one of the attempts, but a continuity is actually widely used most of the container run times uses container d in underlying infrastructure or run seeing some way, but It's not so easy or javascript developers So if you go google or whatever search engine or whatever Documentation you might find you're going to see that most of the container things are done using go Container d is actually built using go Docker is built using go Run C is built using go and there's like a lot of other things that we can do But they are mostly You know aimed to go laying developers This is one of the examples. There are a lot of other examples But basically what we're seeing here today is that we need to import all the packages and you see that in this right-hand most image here That we have a container d client that is actually well integrated into go and you can just import it and actually create containers and delete containers and pull images and so on just using the go lang CLI which is not actually present in javascript, but then I actually Came across a article that Made me think of it. This is actually not wrong, right? It's not actually wrong to be written in go It's performatic and it's a great code But it's not intuitive. It's not that intuitive for those who doesn't know Go lang or whatever other language So I came across this awesome article by marcos How to get a browse to communicate with container d through geopc? So I thought well, if container d has a geopc interface, maybe we can actually integrate things Into javascript. So I thought well What if I did this using no jas, right? I I know how to integrate it into geopc interfaces using no jas. I know how to do this using The javascript command line and I know how to use this On javascript itself that we have like a lot of libs that Allows us to create a javascript a plain javascript job is seen to face And that are very actually good to do this. So I tried a first attempt. Okay So the first attempt I did was sometimes good mostly bad, but it kind of worked and it was ugly at all But the first thing this first attempt is just a proof of concept To show us that are two actually actually there are two ways or even more ways That we can integrate container d into node. It's actually a simple way to you know Avoid being so You know verbose and trying to actually not to know all the concepts You don't need to know all the concepts behind container d to make this work So the first attempt was to use chat process So basically what I did was that I created a chat process and I spawned this chat process to actually create instances of the ctr container interface And well, this worked very very well, right? So container d has this COI called ctr, which I already told you And it's able to do like a lot of stuff maybe maybe like everything that container d can do is able to do You were able to do via the ctr Command line interface. So what I did was just I put all these commands into a node.js process and try to build something out of it I'm going to show you the code, but let's talk about the pros and cons of this first, right? So you might be thinking well, this is easy. I can do this at home. Of course you can do this at home And I encourage you to do so. It's actually very very nice to see these things working But the pros of this is that it's super easy. It's very easy to integrate using a command line interface and using ctr using chat process. Chat process is actually amazing to use And you have like a lot of tooling on top of that that you can indeed use And well, this is one of the the pros the other things that it takes advantage of All the implementation that's already done in the ctr interface. So you don't actually need to be So Into containers or you don't need to know how container D actually works. So you can Make everything Work inside JavaScript. You just need to know how to call a process and just need to know how to control that And well, it's a faster implementation So basically, uh, you can do things very very fast. I did this isn't I did this Container decline like one hour or so, right? and can be integrating to any api basically if you can Manipulate this through the code. You can integrate this using the apis So it's actually so simple that I did this I did this amazing container the api which is basically a You know vanilla javascript Running yes modules in the browser and it doesn't need anything else just some css and some html And it kind of works. So as you can see, I can create I can list images So let's jump to the code and I can show you this actually working So This is our code and basically what I'm going to show you is this small server. It's built using koa So as you can see, it's just a few routes a delete route a Put route and so on so we can create containers. This is our course And we have a body parser and allow methods and so on. So this is a base kpi and this only calls the ctr Okay, so ctr is a container decline that I Called container deep because of obvious reasons and this is actually everything that does the work So it's already running here. I run both the api and the application So this is the static application that I'm running using the Coa static web server So this is all the same just to serve the html And basically what you can see here is that I have my implementation of container d basically using just trial process promises and This is actually it so we have These actions that I freeze we have the list namespace action So I just exact a sync. So I just run an execution a sync execution of ctr client This is really really just the proof of concept, right? So we have this if we have a standard error, then we just error out if we have a Another thing that is not a standard error. We can pause this string Output so basically this is the idea. So I can parse the string output I can run everything that I need into this thing. So Basically, this is the Amazing container api as you can see I'm a backend developer I don't have a lot of experience and I can't do good front ends So I can change the default namespace container d is based on namespaces So I can change the full namespace, but I won't do this. I'm going to download. I have these Network manager here open on dev tools So I'm just going to show you how these things are working But I'm going I'm going to pull this image So I'm going to pull the go vote api which is a simple image It looks like it waits more like five megabytes or something like that Then I can create a container from this image and I have this Website already open for for api this runs on 8080. So as you can see nothing actually happens here because the container is not running But I can actually run a task to run this container So once I create this task my container is going to run here And as you can see, I can actually use my api and I didn't need to integrate anything into anywhere So I just can I can just kill my task to leave my container to leave my image And everything is going to be back to what we had before Okay, so this was my first attempt of creating something that was going to be used as An integration for container d right so This is not it's pretty good But we have some cons as you may have seen on that The first thing we have is that it's heavily dependent on the environment you have so Whatever you were doing You just need to you can just run this client if you have it installed in the same machine as your container d binary so If you have like on you have a machine with container d inside it and you need to run this api in somewhere else Basically, you can do it because it needs to be installed in the same machine as the api Of course, you can actually run container d run this api and then run a reverse proxy or something on top of it to communicate Externally or remotely to this api. This is uh completely feasible, right? But uh, there are other cons on this Basically, we don't have any control over the process so if by any chance the ctr runs into a problem or something that we haven't anticipated and Being honest, it's really really difficult to anticipate any sort of Errors that happen in a command line interface because it's not meant to be Giving you all details of the error because it's basically built to be used by humans, right? Not machines So you can't control all the process you just control its inputs and you receive all these outputs And this is everything that uh, it's bad about it because you receive a output. That's basically Uh, a giant string And you need to parse this string to you know, figure out whatever if you had any errors or something Some of the clients like container d clients that you are they have a quiet mode that doesn't output headers or something but uh, it's You know poor I can't actually get the container status or the image status or task status because uh these uh These informations are actually hidden behind a parsing or a table like structure something like that And basically what you have to do what we have to do is to parse this giant string and use string parsers And so on so we can put this thing into data structures that we can manipulate Okay, so this is the first part, but the second part is that uh, this problem doesn't allow us to give a proper error handling Because we just know that this is going to be an error when we haven't standard error output And the standard error output is meant to be used for humans. So it's a human Readable error and it's not a machine Like error, it's basically What we do is just receiving the error string the error message and we can't know for sure If that error can be corrected or not instead of like we can just parse it But you know, we can parse it by rejects. We can use Whatever thing that we can but this uh, is basically running all the errors in one place and trying to figure out How to parse everyone using regex and using uh, you know every sort of hacking that we can do in strings So this is really really bad. This is one of the reasons why these api and these front end doesn't actually return Uh a good error handling sometimes when uh, we run into an error like the image red exists or the image doesn't exist Uh, it returns like a chorus error or something like that And I can actually return this because I'm erroring this out and this doesn't Return my error properly, right? Uh, you need pseudo to be run. Uh, although this is actually possible to be removed. You can, uh, configure it, uh, you can configure its, uh container d to run this, uh in Is and as another user, okay So there is a config file and I think it's slash edc slash container d config dot toml that, uh Allows you to change the default user id and the default group id that container d is going to create its Sockets and it's going to create all this process So you can actually change this to allow you to run container d without pseudo, but it requires extra configuration, right? Uh, and actually this is The most important part, uh, we had the the error strings. We had all the output strings, which is kind of boring It's kind of uh complicated to treat but it's doable But uh, the only thing that we need to actually be, uh, a Aware of is that this is a huge security failure because uh, we can input anything into that So if we run into any attacks or if we run any into any hackers or something We need to sanitize all user inputs for our api because this, uh, otherwise can be passed on to our, um Container decline and this can run some things that we don't want to run inside our machine, right? So you actually have to be very careful of that Uh, so the conclusion is, uh, we can't integrate containers into javascript This is the first way this was the proof of concept that I needed to make sure that this was actually possible Uh, but I wanted to do this, uh, a real integration like I wanted to integrate this Without manipulating external stuff and manipulating things that are already there So I wanted to actually create something that was going to be native Okay, but not so native because it's just jrpc, right? So, uh, the second attempt I did was to read on to the container g Joppy c interface so container d was meant to be extended You can extend container d and it has a joppy c interface that allows you to do so So, uh, if you have like this is these are the all the brutal files and you can see these are all the concepts that container d actually has Uh, containers, content diffs, events, images, uh, introspection, leases and namespaces, snapshots, tasks and Whatever thing else that you want to do and this is like the image service Okay, so we have a get a list to create an update image and so on the problem is you need to know how to do it, uh, because The interface needs you to control every aspect of a container creation pipeline from downloading images to Downloading blobs reading manifests Creating containers creating an oci spec for both the image and the container creating All the bindings you need from The file system to the container creating all the flags needed and everything else. So basically If you have Anything that you run run like, uh, you need a lease for researchers need to read a lease and need to create a container and mix that up together And uh, in other words, you need to understand how container d works under the hood So in every aspect of the container and creation and every aspect of how containers actually work For these you might need to you know, read the oci specs for image The oci specs for distribution or the oci specs for the artifacts or containers and whatever. Okay, uh, the runtime spec for instance, so This is the the example I want to show you We if you need to create an image like the list, uh, rpc here doesn't actually get any, uh Parameters is just a filter a string filter to filter the images But the create one actually needs an image type So the image type is not just a full image. You can just download an image and pass on that image To the jrpc interface the image type is just a descriptor So basically the descriptor is just a name and some set of labels a descriptor a oci descriptor a timestamp for creation and updates and basically The image is just a pointer to a set of blobs actually the blobs a pointer to a set of images, but The image interface itself. Uh, I can't create an image without even downloading that So, uh, I can create this image. It's going to be there, but there's nothing behind it. There is no root file system There is no file system at all. So in order to fully create the image We need to download the content So this is the service for content that we need to download it and these downloads are blobs As you can see if I run like ctr content list We're gonna see that we have all the layers from our image like it's just three layers, but Those are the blobs that uh, I downloaded when I pulled the image and In other words, you need to understand all the flux of how to download an image Which is not that difficult But this is not very well documented. The documentation is as far as it's actually scattered everywhere, right? So it's possible to do it. This is what I did. I integrated with the jrpc interface So let's jump into the code so you can actually see this working. Okay is not that immense example or something that's gonna take a breath away, but uh, in other words, uh, it shows you that it's actually possible to do So so what I'm going to do here is just uh, I'm loading the proto files. So the proto files are here Uh, they are completely descriptive and they are completely, uh, downloaded into this thing So basically this is the content proto file and it's very well commented. It's very well documented here inside the proto file But not the whole pipeline process Basically what i'm doing i'm loading these proto files using the the full jrpc module from no js So this is it's a bit laggy because i'm using x to Execute visuals to you code outside of my uh, Linux VM. So this is running inside the Linux VM Uh, so it's a bit laggy But i'm using jrpcjs and the proto loader to load an image definition And the content definition here down below As you can see so i'm just loading the content dot proto and i'm Loading the images proto file i'm creating a client So i'm creating this image client from the continuity services images v1 images And i'm binding it to the eunix Socket, so this is the address that we're going to bind to our jrpc server Okay, container d actually allows you to expose these as a tcp and over htp server And i can't create this without any, uh, you know, uh, jrpc requires you to have a certificate because it uses htp2 Under the hood so uh, we can actually create a empty certificate using this And we can add this metadata. It's just a header base basically, so it's a header Named container d namespace and we are going to use this namespace js ctr Right, so js continuity and this is the payload that i can use to create an image So it's the name of the image the target. This is the descriptor The oci descriptor, which is a basically a size a digest and a media type And the creation dates so i can create this image using this payload and that namespace And i can list all the images using a filter, which i'm just not going to use For the creation, uh, i'm just going to use the content definition, right? So in order to do this First i'm going to run this for you down below here in p.m Start, uh, and as you can see i can, uh Download this image so i downloaded that so this image is now here and, uh If i try to download it again, i'm gonna have a error Okay, because the image is already there as you can see the image already exists As you can see this is a very good error details. It's a very good error code So as you can see we have a code you have details you have a lot of other things and uh, if i execute this on my amazing Container api here, but changing the namespace to js ctr We are going to see that this Is actually what we just did but i cannot create a container from this image because uh, it's just a pointer Right, so, uh, this is what i did right now This is what i had time to create for this presentation because these two Content definitions does not need actually any inputs So this is the output from the container, uh as the content definition as you can see This is just a blob file that is put into json Uh, and basically this is what we have when we download images. Okay, so now back to our Presentation over here. You're running out of time Uh, this is the pros. Oh every every of these every one of those two attempts have pros and cons Okay, so the first pro is actually we have proper error handling as you could see we have a basically, uh, Basically, uh interface for errors and a basic interface for handling these images and these Messages and everything else. Okay, so we can actually return a proper error to the user We have proper return codes So we can actually return proper errors to the user And allows you to fully control all the aspects of the pipeline So if you have anything that you want to be Better or optimized you have caching you can do whatever you want from this pipeline Like you can download the manifest and put them in a cache and then uh download this For further downloads you can just look into the cache and then download all the blobs again or whatever you can optimize it If you can like find blobs into different images that have the same Digest you can just use them mutually or you can add whatever future whatever features you want Okay, so it's basically the whole control the best of the control you have And does not need to be in the same machine. You can as I said config container d to Upload the gipc interface into a dcp port, but it needs you to have a crt Certificate, so you need to have a certificate file both for the You have to actually have to have a certificate file in a private key file to Be able to do that. But once you have once you have this you can connect to your gipc api for from whatever External machine you have so it doesn't require sudo because you're actually connecting straight your continuity and it's practically name native Okay, so you're just using a native module not native module. You're using a native protocol You can actually extend this. Uh, this is the standard way to extend container d. Okay So the cons are that it requires a lot of knowledge. Uh, I don't even know how to start Describing this because I didn't even discovered how to actually execute this in a pipeline completely I'm doing this job. I'm going to tell you about this By the end of this talk, which is already in a hand The documentation is a sparse both for jrpc in javascript and container d So we don't find this very easily You need to actually read the source code for go or not for go But for a continuity and cti to actually learn how these things work and how to build the ocr spec It's way harder as you can see Uh, and might be a problem to connect external servers since you need these certificate files, right? Uh, and you need and everything to actually connect the external servers. So what are the next steps we can do to this? Uh, actually this opens up a very good opportunity to integrate javascript into containers and make like ci interfaces or whatever actually you want to do Using containers. So what I did is that I'm starting a project As you can see you can actually go into my GitHub account and you can help me do this because it's really really in the start of this project It's container djs. There isn't anything that is actually, uh, like it. So I'm building this from scratch I'm going to use Everything that I've been learning so far and everything that I can find on the documentations to build a very Native and very, you know, useful client for container djs. So javascript folks can, uh, you know Use this to build their own tooling and to build their own better version of containers docker and you know Leverage all these incredible tooling that we have into other languages as well, right? so let's add more knowledge and Actually create more documentation for this. So if you know how to do it create more documentation post about it blog about it and actually Share this knowledge so we can create a better ecosystem both for containers And both party the javascript folks and both for those who actually love containers and love javascript like me so, uh, basically, this is what uh, this is what my goal is, uh to create more documentation into, uh Microsoft docs and whatever other platforms we have to Make more people able to create containers and to understand containers and to show them that containers are not that Hard as it seems to be, right? So, uh, these are the refs that I took. So basically all the, uh, GitHub files and all the repositories that you're going to see here all the code that I showed you is in these repositories So, uh, this is the integration example using just going and continuity with a client Jesus, this is the js continuity example that I showed you right here And this is the ongoing work to the container djs client, right? This is mark's article about jypc web interfaces and he has a lot of good articles in his medium I really suggest you to go there and read it. Uh, it's amazing How he can integrate this into several other languages like java He can do this using jpc web. It's amazing and well mark's actually one of the people that I gave me this idea to Try to integrate this into javascript and make it work So I can share these things with you if you want these slides It's on slides. else on those that have uh slash integrating containers into javascript. So i'm just going to leave this here and Uh, thank you so much and it's a pleasure to be here If you want to talk to me if you want to ping me out on anything info dot else on those dot dev And well, I'm always available on twitter into uh facebook or linked in whatever github So if you want to help just come on and ping me. So thanks a lot and I hope you have a great day