 Hello everyone welcome to a Jenkins online meetup. I'm Liam Newman your host and today I'll be talking with James Strachan The lead on the Jenkins X project about Jenkins X Automated CI CD for Kubernetes So James tell us about that I'm sure I'll just stop my slides because I I'll end up telling you about that in like two slides time But let me set the scene a little bit before I talk about Jenkins X So I want to give a little bit of background about the problems that people have and then I'll introduce Jenkins X So let me do a quick step back so A lot has happened in the last five years about how we build software package it release it deploy it manage it, right? If you're trying to be a high-performing team Almost everything has changed in what you should be doing these days. Okay, so There's a huge move from on-premise to public cloud That's been going for probably about a decade, but it's really exponentially ramping up in the last few years It's much easier to elastically scale your infrastructure on the cloud rather than Guessing what hardware you're going to need next year and then buying office space and machines and doing your own kind of Private data centers. So there's this big move to the public cloud In addition, there's this big move away from virtual machines to containers And the main reason for that is really just cost A container is smaller uses less cpu disk memory than a virtual machine so Traditionally, we've often used virtual machines as a way of deploying software into staging and production environments Increasingly, there's now an economic need to move from virtual machines to containers. So that's a fairly big change Then in the last kind of four years, I think all cuban 80s has come along, which is basically the industry standard way of orchestrating containers at scale cuban 80s came out of google And it's basically an implementation of what google has been using internally for about a decade Everything that google runs on the container or in the container So whenever you use google search google maps Anything that google does you're running you're using containers And the big problem with containers is how do you manage them all right? Once you split all your applications into lots of little containers, how do you keep them up running? How do you do load balancing? How do you scale up and down? How do you make sure they're all running all the time and they can talk to each other and so forth? And that's really what cuban 80s does and why cuban 80s exists And this year 2018 is basically the year that cuban 80s won cuban 80s has become the standard way of running containers at scale on any cloud Right, whether it's public cloud private cloud, whether it's your laptop In 2018 all the major cloud providers will have something called a managed cuban 80s as a service And what that basically means is this year you can pick any reasonably popular cloud provider Certainly the big three in the western world and alibaba in China So you'll be able to go to google and microsoft amazon's website and say please give me a cuban 80s cluster And you pick how big the cluster is how many nodes you wish to use how how big the nodes are And so forth and then the public cloud vendor will manage cuban 80s clusters for you They will keep those nodes running if a node fails they will automatically restart them You'll be able to elastically scale the cluster up and down to your heart's desire and you'll be basically build in terms of how much compute nodes you use how much Computing and memory and disk use In many ways cuban 80s on the public cloud is basically free now Like a completely managed hosted cuban 80 services basically free You just pay for the compute disk and network of your actual containers that you've run on cuban 80s, right? Which is pretty cool. So this huge change from On-premise to public cloud from vms to containers and the move to cuban 80s And adds lots of complexities for developers, right? Not many developers even know what cuban 80s is yet Never mind how to use it well Then in addition there's this huge movement in the it world to move away from monoliths and microservices And we're all trying to become high-performing teams So we we're trying to figure out how to develop software quickly And deliver business value to customers as quickly as possible so that we can get feedback and we can iterate, right? So rather than spending years in the library tower working on a huge monolith We want to Create little microservices and get them in front of users really quickly and get feedback and then iterate quickly and quickly So we basically want continuous integration and continuous delivery With containers on cuban 80s on any cuban 80s cluster public or public cloud Which is a kind of an issue, right? This is suddenly there's all this stuff We all have to do So if you're a team right now who's trying to deliver software value to your customers You have all these things you need to figure out You need to figure out how to set up your cuban 80s cluster How to set up your environments like development testing staging production How to move your software into containers? How to deploy those containers on cuban 80s cuban 80s uses this yaml stuff for generating deployment descriptors And then there's this other thing called helm charts Which is seen as the canonical way of packaging applications for cuban 80s Helm is like a package manager for cuban 80s Then people need to figure out how are you going to do continuous delivery and promote released Applications between testing staging and production And then there's this thing called git ops that is a term that we've worked folks made up last year Which is a really good term That increasingly when we start to do high performance High-performing teams and we're trying to do operational stuff like how do you manage? What's running in a staging environment or what's running in your production? environment A common practice is git ops which basically means Store all of your environment specific configuration in git So everything's versioned and audited and has history So then you have a somebody makes a change in production They make a change through git not directly with the production system So that if all of a sudden production goes pear shaped You can just look who did the last commit in production Maybe we should roll that one back because that might fix the problem So git ops is the new hotness in how to do operations So we've got the cuban 80s the containers the Moving to helm moving to continuous delivery adopting git ops Then we need to automate everything because we need to spin up lots of different microservices Whenever we feel like so we can do experiments and then we need to get the feedback back to the development team from all of this kind of stuff Then once we figured out all of that stuff We then need to actually do the business value that we meant to be doing in the first place So basically today we have all this stuff We have to worry about figure out before we can really become a high-perform team And that's really where jenki's x comes in jenki's x is really trying to help you and Continuously deliver business value through software Without really having to worry how to set up cincd How to deploy in cuban 80s without having to totally understand what the helm chart is or the docker images How you should package applications for cuban 80s And how git ops should be even set up It's trying to optimize all of that kind of stuff to help you get going quickly, right? so What does jenki's x do for you? So the first thing jenki's x does is it configures a whole bunch of tools One of the lovely things about cuban it is is there's a massive vibrant open source ecosystem One of the downsides of cuban 80s is what tools should you pick and how should you plug them all together to get A fast efficient Cicd experience All right, so one of the this is the typical Table looking with open source where you have a lot of options, right? Exactly and and jenki's is notoriously Challenging in that world as well your face with jenki's often traditionally is There's 2 000 plugins or a thousand plugins. Which ones are the ones that you think you need for your use case, right? So kubernetes doesn't solve that and jenki's x doesn't solve that particularly In terms of the set of choices that you have, right? Well, what jenki's jenki's x does a few things what jenki's x does is it's an opinionated choice of open source solutions Plus jenki's x integrates them all together for you So rather than you having to google on the internet, how should I do see i'm cd And you find a resilient different tools that do a piece of the puzzle What jenki's x is trying to do is Glue all of the best of breed tools together into one simple to install package Then it also takes care of upgrading these tools because each one of these tools has its own release cycle Right, so there's tools like helm and scaffold and nexus and monocular. There's a whole bunch more each of these Tends to have a fairly rapid release schedule and changes quite frequently So another thing jenki's x does is it takes care of upgrading So that as new versions of scaffold comes out or helm or nexus or monocular or Chart museum and various other bits of pieces plus jenki's x itself and jenki's itself So all of these tools are changing all the time. So jenki's x takes care of Installing and configuring and upgrading all those things So he does do that which is a which is a good thing. So that's a good thing And then the other thing he does is not just integrate all those tools together It then alternates how you set up your cincd pipelines for your actual applications that you're building So as soon as you have an application, you might be creating from scratch You might be importing it from some source. You've got to get somewhere You might be using a quick start or whatever As you bring an application to the table And jenki's x will automate setting up the pipeline to do cincd It'll automate creating the docket images and it will automate creating the helm charts So we'll basically automate the cincd stuff Using best practices for how you should be doing this on kubernetes, right, which is using github for promotion And it's using a clean masters and masters always releasable It uses semantic versions of every release which we'll see in a moment when we do the demo So it's using github to do all the promotion. So we release everything once With a unique version number and then we move that package that helm chart and those docker images We move them from testing to stage into production through this github's process for tool We'll see a demo of how that works in a few minutes And I can show you a bit more behind the curtain what actually happens when all that kind of stuff happens So this is the big thing that jenki's x is trying to do for you It's trying to give you the best suite of tools all configured for you And then he tries to automate your ci and cd pipelines to do the right thing in the kubernetes We docker with helm with pipelines with github's So just to be clear though, um this this works jenki's x works when uh Basically that your whole uh Test of staging to production are all in kubernetes if if yours are this won't this doesn't handle the the case where you're Where your production or staging or these things are not on kubernetes? Yes, so jenki's x focuses purely on the kubernetes use case now. That's a pretty broad use use case, but Yes, I mean we're actually using jenki's x to build jenki's x and lots of pieces of jenki's x are like libraries and npm packages and docker images and stuff Which by themselves they don't go to staging and production So you can use jenki's x just as a way of releasing things But the the promotion side of things that the continuous delivery the the d new piece the kubernetes delivery That kind of assumes you're delivering the software somewhere And so for continuous delivery jenki's x assumes that's the kubernetes cluster of some kind We are working on some serverless stuff so that if you're doing serverless, but you're not using kubernetes directly so you're more Using something like lambda on amazon or Something like open fuzz or kubeless or something like that on kubernetes possibly We hope hope to have continuous delivery for those But we're not really addressing things like folks who want to deploy virtual machines onto Some infrastructures a service kind of provider or some vmware thingy or open stack or whatever. We're not really focusing on that Longer term we could look at extending jenki's x to handle things like terraform or Ansible or something like that for deployment side But we've not really looked at that yet because we We feel that kubernetes is going to be a pretty big deal. It's the preferred way of deploying containers on all the three major clouds already red hat promotes kubernetes as the way of deploying containers on red hat's operating system Like how foundry supports kubernetes as a preferred way of deploying kubernetes there pretty much everybody even docker supports it Everyone supports kubernetes now as the way of running containers. So we think for Most enterprises kubernetes is going to be the key where people are going to end up moving to deploying their software However, obviously jenkins can do anything you want it to do. So there's always jenkins if you're not using kubernetes jenkins x is very much kubernetes focused for now for now okay So the next thing is this all sounds kind of cold. So how do we get started? How do we set up a kubernetes thingy and how do we put jenkins x on it? So the first thing you need to do is get a binary tool called jx the jenkins x If you click that url or if you go to the jenkins x website and do a couple of clicks You can download the binary yourself. It's a fairly small binary and you just need to put it in your path It's a static living binary. So you you can install it on pretty much any operating system and it just kind of works And if you're using a mac you can use brew to install it if you prefer If you're on linux, you can copy and paste those commands if you prefer or you can just download the binary from the github website So once you've got this binary and you put the the binary on your path You then need to run one command to either install and create a cluster kubernetes cluster with jenkins x on it Or if you already happen to have a kubernetes cluster you can run the second command jx install So the first command is what we prefer you to use It's jx then create then cluster Then the next argument on the command line is the kind of cluster you want to make So aws is for amazon which uses cops under the covers to spin up your kubernetes cluster gke is for google Which is a really really nice way of running kubernetes right now the gke seems to be a little bit ahead of all of the public clouds today, but amazon and zero are catching them pretty quickly and then aks is if you're using a microsoft as your So all three of those commands look and feel pretty much the same and are very similar arguments and everything all that happens in those three commands is It uses your credentials to log into amazon google or zero spins up a kubernetes cluster Then it installs jenkins x on top of it. So if i'm it doesn't have a an account with any of those Do you guys support local running like with minicube or one of some of that? Yes, you can use minicube. We actually recommend you don't though to be honest just because So many people have trouble running minicube because minicube depends on virtualization on your machine It depends on bits of docker for mac or windows running People have so many issues running minicube on their laptops. It's kind of it's quite surprising how much issues people have with it So I I honestly recommend people try the public cloud. They all have free tiers Yes, you need to register. Yes, you may have to type in a credit card, but Literally you can spin up jenkins x on public cloud and tear it down after a day and Normally you're winning the free tier and it doesn't cost you a penny And using the public clouds you'll get used to Experiencing what it's really going to be like, right? I don't know about you at home. I've got pretty lousy wi-fi Doing docker bills and downloading docker images and stuff on my laptop takes like forever Whereas I can spin up a cluster on google in like two minutes And i'm ready to run right on my laptop It takes like an hour and then every docker bill takes like 20 minutes because it's downloading the internet onto my laptop So I we do really recommend if you can please try the public cloud The other thing about the public cloud is you can use webhooks on the public cloud so whenever you're doing like You're getting push events on your github repositories or whatever, right? Webhooks don't really work on minicubes. So you're right Behind your own firewall and then you can't exactly Exactly So we we do recommend if you can if you possibly can try the public cloud plus We kind of want you to use this for real So we kind of rather try it out in public cloud if you like it keep using it and your team now have the CINCD platform you can use for real No one's going to use your laptop for real for CINCD on the team, right? Sure, sure We we'd prefer you to go public cloud But hey minicubes an option if you don't mind waiting a very long time to try that Just because it downloads and stuff like that. Just because it downloads and things like that and the lack of the lack of webhooks Yes, so you can manually keep triggering build yourself Because there's no webhooks on minicubes. But yes, so create a cluster preferably On the public cloud, but if not, you know, it's fine use jx install If you or use minicube instead of in crate cluster, you can do jx crate cluster minicube That does kind of work if minicube works on your laptop Assuming assuming this thing that might be difficult Yes, exactly. All right, and there's all kinds of I mean lots of people have tried doing this with minicube and had loads of failures with it and it's always to do with Minicube and not to do with the local configuration basically Yes, like like which version of vagrant and hyperkit and docker for mac if you got any laptop and all those kind of things So all kinds of stuff can go wrong basically minicube. So, so please if you are trying minicube, but it doesn't work Don't assume that's jx doing bad things. It's just minicube is is tricky to run, right? Plus another thing some people don't have much memory in their laptops Um, what we kind of want to be doing is yeah Yeah, like we want to run quite a lot of stuff in kubernetes, right? We want to run Jenkins and builds and nexus and a bunch and then you actual apps A lot of people, you know, don't have a gigabyte some memory for you or whatever. So yes So edit so pick one of these that you like minicube is okay We but we'd rather use a public cloud. Okay I have a real quick video of what it looks like to install on google But they all pretty much look about the same. I'll just hit play. This is slightly sped up So you basically type jx create cluster gke I didn't want to do a live demo of this just because it's kind of boring Because it doesn't really do very much And it it takes like two to three minutes and you mostly sat there waiting for the cloud to spin these things up But basically you type in jx create cluster gke hit return It asks you for which nodes you wish to use in which regions it sets up something called an ingress Which is like a load balancer thingy Installs jx and all the various pieces then finally you can just see the last piece it it creates Two git repositories, which we'll come on to later when we do the demo Two git repositories which are used for github's which are your staging and your production environments git repository Which those bits will become clear when I start actually than a demo So what does it what does that give me right if I do if I create a cluster with jx What do you get what's each team gets so Each team basically gets their own development tools environment. So each team gets their own Jenkins master Which means you're not going to be sharing one Jenkins master across gazillion teams And whenever you go to the console if you go to the console You're not going to see every single build for every single Person in your entire company. You're just going to see your team's builds And each team gets an elastic pool of build pods In other words, whenever your team asks to do a build for whatever reason So you commit to master or you create a pull request or whatever And a build will be triggered automatically using kubernetes as a scheduler So rather than having like a fixed number of agents or whatever in Jenkins We use the kubernetes plugin for Jenkins under the covers so that each team has an elastic node pool of agents Which is limited only on on your The capacity of the kubernetes cluster basically so so long as you have capacity in kubernetes you can run a build Each team also gets their own monocular, which is like an app store thingy Then each team also gets their own staging environment and production environment And what that basically means is each team can release Apps to their staging environment Without asking for approval without, you know, raising tickets and getting different people to let them in So it means from a from a classic microservices world you can go quick Right each team is empowered to Build and deploy their own software in their own environments without impacting any other team Under the covers the staging and the production environment are using a feature of kubernetes called namespaces What namespaces mean is the staging namespace is completely isolated from the production namespace So that if you deploy a microservice or something to staging It can't see anything in production and they can't interfere with anything in production These environments are completely isolated so that you can't accidentally use the production database for your test or something When you drop the database in your test harness You don't accidentally do that on the production database and vice versa So it gives you that level of isolation between environments even within the same team And then each team's environments are obviously isolated from each other too. So if you have like 10 teams all shown in the same cluster You're not going to interfere with each other directly Okay Right, what I'll probably do now is do a live demo on in this case on google cloud I'll do a live demo of Creating an app on on Jenkins x and then we can walk through and see so you can see how it all works And how we can automate cncd. So I'm in a little command line shell here We have a little command jx is the command line tool If I just said jx I get a big list of there's gazillions of commands to type So i'm gonna on the interview for just a second and go for it and make that larger Okay, sure just a couple of pops up. There we go and maybe uh Scoot your screen. Well, okay Let me shrink you a little bit. Yeah, ask your uh, your terminal window down a little bit now. Yeah, drink that Make it a little bit smaller So it's called crazy pink Yeah, let me just shrink it a little bit one more So you can have the edge Yeah, there you go Is that okay? That'll work. Yeah. Oh, cool. Just let me make it a bit wider. So it doesn't wrap her horribly Yeah If this wraps horribly Yeah, it wraps a little bit horribly. Let me just make it slightly bigger. So it doesn't wrap too badly There we go. That doesn't look too bad. Okay, cool. So here's all my environments that I've got right now So the development environment is the first line. That's where the software tools run like Jenkins and those other things I have something called an edit environment, which is where um I'll show you that right at the end, but that's where I can develop I can I can work on things inside my id only I can see And then we have the team staging environment and team production environment I'll talk a little bit about what those are and how they work later. But first what we're going to do is create a new application Now we support a whole raft of different languages and libraries and frameworks and stuff I'm actually going to start with a spring boot application Just because spring boot is one of like the most popular framework of the java ecosystem If you're going to make a microservice these days, people typically tend to reach for spring boot first So we'll start with spring boot. So there's a command jx create spring And what this does under the covers is it uses something called start dot spring dot i o that the spring folks create So there's this spring start initializer, which is like a wizard that helps you spin up new spring boot microservices And I get to choose which programming language I wish to pick and I'll go with java Which group id do I should use I'll just use a default. What's the name of my app? Let's call it web in great But there we go There's a handy name now. It's asking us. This is a spring boot thing now It's asking us which spring boot dependencies do you want to include in this application? So I'm going to hit space to pick the actuator And then I'm going to search for the web stuff and I will add the web thing to So if I did the actuator and the web dependencies Oh, hang on one second. Excuse me a second. I was in the wrong name space All right, let me do that one more time Forget you saw that I was messing around on something before this was a and I forgot to switch back to the right name space Okay, try that again. Um, and we'll call this Do this one more time That one group artifact id we'll call this webinar James actuator and we'll do web. Yep. There we go Even that's better. Uh, and then it's going to ask do we want to initialize git? Yes, we do So this is the these are the files and it's creating Which is basically the same things that start.spring.io would create if you went through the start.spring.io website And what it's going to do is it's copied these files into my local file system in this directory Um, and now it's going to add them into a git repository, which is cool So it's added a git repository now it's run something called a build pack and it's added in a bunch of extra files to enable things like Building things with docker. There's a docker file. He's added things like a Jenkins file. I'll talk about these files in a moment Now it's asking, uh, where should we store this git repository? In a git server and so he's asking us on github. Where do you want to store it? And I'll store it in my personal account and it's going to use that repository name And so now what it's going to do is it's pushing doing a git push It's going to create a new repository on github and it's now doing a git push onto github Um, now if I type it copy and paste this command here It's going to copy and paste that string Well, that command's going to do is it's going to visualize what pipelines are happening in my project to the terminal So as things happen on that project, it will just echo to the screen So as I go through this demo, we'll see things happen on that terminal Okay, so what's happened to the covers? You see a pipeline is triggered So what we did Let me click on this URL so to make it cheating ready So this is the git repository I've just created. There we go So it's a normal git repository with the normal java stuff There's a pomex ml there with you know normal pomex ml stuff So that's a normal spring boot quick start which Is based on the canonical source code if I just look in the source This is a normal source code of a typical spring boot app, right? It's not terribly Exciting. It's just a vanilla empty But notice a few other things with a docker how we turn this source code into a docker image But that's the docker file which was generated for us from the build pack We also added a Jenkins file, which is a Declarity file to describe all the steps we need to make to run the ci and the cd pipeline We don't need to worry too much about the contents of that Then under the covers, we also created this charts folder Which is the helm charts for packaging the application up and deploying it onto kubernetes So we've added a bunch of these files into git for you. So we have a whole range of build packs which Automate building docker images and ci and cd pipelines for those kind of programming languages Whether it's python or ruby or go or no gs or gradle or maven or whatever it is We have all these different build packs that automate that stuff for you. So you don't need to worry about it anymore So when you say we here, you mean this is this is what jenkins x is doing for us is generating the charts It's generating all this the file generates jenkins file. Okay, exactly Now what it does is it looks in your project and goes do you have a docker file? What we do we'll use it. So it uses what you have But it will basically override it will add to these things if they're missing and just for a second could you Could you zoom in on? Command plus that Page there There we go Yeah, so we add all these files in for you And then we automate. So what we kind of did that I typed one command which created this Application so in one command jx create spring we created new git repository resource code inside it What it also did and the covers was it added some webhooks if I went to the settings And we look at webhooks. We'll see it's registered a webhook that says whenever we do a push on any of the branches trigger Jenkins And to do the ci in the cd I'm also noticed that it looks like we've started to do promotion. So that means we've built a release So if I click on the releases tab Notice we've got zero zero one. So we've tacked the source code at zero zero one Um, so we can now look at the exact version of code that is being built as zero zero one And we've generated some release notes the release notes aren't very interesting yet because we didn't do a whole lot in the source tree yet And so what we've done under the covers is the release pipeline has generated a new semantic version zero zero one The next time we do a push to master it's going to generate zero zero two and zero zero three It's using the git tags in the git repository to keep track of what the version has been released So that if ever we wish to jump from zero zero one to say one zero zero We just create a git tag called one zero zero and then the next release will be one zero zero one Another way you can force the version version numbering scheme Is you can edit your pomex amount and by default the version says zero zero one snapshot If you edit that to be One zero zero a snapshot then the next release will start with a one Right, so you can use the pomex ml or the package lesson to define What's your major minor version numbering? Otherwise it will just assume everything's a kind of a patch release, right So we've built the release we've versioned everything zero zero one. We've built a docker image We've built a helm chart and released those so zero zero one is now a mutable version As you can see here zero zero one is a mutable version that you can now use in any Kubernetes cluster anywhere But now we're doing this git ops thing where you see there's a pull request being generated. Let me click on that one I click on the pull request This is a pull request To promote zero zero one into the staging environment notice we're in a staging environment Jaystrak and test three is never my cluster right now. So that's the name of the git environment What this is doing is it's generating a pull request on the environment And don't worry too much about the details. This is the change required to add this new application into a helm chart But what's interesting is we're not copying the docker image and we're not copying the a blob of yamal or anything like that We're copying a reference the version number To a helm chart that we've released into this chart museum repository So this is a little bit like updating a pomex amount to add a dependency where you're adding a dependency by name and version So we're adding a new dependency We're being our james and we're adding zero zero one as a pull request Um, I think that pull request should hopefully merge very soon. What's happening is notice that there's a ci job is now running and the staging environment to validate Is this pull request on the staging environment valid? So it's running whatever our ci tests are for the staging environment to test things like Has this docker image being pushed to the docker registry? And is this helm chart available in the helm chart repository? You may wish to add other validations to your ci pipeline like Has all the docker images passed the any cv or mobility checks or whatever else you wish to check for You might also need to add say human approval like Do we need a couple of plus ones on this to say you can promote this version of this app into the staging environment? So That's something that you can add Yes, uh, so I didn't explain this yet. I should explain it a bit better. So There's a couple of different ways in which approvals can happen So by default each team gets a staging environment and a production environment And by default we set up promotions automatically on staging And manually on production. So whenever we do a release By default automatically we generate we promote that to the staging environment But we leave promotion to the Production environment as a manual step that you choose when you want to do that In addition because some of the covers githops is using git we use pull requests to generate everything And those pull requests depending on the config of your git repository You can have automatic or manual or approvals on the merge operations So you can set up, you know I need two plus ones before you can merge these pull requests And you can leave the merge side of the pull request to be a completely manual thing So you can choose when you're happy to merge those things and by default that's also automatic as well Okay Quick question. So if you click on details there on the That page, what does that what does that take us? So this takes us Yes to the pipeline UI for the staging environment. You can see that this is just about finished. This is finished. So it's gone green So if I reload this page, hopefully this should merge soon I'm not sure why it's not gone green on the on the GitHub yet Any moment now github should realize this is this is mergeable and it's because it's green. I can see it's green. Here we go. It's green So any moment now this should have been github.com's web page and it should merge I'm just going to click the merge button to be honest. Uh, just to speed this a little bit. Sure. Yeah It it should merge soon, but I'm just going to hit merge Well, no no no dimmer can can meet reality without something going just not quite how you expect It's okay. Yeah, I don't understand why that's not ingredient. That should be totally green Maybe github's off in a funny day. Okay. I'm just going to hit merge. Okay. I'm going to hit merge. So now this is merged So now because we've merged to master now the pipeline is running the update pipeline So you've merged master in In staging, staging git repo. Okay So now if I look at the staging repository and we look inside the uh, The requirements which are basically all the apps we should deploy. We'll see we we now save webinar james 001 is one of the apps So now another pipeline is going to run Which is going to apply these changes. So it's basically going to say to helm Please install all of these charts into the staging environment So once this pipeline completes here, which if I if I noodle around in my Jenkins Um, I'll be able to show you this pipeline running. Um There we can see there's the master branch Um, so anything now this on the staging. So this is the master branch on the staging environment So hopefully soonish this pipeline should treat along and then this should apply zero zero one into the staging environment Um, and again, usually this is a little snappier So hopefully that will kick kick off So limitation there is also that it's uh has to this is your kubernetes sort of your Number of nodes available that you can use so it might be that Yes, yes, and it does. Um, I on my cluster I have auto scaling enabled. So if I don't use my cluster for a while, it scales down the number of nodes And so it sometimes takes a little while for juggies extra warm-up as as kubernetes grows your cluster Can you again, plus uh, spend this a little bit just that means Not allowed to see but just a spinner. Oh, it's gone green. That's awesome. Okay That's been green. So hopefully this ui should update fairly soon. Uh, because it's gone green And if I go back to my other terminal There we go. So it's promoted. We can see it's promoted. So zero zero one is promoted and Oh, I wasn't scrolling the screen. That's what I did and there's the url. If I click on that url Um any minute now that's just starting up in kubernetes. So this is the url of the app that's running in the staging environment I'll give it a few moments to start up if I go to my other terminal Um, which I've hidden slightly under that window. I think we've got down here If I type jx get app, this will list me all of my apps in all of my environments So I might have five different apps. Some are in staging some in production. They're all different versions This little command jx get get app. Let's us see I've got zero zero one in the staging environment There's no pods running just yet. They should be soon as they've got the pods are up. So if I click on this url It should work. Yeah, there it is Now this doesn't look too impressive this page. This is this is actually a working page. You might not look it This is what a spring boot app generates by default because there's no code in it So it doesn't know how to uh render anything which is why it returns a four or four Um, what I'll do now is I'll show you how a pull request works And we'll do a pull request on this application and we'll see how pull requests work with jx And we'll hopefully make this page look something a little bit nicer Yeah, so this may not look like a lot, but we've gotten to a running app Admittedly with an error page. We've gotten to a running app from from basically just typing a couple commands, right? I mean, I mean really we type one command jx create string and then all this has happened, right? So it's gone through a quick merge. It's created repositories create the staging and uh production uh GitHub repositories for us and run a bunch of pipelines back and forth to make uh to To settle this up and now you're going to actually do a pull request Uh into master in in your regular repository and then see that go to state staging in that direction, right? Exactly. Exactly um, so I mean the other thing to to bear in mind is What's quite common in teams is they'll often sit there on the laptop hacking away for a while Until they're ready to show it to someone else and then we'll go. Oh, does this even work in kubernetes? I don't know. I haven't even tried yet What we're trying to do is make people go the other way like is before you even start coding We want to show you that your app works on kubernetes And then every change you make from then on should be making a pull request So that you can tell my new change still works in kubernetes, right? so we want to get people going to staging immediately and then every change you make from then on is Ready to go to production because you're testing it on staging, right? Right that and getting people into that habit of Delivering value properly on the cluster that you're going to run production on Rather than spending too much time on your laptop and not ever really testing on the real cluster So what when i'm just doing i've just created a little branch. I've used the command line get branch get check out Um, I've added a file if I do get add source new resources And get status there have I did a new file? And i'm going to do a git commit. I'll do everything on the command line because some people sometimes get a bit confused with ideas Let's do a fixed add better home page. There we go and git push And so i've done the git push on a branch So if I go back to this git repository And then we'll see that Github is saying would I like to make a pull request so i'll say oh, yeah, yeah, let's make a pull request Um, and let's submit a pull request So I've just used git here. I haven't done anything magical or weird. I've just done a normal git thing I've used the normal github web page to create a pull request Pretty much every first. Can you trust the different that the files changes here? Yes. Yes, just So I've literally just added a new html file. That's all I've done I've added a new file into this static one index. Yeah, right. Yeah, so I've just added the file Not the best diff you've ever seen. I should have maybe modified the file, but that's fine Of course, you're starting from empty empty space. So So notice that a pipeline is triggered because we're in a pull request Now if you remember 20 minutes ago We did the jx create spring and that created a new project Which is a spring root app and it set up the pipeline the pipeline handles pull requests and releases So whenever you submit a pull request on this project, it will run a pull request pipeline That validates is your code change value Do all your of your test pass and whatever kind of code quality Additions you wish to use One other thing we do though with Jenkins X if you do something called preview environments Oh, by the way, you'll notice on here. We'll see the pull request pipeline is is triggering on here, right? So we can see pull request pipelines as well as master branch pipelines So what's actually happening on the covers is we're running all the usual stuff. We're running the unit test checking all those paths Then what happens is we create a preview docker image of the code change before it's merged to master Right. So what we're going to do is we're going to create a preview docker image And then what we're going to do is spin up a preview environment to run that preview image Then we're going to spin all of that up in Kubernetes And then we're going to comment on the pull request with a link to where the app's running in the preview environment Now the reason we're doing this is is for various reasons Imagine you're building a web application It's quite common for teams to do things like code review as as pull requests come in We might want to get your peers to review your changes If you're changing a web application looking at the diff can be kind of confusing and Seeing a diff in a css file doesn't really give you an idea for how how well and new feature looks in an ID In a web browser So the idea behind preview environments is It does two things it lets you test the application that's run before you agree to merge it So if you're trying a new like buy it button You can let your team members review what it looks like before you agree to merge it So it gives you a place you can have a better discussion on the pull request about What you think of a feature by seeing it run Another thing you can do though the the preview environment acts as a check that goes Does this change actually still work in kubernetes? It's surprisingly easy to make a code change such as adding a new mandatory Adding a new mandatory environment variable or something like that That if that environment variable isn't also set inside the helm chart your app now fails in kubernetes So it adds an extra level of testing that goes does your app still work in kubernetes before you merge it, right? And if I look at the pull request now See how it's gone green because all the checks passed But also notice this comment here It says a pull request is built and is available in a preview environment Here and it gives you a link to a url Which is also the link that's in the screen here. It's the same url But i'll click it on the pull request because it's kind of cooler. So if i click that url we'll see Uh, well when the pod starts up, it's not quite started yet In a few moments or this pod is up and running because it adds the link before it's checked it's up. Oh, there we go. It's up So notice that in the url, we have the organization name the repository name pull request one Preview is that namespace or the place in which this runs So each pull request gets its own environment where we can access this application And this is basically the html page I just added. So this is the new homepage So we can verify. Yes, this does look good or at least it looks looks better than what we had before So we can approve this pull request after testing it out and we can say yes Before I do that as well, we can type jx get app And we can see these are the applications and we can do jx get preview and we can see these are all the pull requests So here's the pull request i've just done and that's where it's running So we can easily as a team modify and view all the pull requests that are happening and validate them And then when we're ready, we say merge to master Now we're merging that code change into the master branch And now hopefully another release pipeline will kick off. There we go. So version two is now on zero zero two is now And so This version is now going to get released and we deploy into the staging environment Okay, and that will be done via a pull request the staging environment that'll then exactly Exactly. Yeah, so so just to recap what we've just seen because it's maybe slightly confusing I've been flipping between a few different consoles and Looking at the browser a little bit We type from one command to create a brand new micro service and that immediately was released It was built and released with a semantic version and then promoted to the staging environment automatically through a pull request Then we created a pull request ourselves on that app That then Generates a preview environment so we can test out the application before we've merged it And then when we merge it we then release it was over two So we've basically automated ci and cd Without as a user you're having to worry about what the Jenkins file is or a docker file or a helm chart or even what kubernetes is A couple of other details about this Before we maybe go into more questions is the other thing to remember while Jenkins x is automating all of this for you Jenkins x isn't actually hiding anything So if you don't like the way the docker file is working, you can just edit the docker file It's stored in gig with the rest of your application source code Similarly, if you want to change the Jenkins pipeline to do different steps Different steps on the pipeline on the pull request pipeline or different steps as it's doing a release You can edit these Jenkins files Similarly, you can also edit the helm charts We're hoping that you don't really need to touch them really Um, and we're hoping that our build packs will be sophisticated enough for you to do everything you need without even seeing them or touching them But it's important to realize if you ever need to change them, they're right there in gate You can just go ahead and change them. So you get the advantage of of Having the having them spin up and be a basic opinion here. Here's the here's the best practice Uh, you get that that advantage, but you also get the control if you need it Exactly, you can take back the wheel as it were Yes, and one thing we're trying to do we expect as large organizations start to adoption context They might want to start to tweak what the what build packs are used You know, there may be corporate standards of exactly which versions of I don't know the jvn needs to be used or which Linux based images or whatever so we can imagine people tweaking the The build packs a little bit so that they could fork our getting the possibility with a build pack saying make a few modifications of you know Add some build packs remove some build packs modify them a little bit So the organization at an organization level people can modify those a little bit We obviously want to try and get as much reuse as we can throughout everyone in the industry So we want to avoid people just creating their own brand new build packs in isolation If ever you create the build pack for I don't know some really ancient version of websphere or something We'd love it if you contributed that back up to the community so we can all share it But if you really want to do anything local within your company, you know crack on and and it's all good So we hope to have a nice diverse rich community around the build packs themselves But worst case you can just modify them directly inside your application if you wish Got it Okay, so that's kind of c i and c d Um, so the well, I thought I gave a do a staging pull request for this for your change at this point Just be coming up. It's it should be coming up very very soon. I should show you by the way One common question I get whenever I show this these people say are you hiding the Jenkins console? The answer is well, no, but what we're trying to do is Make it easier for people to develop without necessarily having to keep going back to the Jenkins console all the time But you can do a command jx open Jenkins Which opens the Jenkins console Um, so if ever you want to just look in the Jenkins console, you can just do that and if I go into my pipeline, we'll see Uh, the master branch is turning along And it's now running the promote stage. So it's made the release And it's now running a command line called jx promote, which is the motion I should have mentioned this before actually if ever you want to manually promote something you can just type jx promote Mention the application name and the version you wish to promote and which environment you wish to promote it to and you can just Promote that application to the staging or the production environment on demand A jx promote can also promote any application. So whether it's an application you build yourself Like a custom microservice or it could be something that comes from the community like Prometheus or Elastic search or the farmer so we can do promotions using the jx command line ourselves outside of ci if we wish So that promotion is triggering long so we should see a pull request There we go. There's a pull request So what's interesting about this pull request is because now this is version zero zero two The diff is going to be much smaller now If you remember the first pull request we saw had three lines changed because we've never deployed the application before Now all we're doing is changing the version from zero zero one to zero zero two, right? Um, and then maybe in in two weeks from now If ever you decided you wish to roll back you can either use the jx promote command line to to roll back a version Or you could literally just do a pull request directly in the by the github web console You could literally just edit that version number back to zero zero one if you wish to submit your own pull request So we're letting Anyone can change the environment's git repository through normal gits through the github web's web console or whatever git provider you're using Or using the jx command line tool, which is nice So ui or command line your choice. Yes, exactly. You should whichever you like We are working on a whole bunch of ide plugins as well for vs code intelligent eclipse so that If you want to stay in your ide of choice, hopefully you can do more of that from just in your ide rather than having to You know use custom tools or you might have just noticed That pull request merged. So I think fairly soon we should have that release chugging in the Staging environment. I'll show you that in a moment. So merge to the master and the staging That means spins up on yes Kubernetes environment. Got it. Yes. So if we go back to oh I'm losing track of my windows. Here we go. So hopefully we'll see this is the staging environment's git repository in Jenkins And now the push to master pipeline is is the master branch pipeline is now running So hopefully that that should start soon. There we go. It started So hopefully in a few seconds that should be applied and then we'll have zeroes over two in staging Cool. There we go. That should happen really soon So while while this is just completing I want to change tags a little bit and just talk a little bit about Development in other words, how we use Jenkins X before you're ready to do a git commit Now one of the challenges with using the cloud is um Traditionally developers like to develop on their laptops, right? Developers have installed a version of java and make and maven and whatever and they've got all this stuff running on their laptop And they run everything on the laptop quite well One of the challenges there is that means you're always testing on a totally different environment to the one you're using for production and staging So we want to try to tempt developers into Using the same operating system the same container images the same tools As your ci and cd pipelines are using and your staging and production environments are using Right, so we want to try and encourage developers to use the thing that's as close to production as possible Because one thing that happens a lot is you use an old version of maven locally and then you are one in your ci and cd Or vice versa or using a different jvm different operating system You might think you have a horrendous threading there, but that bug only happens on windows And you're running on Linux in production. So it doesn't matter, right? So some slight environmental difference Exactly you're developing and where you're deploying, right? Yes, this happens all the time We've all heard the phrase it works on my laptop And what we want to do is go it works on the cloud first That's the thing that we care about more than anything else Running on your laptop should be a secondary thing that we should care less and less about What we should be caring more more about is does this run on in a docker container in kubernetes using You know the kubernetes networking stack and more balancer and certificates and secrets and As we start to build richer and richer micro services with things like service mesh and secrets and more balancers and Wiretaps and all of this stuff. We want to test more and more inside kubernetes and less and less on your laptop Also, by the way, that that just worked. So hopefully 002 is now running. There we go. 002 is now running in the staging environment as you can see And if I do jpeg get that we'll see 002 is there we go. 002 is now in the staging environment. So that's all awesome So yeah, so let's just talk about development for a second What I'm going to do is I'm going to switch to another shell What I put it here And I'm going to open an IDE. I'm going to use vscode for this demo We're working on an eclipse plugin and vscode and IntelliJ And vscode is the one that's slightly further ahead than all the others This application is a node gs application just to kind of show that we can work with node. We can work with go We can work with java with spring with maven with bridle with Pearl php python ruby rust all kinds of different stuff. So this is a very simple This is even simpler than the previous example. This is a very very simple node gs application That basically serves a file. It's that it's not going to Make you lots of money if you copy this microservice. This is not the world's most interesting microservice But i'm going to show you how you can develop in the cloud While using your laptop So a lot of us are all wedded to our desktop IDs, right? I can you I got a drop dropping in. Can you expand that just a little just if you want to see it? Yes Oh, now I need to know how to do that in vscode Uh, that's a good It's okay. It's okay if you don't don't have it. It's oh, yeah, uh, there it is Ding Even bigger. Is that big enough? Is that okay? That's fine. Yeah Yeah In this one, you don't have to notice too much about what what's actually on the screen But what i'm going to do is i'm opening up the command window in vscode And i'm going to do this dev pod command So there's there's something in juggies x called a dev pod So let me open the dev pod terminal and then while it's running i'll explain what's actually happening on the covers Uh, so what this is doing is Right, let me step back a little bit. Well, so I can explain this So cincd in Jenkins x uses something called um A build pod so it basically it's a pod which is you are a group of containers Which contains all the software tools for doing cncd So in a node gs application, that's going to have node and npm and git and cube ctl and a bunch of other kind of tools If you're doing meaven stuff, it will have meaven installed and java and git and various other things So we have these different docker images with all the tools you need to do cncd, right? And these are all versions managed by Jenkins x What the dev pod is is a pod or a container that you can use in development Which has all the same tools inside So inside here, I can type npm version and there I can see an npm is in this container and I can Type in git and I can type in cube ctl get pod and things like that So I can run all of these different tools which you need to work closely on cubanities if you wish, right? So we have basically a container here But this looks like it's running on my desktop But actually this terminal is running in a pod inside cubanities, right? So this dev pod is a terminal into a container in cubanities So inside this terminal I can run stuff in the cloud As if it's kind of on my laptop, right? So it feels like it's on my laptop, but it's not it's all running in the cloud So this means I don't have to install node myself I don't have to install git or make or any of the tools that cncd use, right? I can just share the same tools that cncd is using and maintaining The other thing it's doing if I do an LSAF is all the source code in this pod Now this source code Is synchronized from my desktop into the container in the cubanities cluster So what that means is I can edit everything on my laptop in my desktop id But I can compile and build and use it inside the cloud And let me do a quick example to show you what I mean We've got a little shortcut called watch and this watch command is going to build up the docket image And then deploy the application into my personal edit environment, right? Then it's going to watch the source code for changes Right, let me pop to another terminal and if I tap Not another window, let me go into another terminal. There we go. There we go If I type jxgitapp See we've now got in my local environment I've got a new URL for where this is running In my personal namespace So this is the application we're building right now, which I can browse it in my window What I'm also going to do is I'm going to run a little command To just curl this URL every couple of seconds and output the message from this html Okay, so I'm just curling this microservice in my personal namespace So while that curls me I'm going to go back to my editor And I'm going to edit the html page and I'm going to say hello world from Awesome I've typed the word awesome And you notice you see the build that's just rebuilt in this terminal down here What happened there is I typed in my IDE My IDE synchronized that source code change into the pod that's running in Kubernetes That then kicked off a docker build to rebuild that container image That then redeployed the helm chart And then any moment now this curl command should return the new text Hopefully once a little bounce. So there we go. Awesome So what I did is I did a build in the public cloud And then did a rebuild when I changed the code and it rebuilt in like five seconds Right, so so it took the service down brought it back up in the in the new exactly in a new phone So it means we can make any change to this source code whether it's the front end code the back end code The docker file the helm chart we can make any change we like and this will redeploy on the fly to my personal environment Quickly In a staging environment, but a personal environment that you're a personal environment So you get rapid feedback, right? You get rapid feedback as you're trying things out and playing around with this You get rapid feedback of your changes and that environment has not just your app But also it it's own in this case an npm so that if if you were basically publishing that that Uh That change If you npm release that it would go to the local server and be able to run from there, right? Yes, and so the idea of these development environments is this is where you can just run stuff Without before you're ready to do git commits and pull requests Right, so the idea is when you want to do a real preview environment with a Real public docker image and whatnot. That's when you do a pull request at this stage You're just doing a really quick rebuild and redeploy locally And then it's up to you. What else you want to use in your personal name space? You can link to other services from externally and Let me do another one really awesome And let's a couple of seconds later a rebuild there we go Now i've done about you when I build a docker image on my laptop. It takes a very long time, right? Inside the public cloud. This is fast. This is like a couple of seconds and it's already Redone and hopefully the shell I wasn't scrolling all to the bottom. There we go. I think it's really awesome already, right? So that was like what four seconds five seconds, right? Now what we're doing right now is actually a brute force every time you edit the source code We're actually rebuilding a docker image from scratch and redeploying in a new version of the bug with a new version of the image We're hoping to optimize this much quicker so that you can get even faster turnaround time So you type something for example with node You don't really need to rebuild a docker image use time You can literally just start node in watch mode and just copy The files into that container and you don't have to stop and start the application, right? So for different programming languages and frameworks, we can go much much faster than this But this is just a quicker example of how You can use the same tools that ci and cd is using in development So you don't have to worry about maintaining the same version of node or maven or java or whatever on your laptop But also you can do really fast incremental builds in the cloud Right, I never use docker on my laptop anymore. It's just way too slow an error pro Right, if ever you do a docker build you're pulling huge amounts of images down to your laptop And then you have to do a docker push which is really slow And you're pushing gazillions of bytes back up to the cloud, which is really slow If you do your docker builds inside the cloud, it's usually really really fast And then all you're doing is pushing the code back and forth Exactly just the code not not all the all the artifacts Exactly. Usually source code is much much smaller than docker images Right, usually Usually what one thing to be very careful of by the way is This the synchronization stuff that works inside Jenkins x we exclude node modules and the local target folder If ever you accidentally decide to synchronize node modules if you're a node person node modules can be massive correct what you do like npm install locally So you just got to be ever so slightly careful that you really are synchronizing the source code you want to sync Which is all defined by this magic file stignore which takes care of all that for you But yes, so there we go. So that's a quick test of doing development So what we really try and do with Jenkins x Is we're trying to optimize Every aspect of software development Our initial focus is very much about how do we automate cincd on kubernetes with Multiple environments with promotion through github's and how to automate all that stuff for you But we also very much want to help people in a development phase right so make it easy for people to test things build things Incrementally change their code And over time you want to increase this scope and make a much more powerful and rich environment So that people could basically be productive with the cloud developing software So that's pretty much all I was going to do from a demo perspective I was just going to leave some slides up Um with some links I was also going to ask what how it's some links Yeah, so if you want to try out Jenkins x at all in any way So you fade it out there Looks like james lost his sound Yep All right. Well, um, actually this is okay. Um main point here is uh, that's Jenkins x Here are some links for you the Jenkins x.io website to try it out Or Jenkins x.io community to give feedback. There's a slack channel and you can also We which is uh, what's here? And you can also go to the cloudbees website pages cloudbees.com k8x To get other information What else we also have uh, you can look for the Jenkins online meetups on meetup.com Oh, I uh, james just told me his laptop died. Um, I'm glad we still have this up then. Uh, in any case Thanks everyone for joining us. Uh, I will see you on the slack channel or on the jacons irc