 This is the Jenkins Google season of Docs office hours. It is the 21st of September one day before DevOps world and We live by the Jenkins code of conduct Markey you want to start us on the Calico to introduction? We are yes. Hi for those that don't know me. I'm Markey. I am one of the Google summer of Docs mentors along with Kristen Hello, everybody. I Will be doing a presentation today of Helm and Catechota. I am going to start sharing my screen Give me just a second. Let me know when everybody sees my screen. I Awesome so For anybody that see me do presentations, you know that I I often botch them So I'm gonna do my best as I work this I Did this I I set this deck up which I will share everybody once this is done I'll add it to the agenda I notes I Put this together, but I didn't actually think through talking through it So I'm going to do my best. This will be sort of an introduction to helm in Catechota We're gonna start off with the helm introduction What I would say and I'll leave this up to Z knob how you would do this if you want to ask questions as we go through it Or if you want to wait till the end either way, I'm fine. This is more of items for you Okay All right, so for the Jenkins for Jenkins and helm there is a chart that is Currently available. It is at the repo at Jenkins CI and then into the Jenkins chart This chart installs a Jenkins server which allows to spawn agents on Kubernetes use a light utilizing the Jenkins Kubernetes plug-in a First commit for this was in 2016. It was by Vic Englishus I think I said that correctly. He is from Google super awesome person It has had 300 contributions by 180 contributors We move this in November 13th on November 13th Let me start that over again migrated this to the Jenkins CI Jenkins repo as helm stable and the reason that we did that is because November 13th 2020 the current stable and Beta helm repos will be deprecated. So everybody is now having to move to their incubated repos Kind of an overview on how to install this chart You're going to need a Kubernetes cluster and then you'll also need helm version 3 if you use helm version 2 You'll need to change the chart a little there are differences between the two versions and that's not backwards compatible Once you have these two items Pre-requisites ready to go you will install this by doing a helm repo at the Jenkins repository and Then you just do a helm repo update and that will actually pull all of the metadata in and then you can just do a Helm install you can call it my release the actual as you see here It's called my release you can change that name to anything you want And then you'll just actually reference the repo and then the chart that you wish to install What do you get you get Jenkins running on Kubernetes which spawns agents on-demand Using this is what I put the awesome Kubernetes plugin. I am one of the maintainers of that plugin as well as jcast configuration support Which is the default configurations out of the box which can be customized via the values.yaml file I will go into a little bit more of what the values.yaml does but essentially on a high level That is where you will make all of your configurations that you want the helm chart to actually inherit Go ahead. I should have asked this process question up front Markey, do you want us to ask questions during or towards the end? What's what's your preference? It does not matter to me I would whatever Z not likes to do or whatever the everybody on the call would like to do doesn't matter to me Okay, so then then I had a I'll take the doesn't matter to me not allowed to ask a question now and the values that I'm used to running a jcast configuration with a Jenkins.yaml file that has my whole Jenkins configuration in that file or most of it How can you distinguish for me or will you later distinguish for me how it's different? The Jenkins.yaml that I use compared to this values.yaml You what you would need to do for the configuration piece of Jenkins is move your Jenkins.yaml file into the values.yaml Oh, okay, so it's that super easy. Yeah, it's super easy super. Thank you Okay, you're welcome and then you have persistent storage for your Jenkins underscore home I am not going to actually do a demo in here because it's very straightforward And what I'd like Znop to do is to really get an understanding of how you do this I would like you to try after we're done to do that an installation yourself and you can do this with MiniCube And then I'd like you to ping me directly Christine you have a good understanding Kubernetes as well as my understanding correct Yeah, yeah, I've used it in stuff first awesome, okay, so you Znop, you could be I've actually tried running a different Hello I can hear you but you're cutting out a little bit. So I thought I heard you say you tried running something Yes, Kelly. I tried running a lineman in hell but I had issues persistent Configuring something about permission Okay, so that probably means that you don't have a service setup either at the cluster level or the namespace level And excuse me not a service but a service account For whatever it is you're trying to install So in kubernetes, you have there's our there's things like our back and our back works in sort of two Uh two abstractions one is at the cluster level and that's called the cluster role binding or at the namespace level Which is called the role binding And you have to have services for whatever you're doing that's going to be utilizing permissions within one of those scopes I can show you how to how to do that we can You if you can maybe after this you can show me sort of what it is you're doing and I can walk you through that if it if that's possible Okay, thank you One of the things that I will note on when you run the demo is in order to view the logs You'll have to configure a log recorder And that on the last line of this slide you'll see you add that name Or dot c sanchez dot Jenkins kubernetes And you set it to all and you'll be able to pull specific logs just for the kubernetes plug-in So uh mark as you were saying about your jcast configuration Here's how you would actually do that how you can supply your jcast configuration file And this is just sort of an overview of what that would look like to add that to the values.yaml And how does the auto reload work? So the actual drink controller will Will do the trigger of the load it'll do the jcast reload for the jinkens ci S config It'll update the configurations and then watch for changes very straightforward And if you're familiar with how eddy's uh Station works this will seem very familiar to you So so for me as a jinkens administrator, that's quite impressive. So that says if I change The configuration is code definition the running jinkens will automatically reload it and that is correct Oh, that's cool. Okay, so there's yeah Oh very nice. Thank you. Okay, so the so for me as a jinkens administrator You've made my life easier that my act of the port of storing something to The the git repository where I track the configuration will or can automatically Upgrade that and install it into my Into my Kubernetes cluster. Cool. That is that is correct And so how does the auto reload work is the jinkens controller called the java java ops? It does a reload token. Uh, it takes the pod name and sets that as a variable It really died car, which is where the cast configuration is and then does a reload call to the api and essentially that's what it looks like right there Uh, I'm going to leave out, uh, how you can configure ingress and the reason being is is Documentations the knob will you'll definitely touch on this I felt that for this type of an introduction going into it's and and ingress objects and the ingress controller is a little bit more In-depth because of the different ways you could do it. You could use nginx. You could use hf proxy And it's a little bit involved. So I didn't want to like confuse you And so what is your setup look like when you do this initial helm configuration you off the containers Which will be in a knit container your jinkens controller and then they're reloading of the jcast config You'll have your two services, which will be for the ui and the agent They'll be your persistent storage and then you notice, uh, your rback here And this is the problem. I think you may be running into zenoff is the rback Which it has a schedule agent and then another job that does the watching for the configuration is code config map These all have service accounts that are allowed for This particular chart it allows this at the cluster level not the namespace level So from a permission standpoint when you allow things at the cluster level, it's a little bit more insecure But this is just to get somebody going I would say if you wanted to lock this down a little bit better You would move this to a namespace level which would be the cluster role and then the uh cluster binding cluster role I always forget those So understanding the jinkens azure permissions and kubernetes Which permissions do the agents have in the cluster the permissions of the service account of the agent pod? And this is again what I was talking about Which service accounts do they use nothing is nothing is specified default. That's uh Uh, that's more for security. You never want to just have default admin And then you can also specify a service account via the agent template and via pod specifications So in the kubernetes plugin you have the ability to use a pod template So you can also further specify what you'd like your agents to look like as those get spun up in the cluster And here's sort of an idea of what that pod specification would look like And then here's the pod agent template where you can set further configurations And how are permissions granted to a service as I was saying earlier? There's two types the role binding is permissions per namespace Where the cluster role binding is permission to the whole cluster and this picture sort of gives you an overview of what that looks like I think for the documentation it will be really to call this out because this is sometimes An item that is overlooked in the wider community The specifics between role bindings and cluster role bindings and how those are used a lot of times It's been my experience people will just use cluster role binding because they want to just they They don't want to have to deal with the security aspects of that But I think it would be important for us to call that out in documentation to steer people correctly Why it is a bad idea to run agents in a jinkins namespace Remember jinkins kubernetes plug-in meets kubernetes permissions as well as to be able to Do things with that service account and if that service account gets deleted. It's just super bad So I always have agents running. I I think it's a best practice to have Agents running in a different namespace other than where your jinkins is installed. So generally You'll see a couple online if you've seen the online talks that I've done. I will install the jinkins Controller in the jinkins namespace and then I will have agents spun up in an ordinary namespace when a job runs And this tells you a little bit more separate control and their resource assumption And this is again how you can do this using the helm chart Which is already out of the box set up to do And I have to so I didn't realize that what I did these slides I you can actually see the behind this picture the the helm command that you could use Which permissions does the helm chart grant none your best You'll know best which permissions you need. That's why we don't just sort of grant cart Admin root access So I think that's best And if I'm going too fast and you have any questions, please stop me Here we have what we've called an agent namespace which has a separate controller in the agent pod These are allowing limited resources only on the agent namespace and that's better for secure And so resource consumption of the cluster overall So okay, so can you back up just so so this one the idea is that As The Jenkins will scale up and down the number of agents it uses and by putting the agents in a separate namespace name space from the From the Jenkins controller from the Jenkins master That that allows me greater safety or greater Ability to say no you can't use Can you can you elaborate a little on this agent namespace? Yeah, so so what this does is two things one it gives you security Because now you're not commingling everything in one namespace and if somehow an aid Let me take a step back agents have the ability To be ephemeral meaning they can spin up for the life of the job and then tear down Or if you want an agent to be long lived, uh that Long lived the reason that you want to have these in a separate namespace is it gives you the ability For secure Jobs not to run in any part of the agent namespace the second part, which I think is actually a little bit more Better than than the security aspect is the resource consumption A lot of times clusters are under net ease clusters are a heavy load already Especially with the Jenkins controller and all the jobs that may be there and sometimes jobs do not get configured correctly So you may have jobs that are Retaining large logs. I've seen that at some companies and you may take the cluster down to be able to Better control resources. You can separate the agents out into their own namespace This allows you to control that namespace as opposed to controlling the Jenkins namespace Uh as a whole which is where the controller will live Did that answer your question mark It did thank you I had missed the I had missed the subtlety of persistent agents And thus the additional safety that you gain by keeping them in a namespace, which is independent of The the controller of the Jenkins controller or Jenkins masters namespace. Yeah I I found that it was better to have the separate namespace not only For security resources, but let's say I want to just be Let's say I have log aggregation happening in the kubernetes cluster And I'm really going to the api to look through the api logs If I have to look through the full Jenkins namespace and weed out all the controller logs and all that it becomes super difficult This makes it a lot easier Thank you Plug-in installation. How does it work possible improvements? Uh, I'm not going to go too deep into that, but Some parts I will say is how to configure your credentials There is a kubernetes credential provider plug-in in the helm chart. You only and now I see you have a typo in here You only need to set the our back read secrets to true and it will create the necessary roles and role bindings Again, this is at a namespace level And I've put a link to where that is in the actual code If for that helm chart these are just some things that kind of thinking through So any questions about the the helm piece? This is at the end before we go into the catechota introduction That was a lot zina vine I know And I I know you may have a lot more questions And I want you to be able to use kristin and I as resources at any point if you run into any problems Um, okay. Thank you. So for now, I think the only problem I still have still remains um the permissions So I'll um come keep more with you after The um two sessions so I can explain better Where I'm at what I've been able to do and the issues I'm experiencing Okay, really quick question when you you're installing the the jinkens helm chart and that's what you're running into the problem No, I've installed the helm helm chart and I used it to install Jenkins So the issue I'm having is with um running Jenkins. The container keeps crashing So That's where I'm having the issue Oh, so the container is crashing. It's not a permissions issue that the actual container is crashing um, so when I try to get the um to get Exec into the pod or the container is complained about permissions then um Also, sorry. I'm trying to go Let's do this. Let's go to the katakota uh introduction that I have is it is a little bit shorter Maybe what we could do is you we could do a screen share once we're done with the meeting Maybe we could stay on a little bit Yeah, and then I can and and kristin if you if you have time would love to your input as well Thank you So katakota What is katakota katakota? It's an interactive learning and training platform for software developers Basically, it gives you a new environment without the need to install any required components by themselves Katakota provides isolation for uh, so you're basically you can spin up your own environment and learn about whatever the subject matter is And the subject matter is called, uh sessions It has an integrated editor that allows you to uh experiment with creating configurations updating or exploring sample applications It helps gain a deeper understanding of how the technology can be used For example, a user can copy a snippet of code like I've done here into the editor And run that against a virtual environment The interactive environments can be embedded into websites or documents That allow us to be able to maintain a consistent look and feel I added a link on here on how to do that. I thought that would be really beneficial for you. Uh, zina You can create your you can create your own content and uh scenarios And I'll go in a little bit more on how to do that. I've given a link here to the katakota docs so At a high level right now currently there's a kubernetes scenario. There's also a jinkins scenario I'm currently working on a jinkins and kubernetes Scenario and my hope is is that will be released the first week of october And then we can get that into the into the documents that you have what I'd like to do is also Uh, when I create the repo I uh, excuse me when I create The the release I'd like to be able to give you access to that zina And maybe possibly you become one of the maintainers of that I think it would be a really good opportunity to Get more involved in in actual that code releases and it's not super difficult to maintain And I would help you all the way through Thank you so much. That sounds really great That is my presentation for this. Uh, do you have any questions? No, no it's does anybody else have any questions Kristen did you think it was informative enough or was it too high level? I think it was a good a high level um discussion I maybe it sounds like now that we've got some free time to start looking at maybe particular problems that You know zina have you've run into? Yeah, especially the skidding started really It's always good to have a good day. I never see a problem having like a baseline Exactly working working from there. So at least we can at least start with the same common knowledge and then Exactly, I didn't want to go too in-depth because I didn't want it to be like over technical in the sense that You may just get lost in all the details. It's easy. I always found it's easier to To have a little bit of a guide and then do something hands on myself right Zina did you want to maybe share your screen so we could see what the error was that your pod was getting? Yeah Markey while she's getting set up to share her screen I had the mental model that katakota was Largely a training development training delivery kind of platform Oh, that looks like she's already got her screen shared. So let's delay my question until later Well, I'm still waiting for her screen to to show up I can't say mark that you are correct. It is it is more of a training Tool, but the beauty with this training tool is you're able to You're able to write scenarios really with ease And you can take scenarios and combine them by kubernetes and jinkins. It just makes it easier to deliver training content Ah, okay. Thank you. Thanks for the clarity. Thank you Zina up. Can you do me a favor? Can you clear your screen? Okay Can you type in uh cube ctl? get pods space dash dash dash names, uh, no no space on this last dash Okay No, no, no, you just do all dash dash dash all dash namespaces Dash Make take the space off bring the It should be all one word, right? So all dash dash all dash namespaces There you go and hit enter. So marky what this is doing is it's querying her kubernetes cluster Asking it to list The pods in all the namespaces in the in the cluster. That's correct and the reason that I do this is Rather than saying, okay, what namespaces jinkins in it just helps me sort of go a little faster So zina up. Can you do cube ct? So you you just broke up Mark you may need to say that again And I'm charmed by seeing crash loop back off. So I think that supports what yeah, what so You know cubes ttl get pods all namespaces allows me to see all of the namespaces And get quicker to like what I'm looking at rather than think it's in the jinkins namespace. This helps me Get get there a lot faster. Uh, can you backspace two times zina up? and do a dash n space jinkins logs log s And can you copy that jinkins pod name and then paste it? And then enter Can you do, uh, can you do an up arrow? Can you take out logs and if you just do option backspace, it'll move you back faster And then type in Uh, describe And then, uh, can you type in describe and then enter? Can you scroll up a little hit enter? Can you do, uh, uh, up arrow? Can you backspace all the way to describe? Taking out describe and then just do get s a and s a is abbreviation for service account Okay, okay And you can hit enter Can you now do, uh, up arrow space jinkins space my, uh, dash o space yaml What does dash o space yaml do? So that now what I'm asking, uh, this to do is the option flag and then to show me the yaml output of that jinkins service account And my apology sometimes I forget to actually say like what something means and I will get better at that So what we see here is that service account's actual yaml file Okay, zinup. Can you do cube ctl git And this is going to be all one word cluster roll bindings Yes, and hit enter Can you do, uh, up arrow? Can you go back to before git and type in a dash n space jinkins? Uh, no before the before git So it should be cube ctl dash n jinkins git And then can you take off the word bindings and just have the cluster roll and enter So the first one that we looked at the cluster roll bindings was the cluster level permissions Now we're looking at the namespace level permissions and can you scroll up? Okay, so I think the problem here is that the service account that you have jinkins is not tied to any service, uh, excuse me to either a cluster roll or a cluster roll binding Okay So Give me a second And I have to just look at something Trying to see if I have a good example to show you I'm gonna can I share my screen real quick just to give you a An idea of what you'll have to change to fix this Okay Tell me when you can see my screen Yeah, I can see it So in this example that I have here, this is a service account and its name is jinkins In here you can see that I've uh I've created cluster rolls and then here I have a cluster roll binding Okay In my cluster roll binding you can see that in the api group I've actually linked the service account in here And I don't think you have that linkage happening in your cluster roll bindings and that's what's stopping it So just to reiterate what I said you have an account the service account already created and it's called jinkins But you don't have that service account linked To the actual permissions in the cluster roll binding Now remember a cluster roll binding means this is at the actual cluster level and not namespace level if it were namespace level it would be a roll binding Okay So what you'll need to do is using the command that I I gave which is the cube ctl Get cluster roll bindings Okay, you're going to want to create a new cluster roll binding That's called jinkins that ties This service account that you have already to that and that will allow the permissions And you can see also This is the cluster roll And the cluster roll these are all of the actual Commands that are allowed and my guess is is your help chart is trying to execute one of these commands But because the linkage is not there It won't work. I'm going to send you the link to this repo I'll put it I'll put it in the chat right now. Give me just a second I put that in the chat what I would suggest doing is creating a Cluster roll and a cluster roll binding based off exactly what I have here And it should automatically link your already existing service account and then your permissions should work Okay, thank you. I'll try that After this session and reach out if I have any further issues Yeah, totally if you get stuck or anything, please don't hesitate at any hour of the day I I usually keep strange hours To ping me. I'm always online. Well, most of the time Thank you You're welcome That was all for me mark Excellent. Thanks. So risha, would you do you want to Bring any other topics up for our session today? um the blog posts um, I've worked on most of the comments um From the mentors what um, they're just a few of them Which I don't understand clearly. I initiated conversation with christine already on one of them um, but I think overall I've worked on the comments from the mentors Excellent. So so did was that blog post initially created in a google doc? And so now you're at the stage of you think you're ready to transform it into ASCII doc and submit it Yes Excellent. Great. Okay I apologize. I'm not sure if I've reviewed it, but if christine's reviewed it and if marky's reviewed it, that's great I will certainly see it during the the The uh code review in in the poll request to jenkins.io Okay Yeah, so that's all for me Okay, and I heard uh, I had a heard a good suggestion earlier today in a congress session with olag He liked the idea that you were that you started last week of hosting the The initial documentation of google doc for people's comments He suggested you might also want to share a link to that google doc To the jenkins doc's mailing list so that we could encourage other people to help us Review it and comment on it even before it becomes a poll request to jenkins.io Okay, then please Zina, please make sure if you do that that you set that link to only allow comments do not allow editor access Yes, yes, please Yes, we do not need to be spammed in your document. Absolutely We've had that problem already. So I'm I remember well Well, there are even times when I wonder if I want to turn off comments because of obnoxious comments So for sure, it's great to send a link to it. And if you're willing to allow public comments initially That's a real positive if someone starts becoming obnoxious or annoying or unacceptable You're welcome to disable comments too and just make them have to send their comment to the email list Okay, thank you. So, um, I'll try and resolve the issue. I'm having um, based on my market suggestions once I'm done with that, I'll Push the link to the mailing list. I'll make the necessary corrections in the doc and push it to the mailing list Excellent. Thank you I will add these slides to the uh, to the uh, the meeting agenda notes shortly Um, also, I wanted to ask a question. I'm marky. Is it okay if I used um content from this session in the documentation? Yeah, I'm fine with that as long as you're okay with it. I find in as long as christin's okay with it not a problem Okay Okay, that's also good Great. Thank you. So zina one of the things that would help me Is if we we just keep notes in the uh in the google doc for this office hours I'm I'm typing some things in there now as a placeholder But that way I have a plate. We have a place to embed marky's marky's hyperlink. So marky overview of kata kota and helm and includes diagnosis and Investigation Mark, can you link the the knock that does excuse me the doc for this uh For this office hours in in in our chat here. You bet. Yeah It's right there. I think at least that's the one where I've been editing suggestion Oh, you you had asked about let's see it was uh blog post pull request a blog post reviews done ready for a pull request and then the other was documentation share the link doc's mailing list and reminder allow comments but don't allow edit for public You don't really want everybody to be corrupting or disturbing your document I also linked the slide deck that I used for this in the notes Excellent. Thank you marky Just for my safety, that's yes. Okay, it is it's publicly available. Great. Okay Thank you anything else seen out from you today I'm nothing else from me All right. Thank you. So we'll next meet on thursday um Let's see is thursday thursday it may have a collision with devops world is or at least one of the other mentors available to To attend on thursday. I think i'm available, but i'll have to check double check my my schedule If you're not, I think I I can or christin if you want to whichever Or I think I should be here. That's not a big Don't have them like not hosting any devops world sessions or Okay, excellent. So we've got at least one mentor available. So thursday's meeting is we'll go as planned excellent Thanks everyone. Thank you very much recording a link to the recording will be posted in about an hour after I get it Archived and placed on youtube. Thank you. Thank you Have a great week everybody YouTube