 Everyone out there in internet land welcome to another episode of the red hat open shifts twitch stream I am very very very tired this morning, but I am very very very excited to welcome the one and only Christian Hernandez Oh the one and only yes. Oh one and only that's what I'm calling you from now on nice So Christian I are gonna be talking about get-offs today, and I actually have the Honorable position of starting first because Get-offs and dev ops kind of have a great tie-in, and I'm kind of known for dev ops But first before we get any further let me introduce myself and let Christian introduce himself. I'll go first I'm Chris short. I work at redhead as a technical marketing manager I've been on the Ansible team and now I'm on the open shift team, and I have done dev ops since 2011 and I'm very happy to be here today with y'all. Thank you Christian. Yeah, so as Chris short said I'm the one and only Christian Hernandez. I'm also a technical marketing manager. I'm actually on Chris's team and I I've been oh dev ops has been around for a while the buzzword has been around for a while But yeah, like it was around around 2010 2011. Yeah When I really started getting into it. I come from an ops background So I came from the other side usually dev ops starts from the dev side and pull the ops in I actually started from the ops side and I thought hey this this is actually the way we need to do things so So and I'm kind of the same way my background before joining civilian life was very much in the ops world You know getting packets from one place to the other be it from computer over satellite to other computer kind of thing right like so for me dev ops was kind of like oh I just took these things and kind of plug them into the right place and I actually have a talk that I do That's titled what the military talking about dev ops Because a lot of the practices that we did like Kanban boards We didn't call in Kanban boards, but we we had them. Yeah, they were our, you know regular status boards and what you could do and any given facility is walk around to the status boards You know, they were dry erase boards at the time, but they eventually upgraded to you know TVs and you could get a general idea of what was going on in the facility And be that facility be it like a sat-com facility and Intel processing facility Whatever was happening you could see it happening and you had an understanding live And that to me giving giving people that access to the The status was huge right and so you know eventually we saw everybody has a status page now right like everybody Needs to get out to their customers exactly what's going on With their service so that I think is a very very, you know like a Steamed thing that's come out of dev ops, right? Yeah, but that's let's kind of get back on track here Yep, and you know, so you mentioned dev ops was buzzword kind of 2010 2011 So I have an official definition of what is dev ops, right? Like I wrote this years ago Actually last year Open the dictionary. Yeah. Yeah, so but like here's the thing part of the reason I wrote in this definition is because Everyone left Ghent in Belgium. So the first dev ops days was in Ghent, Belgium In 2009 and everyone left Ghent knowing That dev ops didn't need a definition It would define itself as what dev ops was Yep Fast forward 11 years We still really don't have what definition of what dev ops is There's still a lot of confusion about what dev ops is or does or is it just culturey stuff? Or is it just tooling or is it, you know something in the middle? So, you know, my definition of dev ops was very very Scientific right like dev ops is the professional practice of frequent continued and iterative improvements through measurable changes The goal of which is to become a high-velocity organization thus improving business outcomes now That's very very scientific and there's a lot of background behind that and I can share the link to this and the channel real quick But I don't necessarily like my definition as much as I like someone else's Emily Freeman wrote a book Dev ops for dummies actually have a copy of it Just approved everyone. I actually have a copy of it right here. Not a print out like Jason's not no I didn't print out a page right like I have an actually have the book I have the book and actually I think Emily signed it too. Oh, look at that. Yeah, she even signed it. So nice But her definition I think is way way better and it's way simpler So I was looking for a video before this call and earlier this morning She did a video about the definition of dev ops and it was illustrated by you know, the if you've ever been to a dev ops days You've seen the The the live illustration type stuff. Yes. Yeah, the white board thing. Yeah, like it's it's The the person drawing like what the person is talking about on the board at the same time So Emily got together with that person and made a wonderful video and I can't find it to save my life Like I can't find on YouTube. I can't find anywhere But it's it's like four minutes of what the definition of dev ops is and it's very simple and it's very very Embraceable but the the the the industry wide definition does not exist, right? Like I think dev ops The way Emily is to find it is the right way But the industry has kind of said well, now there's dev ops tools and now there's dev ops You know orgs and now there's dev ops teams and there's all these different ways to do dev ops and that's fine but You know check out Emily's book check out Emily freeman.io her website She's got a lot of good stuff on dev ops and what dev ops is and so forth so on But with that being said what is dev ops dev ops to me is Measuring What you're doing what changes you're making to systems and then? Aligning those changes to goals, right? So if our goal is to a increase revenue Well, what changes can we make? To decrease the bottom line, right? Like if it is Consolidating around Better solutions, right? Like if you look at dimming and talking about like Losing your supply chains, you know like trimming those down. Yeah, or you know any kind of practice like that Any kind of goal that you need like align your IT organization towards that goal and become a learning Organization that does things proactively as opposed to react if that makes sense. Yeah. Yeah So I think that is where get ops comes in So if you say okay dev ops is all these things and it's and it's you know It's pushing towards this high-velocity organization and it is a cultural thing and it is process changes And there is some tooling mixed in let's just go ahead and say it's like people process and tools for now What you know what we're kind of missing from the the dev ops toolkit is like a great way to do dev ops to implement dev ops and Yeah, it's it's kind of like we have we have the Philosophy right we have the philosophy down people are starting to get it the philosophy is out there So what does the practice look like right? So what what how do we you know? We have we have what we think it is we have the philosophy behind it So now okay, well then now now now we're at the tooling phase, right? Like what what's what's the tooling behind it now? Like how do we get to the point where we feel like we have started down the road of dev ops, right? Like mm-hmm. We're doing the dev ops. We're how do we get there? So we know we need to embrace some cultural changes in our organization to think a little differently but we need a process and we need a set of tools to do that and Get ops in my opinion is Also a practice or a way of doing things that Forces you to choose certain tooling and along with that and forces a process that is very dev ops aligned if that makes sense so when you look at get ops as You know get is a single source of truth everything you do in get triggers some kind of action be it in dev stage or production potentially and Then having that sole source of truth being the way things become automated and then go out to their various environments To me That sounds like the holy grail of dev ops, right? Like I check in something one place right like my sole source of truth is get and Then things happen as a result of me checking that and based off programmatic things that I've said at the beginning Let's do this So that's where I think Christian I think you're gonna come in and kind of maybe show us the get ops way of doing things and explaining that a little bit better to us Yeah, so Yeah, so by the way great great introduction Chris. I think I think that was a Perfect overview of you know dev Dev ops and how you know in staying weighing that into into get ops because yeah So you have get which is like you said a single source of truth. It also Gives you since you're using get it also gives you a convenient audit trail of everything that's going on right so you're not only making changes and continuously Delivering those changes you are you have an audit trail of what's been going on in your environment, right? You have unique IDs for that each step in that audit trail. Correct. Yeah, exactly. Yeah, you have a unique ID You knew exactly who did what at what time change advisory board. This is what happened Exactly here was the state of our environment at such a such date at such a such time here and This is why right because you have the you have changed You have the review right so you have the review that goes on and get you have all these tools already built in to get and And the idea was it's like well, why don't we just like leverage everything that's already built into get like all these tools are already built into get so And now And now you have the the the tooling right that's that's starting to bubble up around it, right? So I think a lot of the tools Nowadays well before we had things like like puppet right like puppet and chef And then later on and zable came along right and has the idea of you have You have a declarative approach, right? It's like this is this is this is what I what I want My end state to look like right. This is my end state. Yes. Let's declare what this is. Yeah, this yeah I want I don't want this is the idea of like the set of instructions versus a set of facts, right? I want I would this is how I want my cluster look like this. I want my deployment to look like and Having those tools Create that change right so as we talked about the declarative approach, right? That kind of lends itself to like the kubernetes framework right the how kubernetes does things right because kubernetes kubernetes does exactly that right it has the Declarative approach and it has that control loop right has the the you know is do you know? Is there a pod running yes or no if it is if it is it does nothing and if it does it reconciles that right? So there's that reconciliation, right? And so And so yeah, so so I think This is a Good segue to show quickly. I guess what I have here sure in terms of demo Doesn't have to be fast or anything, but you know walk us through That's right. We have some time no pressure here. Do I have I always need to choose the right? See desktop whiteboard. Oh, yeah, yeah the zoom choose the pick the right. Oh am I right? Oh, am I about to show the world something I don't want to show them. Yeah, I don't know. Yeah, here we go All right, I see a wonderful wonderful open shift Dashboard cool cool cool. So So yeah, so I have this I have this repo right so it's as with everything Get off starts with get yet. Yeah, it is the foundation right like let's not forget like yeah So and the reason that is is because get has become the de facto Software version control for software, right? Like we had back in the day. We had subversion and CVS and Curial on all these things and get has kind of coalesced to become the number one You know, everyone has a get repo somewhere kind of thing, right? So that's why using get as a foundation makes the most sense. It is the common tool amongst everything So actually we have a question in the chat That's not actually a good thing to to bring up is like well the first time do you when you set up a cluster? Do you export all configurations to be supported to get the answers? Yes, right? So the idea is yes, like you want to one of the one of the biggest advantages to get ops is so not only the You get the whole dev ops experience and you get the audit trail and all that but you get to quickly recover right so you quickly recover from From like catastrophic failure, right because everything's already defined, right? So you just redeploy that and and there's questions about Configurable CRDs and options and I'm gonna go through that. Yeah. Yeah. Yeah, I'm gonna go through that stuff But but just in general. Yeah, you basically export All your configurations right you explore all the configurations into into get so But coming back around here Yeah, you start here would get and It's just so I have a simple example of this application, right? I Have a deployment definition, right? So the deployment definition. Let's see. Let's take a quick look at it It may make my screen a little bigger Yeah, thank you. Yeah, so that way You don't want to There we go So yeah, I have this deployment figured a simple It's very simple deployment definition. I have an image of deploying Setting of the replicas the labels just kind of anything norm what you would normally would do for any Cooper-Daddy's cluster and This application here takes a variable, right? So this will This will this this is this is kind of an image I use for like blue-green deployments and I just and Basically draws either a blue square or green square, right? And that's that's that takes it in as a variable So for this particular repo, I have one my some I variable to blue right because I want to see a blue box mm-hmm and Then the namespace as well, right? So the namespace, you know, I need to have a definition for the namespace So literally everything needed everything is defined. Everything is defined here. I have Service right same thing service nothing special here. I am Exporting the port port 8080s where my applications are sitting on a simple web app and a route, right? So for a since it's an open ship cluster I'm setting up a route Right, you can set this to it's an ingress point if you're using ingress definition, whatever So this is essentially this is a minimum configuration, right? You kind of right? It gets us our app in the cluster off and running. Yep. Yeah, so by I went back back to the So this is nothing special in that I can just like feed this into Any Kubernetes cluster and it'll it'll pretty much like run right like right like you gotta apply these files right now Yeah, yeah, exactly But then there's the idea of like, okay. Well, like how do I? How do I make sure it's It's the version I want it to be how how do I how do I make sure no one goes in and makes a change to my deployment I like changes the the scale You know because once you change the scale Then, you know, it's different than my what I defined in my git repository, right? So this this is the the idea of drift like how do I detect drift and And how do I how do I reconcile that right because? Kubernetes job is only to reconcile what what you tell it to but what if what you So what if someone else tells it to do something right? It'll do it Right like it'll do it and what if you don't want it to do that, right? What if you always wanted to sync there? So Um So I have here a cluster right and one of the many tools that are out there. There's there's a lot of tools Tools to do what they call a continuous delivery, right? I want to be able to deliver my application So let's take a step back just in case Continuous delivery is the idea of our changes happen as often as we need to make them and And then We hit some kind of approval step to like push them out to prod, right? Yeah, whereas continuous integration is Okay, all these steps are here and available to get applied. I'm just gonna automatically approve like once they're approved I'm automatically pushing them out the door Like I think that's the that's the disconnect. There's a lot more to it But yeah, well, that's why it's like like you have like the ci right is where where I am iterating continuously, right? I'm just like, you know making changes making changes making changes like you said um delivery is actually like, you know Once once you once you want to make an update, right? Once you make an update to your application, you can make as many updates as you want But that doesn't mean you're delivering all those updates all at once, right? Right. Um, that's why they're using You use you always hear ci cd. I think kind of combined, right? Yeah, but some organizations embrace one or the other right like kind of simplify things for them or based off, you know Whatever compliance reasons or regulatory requirements. They have, you know, a lot of organizations will embrace cd Uh, and and just be like, okay. Here's our, you know, final review step Here's a release process go, you know, and then the checks the box That process checks the boxes for their regulatory requirements Exactly exactly. Um, so, you know before we dive Too deep into this there's a question in chat and I want to address it and we can pick it up later Maybe but you know, is there a good discovery mechanism for what's already in your cluster? Or what's already deployed right now, right? Like How would I get all that into my get-ups like pipeline repo to begin with? Yeah, so there's um Um, there's actually a really good article written by I believe the folks over at weave works about how to how to export um, essentially Your cluster right and it's and it's and it's just using on native tools. Um, You know, if I can go over to Like my cli and I do like You know, like oc get like routes, right? Let's just take an example. This is oops. I do a dash a for all namespace. So this is my routes for all namespaces Um, I can just do dash o yaml Right and this is and this is in yaml format all my routes From all my namespaces in my entire cluster. It's everything. It's everything, right? So, um, and and the article actually goes Goes through pretty pretty well. Let me try and find it while you're going through that. Um, yeah So that way you can paste that into uh paste it in the chat Um, it goes through how you have to prune some of this, right? So like the creation time spent a tiny Yeah, you have to you have to prune some of this stuff because some of these things um Won't apply right because when you create a new cluster or when you're restoring a cluster or whatever Some of those things aren't aren't going to apply and some of these things are, um, injected into the, um The the the manifest either by kubernetes or some sort of muting webhook, right? Like if you're using something like istio or something, um It things get injected that you don't really need to define that it's just um It's it's just a plot. Um assumed, right? This is my kubernetes is gonna apply some of this and you can actually see that um in my Uh definition, right? Like if I go to here my route, um, There's uh, I set my creation time stamp to null right because it's i'm not gonna You know, I'm not gonna set a creation time. So I'm that's gonna be set by the Runtime cluster itself. So um and You can go through that and you essentially go through that at um Object by object, right? Um, there's There's no convenient way. There's no like, you know, um, you know, like cube ctl export cluster, you know Yeah, wouldn't that be great. Oh, yeah, I'm like, give me all the ammo out of this cluster. Yeah, like dump everything Yeah, give me everything. Yes. I'm gonna submit a pull request. Yeah, I don't see you're right. Exactly. Yeah, I'm gonna see Yeah, I'll plus yeah. Yeah, give me the give me the link. I'll thumbs up that Because because this would be actually be cool It would be cool. I mean to be honest with you, it would be useful to me, right? Like if I were a new Employee at a company, you know, we're like running clusters, right? Like I would love to see this and be like I know what red flags look like. Let me go find those. I know what, you know, like Let me find all those services that I need to make sure are being, you know, logged and, you know Protected correctly and evaluated for security reasons, right? And you know, I would want to see all that stuff, right? And that, you know, my not my normal sys admin toolkit is nmap Right. So yeah, yeah, yeah, I can't in map in my cube cuddle, right? Like if I could Yeah, like I would want to Dump. Yeah, this isn't a terrible idea. I'm sure there's a lot of reasons why we don't do it in kubernetes land, but You know, the thousands of apis you'd have to hit the time it would take to gather some of this data Yeah, it could take a while, but yeah, it could take a while, but um, yeah, I don't know Let's I'm still trying to find this article too from we've worked. Oh, okay. Yeah. Yeah, it's it'll I'll if I find it later I'll I'll I'll post in the chat. Um, yeah, thanks the um Yeah, I remember I came from a startup as a lot of people I did for my life at red hat Yeah, and we we actually, you know, you grow right and you go through the growing pains and we actually hired a couple of network admins Right, like we finally had like guys like going out for the networks and like the first two months Three months maybe it was just them mapping out the network That was like they were they were just trying to like figure out where everything was and where is everything give me a good inventory So the same thing in kind of the ansible space, right? Like I if I were and I've done this before it I forget how many organizations I show up. I say All right Let me see what all I've got going on here and like I the first thing I do is start building a solid ansible inventory of all the systems That I am going to be touching So and like How to get that inventory? How to assemble that inventory? Yeah, how to figure out what's in your purview and what's out of that purview Is often difficult, but I would always start with Hey team, we need a solid ansible inventory before we can do anything anything. Yeah, right Like I mean, yeah, we could we could say, okay, you know, here's the ntp service Or here's here's services We know we need as we discover these systems that we need to apply but Yeah, like we have to start and assume that everything is unmanaged and we have to discover everything now everything Yep, everything new and put that in the inventory and have that managed and I'm actually glad you you you brought up Brought up ansible right because One of the tools I'm going to be using is There's a lot of there's a lot of get ops tools that do CD right There's there's uh, there's flux CD right from the weave works guys. There is there's argo CD the coal guys The guys over at coals Came up with something Um, if anyone in the chat can help me out It reminds me it's kind I keep calling it pneumonia, but it's not spelled. It's not spelled like it It just looks like it's spelled that way So if if any guy if anyone knows, um Oh, yeah, see it's called, um It's a greek word Omnia, omnia, I don't so I mean can you blame I mean there's no p but I mean can you blame me that I can't blame you I can't blame you for pneumonia. No that it says pneumonia. Um, they're they're they have it's it's um similar technology Right and and you can do it with ansible too So I'm gonna be I'm gonna be using argo CD, but it you don't necessarily need to be using Um Argo CD or even flux or you can use ansible you can use a bash script Yeah, like anything could be this component, right? Yeah, that's the important thing to point out Yeah, so I'm gonna be using argo, but um, yeah, I want to point out that that it's um You know, it's it's it's not the only tool that you can use right and it's it's um I'm gonna try as as as much as I can to focus on the process um, but Someone in chess is don't scare us with this name Yeah, good point. Yeah, let me look it up real quick. Yeah There we go. So um Um, so yeah, so the just just the idea is is it's the process try to try to focus on the process Although a lot of us here at red hats. We love ansible. Um, we there's uh, there's a small cult following for uh, following for Argo But it's an argo came out of into it. I believe Correct the guys it is a cncf project. It came out of into it Yeah, uh, and we've worked with them quite a bit on it. I think Yeah, so we we actually worked um a lot on the operator So the the operator still considered community because Um, you know, it's built by the community although most of that community That built the operators from red hat. We still have to label Well, we have a funny relationship at red hat, right? It's like, well, this this is built by red hat Why isn't it supported? Well, it's technically a community project. Like this is like we helped somebody build it, right? Yeah Yes, yeah, yeah, there's uh, you know, so there's before before we scare anyone out Let's just scare anyone off. Okay It's a greek word, right? Like yi voya avoya You voya. I can't get the greek pronunciation, right? I'm sorry Uh, a minor greek goddess of law and legislation. Her name can be translated as good order governance according to good laws Yeah, so this is actually the perfect name for such a project even though I can't say it If someone from coles technology can like send me the phonetic like pronunciation, that'd be great Yeah, phonetic pronunciation. That would be great How do I say this in a time of covet 19? Yeah, yeah, exactly Because I don't want to say pneumonia. No one wants pneumonia right now. I don't want pneumonia Yeah, exactly. Um, or someone someone mentioned that I think argo city and flux are merged. So um, so yet kind of yes sort of Yes, I know Yeah, yeah, so the um the guys over at at argo and the guys um Over at flux, right, which is a weaveworks project. They're also in the cncf fluxes They um, they decided On their own, right? It's like Your tool does a lot What our tool does How about we don't duplicate a lot of the effort, right? So, um Yes, they have different they have Not so different they're different ish different enough End user experience, but the core is the same So what they what they came up with was something called get ops engine Yes, which is basically you you think of it as a shared library, right? So they're going to be working on this little shared core as it were And then from there the the tools Are going to branch out to the respective end user experience The long-term goal. Yes being that it's going to be a tool to rule them all Um But um, yeah, that's still very very uh, yeah early on in the stages. They have great goals They have great ideas and it just needs to happen, right? Like it's just a matter of time now So, yeah, they announced it. I think very late last year, uh, that are not late But definitely like october ish november ish. I feel like um No, yeah That they were like merging efforts because I remember covering it in my newsletter and I was like This is going to be really cool Once this likes gets up and running Uh, and the reason we're using I think part of the reason we're using argocd right now During this demo is because like this isn't fully baked yet All right, like to a point where we Want to show demos of it if that makes sense, right? Like it's almost there, but not quite Yeah, yeah, exactly exactly. So it's uh, I remember um I was doing a presentation on git ops and like the night before and I had to put in a slide I did in my slide because oh, yeah Just announced this. Yeah. Yeah. Yeah. Now. I'm like, oh, okay I guess I'm gonna talk about it. Yeah Yeah, something yeah, something changes and you have to like, okay. Well, how am I gonna Someone's gonna raise their hand in the crowd Yes, exactly. Yeah, so I had to read the press release um, so uh, so yeah, so that's um, so for for uh Sack sacking cloud satchin cloud. Yeah satchin cloud. Yeah, they're they're not um, they're not merged. They're, uh, They're just working on a working together. Yeah working together. So Um, duplicated not duplicating code is always good. So um, yeah I mean if you got similar groups with opal source projects with, you know, the same goals They should work together. It's all about collaboration, right? Yep, it's all about collaboration. So um here Yes, so I have so, um So I'm gonna stall um, Argo CD, right? So that's this is this is where This is where you log into your cluster And just like with anything in open shift, um, and I know this subject's near and dear to your heart Chris is that um, we'd sell it with an operator, right? So yes So everything you know, you know everything Everything in uh an open shift Is installed with an operator. So um, we have operators to install operators actually I learned the other day We even have operators installed. Yeah, so that's the the cbo, right the cluster version operator. We have um operators that install operators so So before I start here, um, I'm just gonna create a project to house this Right, I'm just gonna call it Argo CD Um, just to make it simple and then I'm gonna go to the uh operator hub And then Hopefully this is a 4.4. I think I think I wrote a 4.4 Um, and I can't order 4.3 anymore. By the way, if you were oh, okay, cool If I did the art. Yeah, I went rhpds though. Yeah, um So, um So here when I type argo gives me two choices. So right off the bat this will I'd probably generate a lot of, uh, um A lot of questions, right? So, um, I have like the the argo cd and then I argo the argo cd operator with helm So someone asked about how to do, um You know, how do I how do I variableize? You know my this this is a way to do it, right? So this is a way to to deploy and sync your helm charts Um with your cluster I'm gonna be using customize. Um, I'll go over that when we get to it So I won't be using the helm one. Um And this has a little community little community badge right here And it gives you a little warning that hey by the way this community So if you want your support you go over to those guys at the, um You know at their at their github, right and I just click install. Yeah, um So I'm gonna install this argo cd and the alpha channel. Um, because obviously this is alpha stuff as as chris short pointed out it's, um This is all this all stuff is just all all new so um, and then um, so this approval strategy A lot of people ask me about this and this is more has more to do with operators, but um The approval strategy meaning that um For your upgrades, so I think if you hover over it Yeah, so um, if you hover over it, it tells you right, um, right so the uh Yeah, if you if you're worried about like this thing breaking your infrastructure and you want to manually upgrade it Yeah, does it click manual? Yeah Do manual manual upgrade feel if you if you're like, you know what? I want the latest and greatest just give it to me whenever it comes out Automatic's fine. Um a lot of people like living on the edge. So so that's not necessarily living on the edge, right? Like if you, uh, when you Could you actually hit cancel and go back to the install page? You know where you click install? On the operator operator. Yeah. Yeah. Sorry. I like making you back up here my bad So like if you look at this page, right? Like you see the capability level, right? Like this does All five capabilities of that capability matrix that we put out here, right? Like it'll do autopilot You know, it'll manage itself Right, like it'll manage itself and manage its resources for you So this is designed from the ground up To be completely like auto contained and self managing. You just feed it what you want it to do Yep, so you know doing automatic here makes a lot of sense Um, you know, you're just trusting that the community operator is going to work. That's it Yep And then yeah, so I'll I'll hit subscribe. Sorry. Sorry for the uh re segway. Yeah. Yeah. No, no, no worries. So the um We hit subscribe. So this is like what what what happens, right? Um, is our go installed. So the answer is no People are like, well, I installed the operator, but nothing happened I'm like, well, that's true because because the operator gives you the option to install Correct. Yeah, correct. Yeah. So you you installed the ability Right. So you just installed the operator the the the thing the basically it's a crd with a, um Uh with a whole lot of logic built into it, right? So that's a There it can give you things, but you have to ask it What correct? Yeah. Yeah So if you go here, uh, like if you go to workloads and you go to pods, uh, you'll only see the argo cd operator pod, right? Right. And this is just the thing waiting for instruction, right? Like I'm waiting to spin up x y z kind of deal Um So There's a what can we do a lot of things here? Yeah, what can we do here? Right? So there's a lot of things that you can do um You know spit give it, you know An application so you can actually have it start and we're just creating applications for you um You can have it to create just a project for you or you can just you know what just give me an entire stack of argo cd And that's what we're gonna do, right? So, um, give it all the Give me everything right because so here this gives you um Uh a uh Kind of a manifest that you feel so this is your your custom resource, right? So this is this is how argo city is going to be deployed and i'm going to cheat a little bit because um, you have yaml Yeah, I have pre pre pre existing yaml. I know eric jacob eric eric's Never happy when you pre he wants everything everything should be done live but yaml in public is like math in public I feel like yeah, yeah, exactly Yeah, like if yeah, like I would never have gotten the space scene right like the second you start throwing pipes and brackets at me And yaml. Yeah Yeah, let me lint this. Yeah exactly. So let me let me just pace this and i'll go over what it's doing, right? Um Let me make this a little bigger as well Thank you. So Yeah Yeah, I guess so for the for those for those who have the large monitors um So here just kind of just standard metadata stuff right like um, this is argo cd in the name in the namespace argo cd um, but the the spec stuff is what um Where I'll like the magic happens, right? You can just you can really start thinking about automating some of this stuff. Um When I say the uh, I want to say hey, give me a route give me an argo c route because I want to see the web ui Right, I'm not going to do everything from the command line. I want I want to be able to see the ui Um this dex connector is actually really cool Yeah, um it actually It actually plugs argo cd into open shifts oauth, right? So this is very this is very powerful. You don't um This does a lot of work for you Yeah, this does a lot of work for you because leveraging open shifts rback system and it's oauth for argo cd Is really really cool because then you can start delegating um Not only you can either delegate like you say all right You take have argo cd take care of the entire cluster or just give it to a team that manages their own namespace Right, right like you can say You can say like this team of infrastructure folks now have the capability to do argo cd on this one project Yes, correct. Yeah awesome, so it's um So yeah, so it's it's it's it's cool. It's actually a shout out to um To andrew block from the red hats community of practice He's also consultant. Yeah. Um, he he wrote he wrote this. Yeah, he wrote this by the way He wrote the the connection And uh, he actually he actually writes a lot things that become so the the quay um operator He wrote that and basically gave it he gave it to red hat now red hat supports it So like he he's doing a lot of cool things. He does so much stuff in the background Yeah, so if Yeah, yeah, he does and he does it on his time off. So andy if you're uh, if you're on yeah, if you're on you're on later Thank you so much buddy. Shout out shout out. Um, and so um, so now that we connecting decks On the back end with the oauth Now i'm going to set up some default policies, right? So the default policy I'm going to say does read only right so anyone logs in they're just going to get uh read only Okay, um That's default policy unless you're a cluster admin, right? So here i'm setting a policy of saying If you are a cluster admin on openshift, right system cluster admins I'm going to map you to the admin role on argocd So that's this is this is where you start doing um the mapping so you can obviously you can start Imagining you can have some pretty uh pretty complex policies here. Um, and then i'm scoping it down to the group to group level So um because i want to manage a byte group. So um So if you guys um Want to know where that is Uh, where is argocd? Argo cd? Uh proge right proge. Yeah. Is it proge? Get hub Is it here? No There's an operators uh User guide operator operator paniel. Maybe that's what it is No, it's This is a bunch of fun stuff by the way argo proge. Uh, there is so What i'm looking for by the way looking for the authorization stuff No, i'm looking for um, how i got these options, right? It's because like the question is like, well, how do you know to put all this? So i'm like it's it's actually in the docs. I just need to find it. Um It is in the documentation um Did the user guide probably maybe Uh, oh, there's a search Oh, hello Hi, sir. Yeah, let's uh, are you are you better than some other search bars out there? Yeah, let's see here cluster bootstrapping Couple of blue strapping helm. No No, no, you guys are gonna see all this later here. Um Upgrading user guide developer guide user guide. I would think it would be under user guide Um Oh, there we go. Someone helped me out in chat. Thank you. Oh, wow. Thanks, buddy Your your google foo is um way stronger Way better than mine. Yeah, see that's what I was asking because I knew about this arbeck page I just didn't know where it was Yeah, see here. There you go. Yeah, so we have the okay. There we go. All right. So all right So yeah, so you can get um um Pretty granular, right? So it's like I want You know db admins For applications to be able to create in the staging so like you can you know, you can get pretty uh, pretty Uh, pretty granular with this you can get super granular, right? Like yeah I want I want christian to be able to deploy this pipeline, but not this this pipeline kind of deal. Yeah, like Yeah, I want to be able to people to view it People to edit it create override sync update. So yeah, it's it's It's um, it actually gets pretty hairy Um pretty quick, right? Arbeck is hairy in general, right? Like yes. Yeah. Oh, yeah. Not for the faint of heart It's just I mean I I kind of equate it with II AM Because it is just as granular and just as Uh, you can put your foot in it as possible. Yeah, yeah, exactly as you can go. Yeah, you can go hip deep on this stuff um So anyways, thank you, uh for for your google, uh Yes, please. Yeah. Thank you so much for joining today and thank you for your google food So, uh, so yeah, so once I click create Um, this will go through righteous phases Um, pending installing right, uh, we can actually watch the paint try a little bit here if you go to pods and um And uh and and see these uh containers creating right so this is the What we're actually waiting for is the last one to come up is the argo cd dash server And that's the one, um when it's running It'll um, well, we can't log in until dex is up and running. So Because we can't log in on the web. It's a series of steps. It's declarative, right? Yep. Yeah, you you gotta let it work and then you can work with it Come on It's a like watching paint right there. It is all right. So So, so we're in right so we're in like fling. Um So let's go to networking routes and this will have the argo cd here Um, so let's accept the cell scientific it Accept the risk From rhpds. Yes. Yeah, so we don't use so just for the audience's edification Like we can use some let's encrypt stuff on our clusters, but it kind of slows down the deployment A smidge like a tiny bit. So we typically just use self-sign clusters Yeah, self-sign search for our clusters just for demo purposes So, um, so for those of you who've used argo, this might look familiar except for this little login via Open ship button thingy here. Give me the button. Give me the button login via open ship, right? So this is like the sso login. So this is connected with dex on the back end. So This login this will bring this up here. Normally would I'm typing your normal credentials for your open shift cluster. Yeah, I always forget the open tlc. There we go So this authorized access. Yep. Now, so it's permission It just needs to read your user, right? So once you logged in you can actually go to um accounts And you see um that you're an admin, right? Yep. So Um, so that mapping worked. So as you can see but um So cool. This is the um The overview, right? Yeah, this is the argo cd page itself Yeah and what I like to do is I like to um Drop to the cli Because some of the stuff you have to do this it's it's still like very there's some stuff you can do the cli some stuff You can do On the web ui. It's not a hundred. It's not one-to-one just yet, right? So I actually have the um It's almost like all the stuff is new or something. Yeah, it's like it's almost It's almost like this is cutting edge technology. Yeah, exactly. So if I do argo cd version Um, right i'm wanting one 1.5.2, which is new enough Um for this right so right So when you run the argo cd cli it actually reads um your um Q config. Yeah, your q config file, right? So um, I actually did an oc login before before I did this, right? So it Okay, so yeah, you were logged on the command line. So I'm already logged in. So, you know, if for those for those of you I I'm missing a step. You're not seeing a step that I already did before is actually logged in, right? So if I do like oc who am I I am the open clc manager, right? Which is a cluster admin, um, right? so here If I do um argo cd add Cluster it's cluster add, right? Um It's saying, um You know it's saying oh add what cluster right because the command's not complete It says well, what what cluster do you want to add? Um, because You know, I I see this one, right if you're logging into multiple clusters It'll it'll list all of them because it's asking well, which one do you want to add, right? And I I believe Um, you added just by the name Right. So yeah, I think you're right. Yeah open shift gives it this uh This context name. Uh, we can rename it if you want. So, you know, it's something easier But I always do um, yeah like that middle part of that like Everything between default and open tlc manager. It was not created by us by the way. It was created by our Demo system. Yeah, um I missed it broke. Did you what I because I missed oh you missed a step. Okay. I missed a step. Yeah, so um It broke. Well, yeah, because I missed a step I have to do argo cd login, right argo c login. Oh, okay. Yeah, that makes sense Yeah, because it is a separate system and has to go off to yeah, so like I have a Yeah, I don't have a an argo cd. Um So it's argo cd login And then let me put up the help menu. Um, usually you give it a username password And a cluster whatever right? But since we are Using dex and uh the open open shift. Yeah, we have to do the sso option Then what just login server name. Yeah sso and then whatever the server name is here The argo server the argo server name the argo. Yes, correct. Yeah, where our girl's running, right? So, um I'm on port How does it do an argo port? Do I have to do it like this? I think it's like a kubernetes thing Oh, yep, it is. Yeah So it opens up a browser and essentially login. Yeah, log in, right Cool. Once you get authentication successful, the cli should return, uh, which it does Right. Um, so now if I do an argo cd version. Yeah It gives me And yeah Cool. So now Where I was before There we go. Not that's not not sso add. Where's the add? Cluster add. All right. So now we're doing cluster add cluster add the name of the cluster Um, in terms in in the context right in what how it appears in my in my kube config file. Um Uh-oh permission is denied clusters create k cluster Quarters create Oh So wait a minute service account already exists Updated updated Is this is this a dex error? I'm wondering is this a dex error? No, this is a oc Error That's talking to the api You're logged in. Are you sure you're logged in to the right cluster? Yeah, yeah, you cleared your So this is server la. Yeah, that's right tlc. Okay. That's good Where's the descriptor hang on? I wonder if I mess something up When I try to add it before I try to copy and paste this out of zoom Yeah, it's just trying to copy and paste out of zoom like it was my terminal. Yeah That's weird. Yeah, that's weird. Okay. Let's see if I can add it In the um In the ui, let's see if it's just one of those things where like the ui might have more logic You're hearing me typing on another screen And and and Andrew's funny, okay, uh clusters in clusters. Yeah, you can't see the problem is you can't add um a You can't add the existing cluster Yeah, well, I can't I can't uh, I can't add a cluster here Right. Oh, there's no option. Yeah. Yeah. Yeah. So you have to do from the the cli cluster bootstrapping Where's the um, yeah, because so the um, let me So here the dex I'm saying Okay, I'm using dex version two two or two. I wonder if that's the the updated version um policy read only roll cluster roll admin So you have yeah, like that's what I don't understand. Like you have the right permissions. Yeah So let's um, so someone says it's delete the roll. Um So you do roll bindings and try again. Yeah. Yeah, so let's do uh, wait, let's do add first Well, let's let's do this live. So cube ctl You got delete the roll bindings, etc and start again interesting ctl get um secrets Are go So there's the secrets that oh my god, that's so much. Um Yeah, I probably though we got those are are put it by the operator. So let's do add And then let's do um cube ctl delete s a Right in the namespace I'm not sure if this will work. So I'm just I'm just doing it Yeah Okay, so delete that guy delete Plus cluster roll cool. All right cluster roll finding um This guy And then you just cross your fingers and hopefully this goes Yeah Can you can you send that to me, please? Sure like through through some There's some mechanism. Yeah, I'll I'll chat thingy. I don't care any chat thingy. I got them all open Yeah, you got them all open Here we go. Thank you And then but we did um So one one of the one of the amazing things about andy say i'm i'm i'm just praising him right here Is that no matter what time of day he always answers me? No And like he's like in some part of the world. I'm like, isn't it like China right now or aren't you in argentina like what were um But like he'll answer okay, so there there is an issue for dex sso Yeah, so he might have pushed a new version of the of the dex operator. Um You need to set policy default and roll Argo cd r back cm. So this is after we get here. So yeah, never mind. We haven't logged in yet If you do add without any arguments, what's the yeah, so oops Oh, yeah, that's a good point. Um Argo cd add and it says oh cluster add Yeah, it says just api Uh-oh, of course andy says I know what you did I don't want to be I'm about to be scolded hold on. Um Maybe it's just a cluster Hold on Wait, yeah, is it just the api? Yeah, so maybe just this yeah, that would make more sense since it was asking for No, you left off the air. Yeah. Yeah, I need I need the a here Give me the a Yeah, does not exist in cube config. So yeah, okay Um So, yeah, I don't think it's the the cluster cluster add and then this guy here Is there a log? log level log level I don't know 10 I bet you about log levels uh info So I think this is info uh debug Yeah permission denied clusters create Super helpful. Oh the problem is that cluster admins is a roll, okay That doesn't held your hand. Oh It's a roll not a group Yeah What yeah, yeah, okay. So now now I'm following okay So this is something that I'm really bad at. Um, so if anyone can help me in the chat or anyone Or you know, I'll ask I'll ask my my back end engineer here Um This So since like kubernetes is just like a bunch of objects, right that relate to each other via label Um, I don't know how to get so so just back up a little bit. So the um Um The the issue is what we're theorizing is that um That this This roll I'm not part of this group here our gocd project isn't um the the the The the policy right so me so like if I do oc who am I So, okay, so the tlc manager is not yeah, it's not part of this, right? Yeah But like how do I see what groups like oc get user? and then See get user Sometimes it's in there. Um And it's not because sometimes it's part of the array, but it's it's obviously it's oc get groups. Yeah, no resources No resources found. Okay, maybe Yeah, how do I see all the titers on yeah, maybe you can help me out. Tyler Tyler is I always love to have the red headers on board. Yeah. Yeah, so, um So how would I see oc get cluster? Roll binding so there are no groups on the cluster. So you would probably need to So you roll binding and then bind it System cluster. Oh, that's that's too much. Um less Cluster admin So there is cluster admins, but I want I want to bet that I'm not part of that group. Tyler says do Oh go back to your yaml Oh, there might be a uh, Tyler's got something to chat here like just add the user No, there's just cluster admins Well Just just cheating of it all the others. Yeah. Um, so I guess I could Yeah, just add the user for now. Um, yeah. All right. So cool. Let's go back to the the manual that someone Yeah linked earlier. I got it. Yeah, if you need it. So this is um So we want to so so how do you do user? Uh Just you pee is it pee or is it you? Our back permissions are breaking down the permissions applications for app Oh, so I think it's just pee is you always have the pee Oh pee is for policy. Maybe right. Yeah And you would just say oh pee and then the g is for the case. This is a group Right. Yeah. Yeah. Yeah. What was your reset in chat? He just copied and pasted out of the the docs. So yeah roll roll user group comma resource comma action comma object So the pee is always there Roll user or group in this case would be user. Uh, yeah, so then let's do Um our go cd our go cd ammo. Um, um And so this would be let's keep that there. Um, just add on So pee For policy, right? Oh So the roll is resource admin So then I would be a open tlc manager like that essentially Maybe take out the scopes I don't know man Let's go here try it See what happens Tyler says what was Yuri said and was shari if I'm saying her name wrong. I'm sorry. Yeah So this is this is this will be fun. So we'll see how the the operator handles this Right to see or if I have to just delete the thing and start over. Yeah, it might be a blow away the project start over thing. We'll see Okay I love learning live Does learn live. Yeah, that's part of twitch, man Yeah, so I so this is so fun fact. This is the first time I've done this on a rhpds cluster so the Yeah, and for everyone out there rhpds is our like partner demo system. So we have a number of things that is just like You know, it's like click click click. Give me this click click click. Give me that type of thing So it shortcuts us, but there's also Because anyone or just about anyone in our technical work can do it They pre-defined some things just to keep us safe and from Yeah, yeah, or other kind of deal Yeah, or like they do things like mess mess with your mess with the groups. Um, yeah Or they give you a user that's seemingly like an admin right like it's close to admin. Yeah, it's like close enough to an admin, but um So let's uh Let's let's just apply this right for replace Let's see what happens And if not, we just blow away the yeah the operator and Okay, okay. What's it doing placed? And then let's look at the workloads pods and see if like things start deleting or not Does it look like? Um, it depends how long that reconciliation loop is too for the operator itself So let's for something this high level. I would not think it'd be long. You'd have to like force it maybe Invalid policy So it does So it it does actually pretty fast because if you see the here in the bottom right, it's just invalid policy Expected five five got two evals Um Expected five is there like a default set you have to put in? Yeah, so it's it's p because um So i'm gonna delete this and just do it apply every place again I'm going to So the only change now besides that is instead of admins. I just put admin and let's just see Oh, okay. Yeah, what happened Okay, so i'm in again. All right, so let's try maybe maybe I just put admins by by No, same thing Uh, we'll see who am I Don't Oh, I know I know Who to ping We'll see who am I who am I? Um, who am I who am I? Where did I where where am I who am I? Um, what group am I? How do I know? Okay, so I'll get So there's cluster admin other admin. Maybe cluster admin dash zero I think that's what it is Is that that that special Yeah, because I don't remember. See but if you're not in any groups Yeah, um add yourself to the group Regardless hang on tyler just wrote a whole wall of a whole wall of time. Okay. So what is what does tyler say? He says so how argo by itself handles things is the policies and arbex stuff is read dynamically So no reboot should be required required. Okay I'm not sure if the operator will see a difference in the cr and just try to reconcile the whole thing and just clobber everything Which in theory it probably should do if you're gonna change the whole higher level, you know accessed everything Because usually yeah user info See so look here Okay, so it tells you so i'm i'm not part of any of the any of the groups Yeah, let's add you to a group buddy So That's a problem. Yeah, so I will I will um Okay, so let's do um, we'll do this live. So oc adm um add Add uh, let's see policy Add roll to user cluster dash admin um to the user Open tlc manager, okay, and in theory this should give me No, it still gives me Oh, the um, no that doesn't add me to a group that um Wait, what did you do? I didn't see it. Sorry. Yeah, so um Added roll not a group. Yeah, it added me to that roll, but not a group. Let's just see let's see if that worked I don't think it will Oh It's doing something thinking it's thinking hold up. Nope. Oh kernel panic I am uh calling the big guns Calling the big guns Add cluster Roll to bind to permission denied um, and that gives me So wonder if I um, let me go back to my original config to change as little as possible Okay, so that is authenticated I could I could just give me authenticated See, we're all about security here. So it's uh, yeah Yeah Yeah security sometimes I mean, it's great if you're demoing security to the security guys. So Uh Could add authenticated this will give everyone access to it Well, you know, it's fire one That's right. Uh default policy roll. I could do that. I could roll admin default policy Maybe Just to show Okay, so ideally Tyler says this should just pick it up, right? Yeah, it should just Like auto grab it pretty quickly Auto grab just do it. Yeah Do it live so cluster added All right, cool. So back to 20 minutes ago now that I So, um, so by the way, um, so don't do what I did right all policies roll admin. Um Well, as soon as Andy gets back to me, he'll he'll let me know the like the right way to do it But I essentially changed the default policy to um To roll admin instead of Roll read only because it wasn't picking me up as part of any group here um So, um, so that's that's not thing here. So so I have a cluster added and like when I go to argocity um I go to my clusters and now You see my cluster added as part of it Right so ta-da. So we have magic. So that's magic. Um, another thing you need to do and this was actually a bug But let's see if they fixed it. Um This is a bug I filed earlier, but it was closed. So Um, so now I had added the cluster, right? Um So if in order So argo can do things two ways, right? So when you sync an application to a get repo It'll dynamically do a get clone Or you can load the get repository beforehand, right? So this this will make it slightly faster. Um, when Um, instead of doing it dynamically, it'll have a local copy, right? So it'll it'll just do the get pulls and Um, for a can tell the differences locally as as well as that kind of like a local cache, right? So right if I do an argocity Um add a repo add. I think it is repo And let's just look at the hell menu here So, uh add, right? So if I do a oops If I do a repo list says nothing Right now I do an add of my get repo A repo list now shows the repositories, right? Um, and if that again if you go to repositories here, it'll Um, it'll add it here You can use ssh, right? Like if you connect it with ssh, you can give it the the key, right? If uh, htps you can give it the username and password it'll it'll store it locally mine is um Um, would you just give it a generic key or something? Oh, this is this is a public it's public. Oh, okay. Never mind. Yes public repo So since the public repo I don't have to give it anything here. I just I just um I just give it the repo, but if you have like ssh or whatever you can upload in there Or a private repo, whatever you want. Yeah, exactly So here, um, so cool. So now That's like the baseline of what you need, right? Mm-hmm. You need a repo, right? Like I want something to sync again This could be dynamic if you do it at runtime, but I like uploading it first Right and then clusters, right? Where do you want to deploy your applications? Um So some of the things in my repository if you notice here in my route I have um I already I basically hard-coded the route, right? Because I know what the url is And I just gave you the name a bgd, right? So, um bgd by the way, sounds from the green deployment In case you guys wanted to know oh very nice very nice very intuitive. So here, um argo stores everything in terms of Um of applications, so there's this this is a kind of a thing and uh if andrew's on I know he'll he'll he'll have a opinion on this but um We have um in it we have a lot of overloaded terms, right? So like oh, yeah So like you know an application here is kind of a different definition of what you think of out of an open shift or in kubernetes a deployment here is different than What would be a big d deployment in um kubernetes versus small d deployment of like what I consider a deployment so, um But just for the sake just to normalize it here an application in argo cd is a deployment small d of Your application to a cluster Right, there's a one-to-one relationship, right an application per cluster. So, um So here when I create an application um in my mind I'm actually talked talked to this about to some other people using argos like to me this registers as a small d deployment but Argos a cause of application so um because to me application is more overarching like my application could be running in different clusters And each cluster has its own deployment one or many deployments. So um We can get philosophically talked about as much as you want here, but um I'm gonna call this bgd, right? Um So here it says what project you want to um install this in This project when it says this project it means it in the in in the scope or in in the view of argo cd Right not in the view of kubernetes. So again, this is separate, right? So this project in relation to argo cd, which is the default project You can have many projects right in argo cd um sync policy i'm gonna do it manual just so you can just so you can see um Repository url, right? Um Gives you this convenient drop down um to give this here Um, whatever url you can paste it here, right? You don't have to upload it first, but uh, right? It'll just Yeah, yeah here. It'll just default you to whatever or whichever ones you loaded. It'll give you the drop down Uh revision means essentially what branch, right? Um I created a specific branch called twitch, right for this uh for this video So, uh Yeah, right calcon being it and then the path right like where Where is your manifest right so here? Yeah, I wish there was like a tree view, but here There is a tree view a download octo tree. Oh, that's right. You had that that plug in was actually really cool. Yeah um and so So here I have a directory right here I have a directory called bgd and basically it's in that directory. So I just want you to go down one So it's not that it's this Okay slash I like putting the slash at the end. I don't know why I saw a demo When I first saw and someone put a slash at the end so that I just do it now I don't think it's necessary. Um And then here the cluster is uh, the cluster I just added And now here the destination namespace. So now that this is um This project lands someplace else this argo cd project Can land in a different open shift project. Correct. Yeah, so this so this yeah, so so since I'm um, it doesn't exist Right. So if I go to overview or sorry if I go to projects here and I do um bgd, right? It doesn't exist because my um Your namespace has been yeah, I'm gonna create it essentially. So it doesn't have to exist. It can exist Right. You can pre-create. Yeah, but um Since I'm storing everything Um, it doesn't exist right now or say so this is all you need And then I'll click create And then uh, our goal will go and do the things it wants to do right? So if I click here the argoy things It'll do the argoy thing. So essentially it'll go and it'll say, all right I'm out of sync. I'm out of sync. Yeah, let's let's sync this thing up Yeah, let's sync this thing up. I have my review. So you told me That you want this stuff in this repo to be running here Um, I don't see it running. So you're out of sync Right. So once I click sync And I say synchronize everything Give me everything in that repo now. It starts the syncing process and look look. It's that fast So yeah, like as fast as you can hit cube cuttle apply. Yeah, it does it also Now any changes are picked up in the get repo Correct. Yeah. So now I have to go to single source of truth is creating a single source of truth is there Right. So I have so I have here bgd, right? So I created the project for me. That's cool It created, um, you know, it's creating the pods. So the pods starting to create created the service And it created the route. So once the pods running that route should be connectable and Hey, look at that. It's just switched to running. It switched to running We couldn't have timed that any better white in your sentence As soon as you were done with your sentence, it came up That's that's that's I want to call that out because that rarely happens that when you time what you're saying to what the cluster is doing Exactly. You have no idea what these clusters are doing behind this thing. Yeah, you have no idea. Yeah, exactly. You're doing it live. So here Like I said here, I drew a blue a drew a blue box because on my repo If you remember and if you don't I'll just bring it up It says my color equals blue Yep, so this is cool, right? So you have Your route everything all stored into git Um, but let's let's let's do something. Let's do something funny here. Let's Let's do something. Um, so let's edit the deployment Right. Um, okay, so we're at a deployment and instead of blue. Yeah So instead of blue, let's do green right. Let's do value of green So I want to I want to point out so you're you're editing the active deployment on the cluster right now Correct. Yeah. So this is I'm not making I'm not making a code change Right. I am not making A change in the code of the application I am changing the infrastructure that changes the behavior of the application, right? So, um, nothing in the code changes. I introduced the change in in the cluster that will, um It's probably already done. Yeah, that's that changes the behavior, right? So if I reload this now now this it's it's a green square, right? I put in green It does green if I go to argocd And um, I go I go on the overview page. So notice notice the status, right? It's out of sync something's wrong with that So it says it says disconnect. Yeah, there's a disconnect, right? Your application's healthy, but I'm out of sync Right. So there's there's difference. There's nothing wrong with your application Everything's running everything's up and running, but there was a change in the environment somehow that changes the behavior Um, because in in the repo I'm saying I want blue right and someone here if I click on here Someone manually edited something. Yeah, someone did something bad Logged in. Oh likey we we Some someone thought that we're being clever and edited my live running Cluster right with keep cuddle and this is similar to sshing into the box and doing stuff as opposed to using a proper Configuration management tool like an ansible or correct the other ones out there. Yeah, it's it's the same paradigm, right? So it's the exact same paradigm. It's the same paradigm. Where is You know someone sshing and changing something and then ansible You know rectifying that for you. This is the same thing except someone's using keep cuddle Right, and we're just using manifests here like you can even load the diffs, right? So our city has a cool mechanism where it's like it loads the diffs. It's like here. This is where the diffs are um, we want it to be blue And that's so slick. I like it. Yeah, we have green right here. So here Um, you can sync it right I can say sync and then synchronize We're gonna force this puppy back to blue like it or not Yeah, like it or not this will this will and and this is i'm doing this is cool And i'm just doing it manually, right? So there's actually an odd. There's an auto sync Right, like if you switch to auto like we never would have saw any other stuff, right? Like we wouldn't have had time between us speaking. It would have been fixed Yeah, exactly. It would have done so now so now here the overview says It's healthy and it's synced so now your app is up and running and not only is it up and running it's it's um It's synced to your source of truth Which is and then if I read all this page it should come up blue, right? Um, so what's the proper way to change this color? Yeah, oh the proper way to change this color would be to um come up here, right and say I'm gonna edit this Well, the proper way is to fork it the proper way is to fork it Yeah, let's not let's not jump through the get theatrics. Yeah, you know, yeah Submit a we're forking it creating a branch submitting a pr For review it gets the review it finally gets the approval tag and off we go But yeah for the sake of discussion We're just gonna edit master it right now. Yeah, we're just gonna add it straight to the to the branch Um green right so the proper way to change the color is to go to your source of truth And say I want this to be whatever process you have in place to get changes into your system through get You do it that way. Yeah do it that way Yeah, but the idea is So to answer your question, how do you change the is do it and get right do whatever process you need to do and get um so here It should come up pretty soon. I can actually do Where's the refresh There we go. So it's not out of sync. So it says out of sync. So i'm running blue Yep, but apparently blue upstream repo says green So let's sync it right. Um before I hit synchronize You notice like there's like other options things like dry run, right things that you would normally would want to do And it even points out which individual thing is out of sync like you can just act on that one So I can yeah, so so instead of the overhead I mean, we're just playing around here But like there would be overhead if you have, you know hundreds of manifests, right? I'm trying to sync all of that if you just want to sing one thing it'll it'll do the one thing Um, also, there's an option called prune. So what it's like, what does that do? So first to synchronize it So make sure we're all in compliance, right? So we're all right. Yep. Okay, cool. And now we're back We're back. So now we're back. Truth are all good here. Yeah Um, so that prune option. So there's the the idea of is You not only have things that are Um that you're syncing with kid, right the you might have Um instead of like introducing a change like changing the color like what if someone adds something? Right, right. Like what if someone adds something to this namespace? So for example, so like since this is a cooking show um uh So I have this config map, right? Um So it's just a it's just a dumb config map, right? So if I do, uh, a cube ctl apply the namespace bgd So if I add this config map Okay, um Now You're immediately out of sync. You're immediately out of sync. Argo CD says, okay Well, there's this other thing that Where'd this come from? Where did this come from? I don't know. This is not in the repo So I don't know what to do with this thing Who's your daddy and what? Yeah, what does he do? Yeah, exactly. So where Someone added a config map. So so the config map or a secret actually could be kind of a dangerous thing, right? Someone Right to a different database. Someone is is trying to get some sort of key value stored and injecting it. Um You know or someone just made a mistake and just added to config. I I know Notice how I did a dash n in the namespace. Sometimes I just forget that Oh, yeah, no, totally right like whoo wrong namespace Yeah, and I and I apply my um my config map to a different namespace like where'd it go? I'm like, oh, it's a cube system. Oh, why it shouldn't be there. Um So when I click sync, um, I need to do a prune, right? So when I click prune, it'll go. Okay. Anything I don't know about it's gone, right? So once, uh Once it starts the click refresh um Did I oh did I I click out of it? Oh Yeah prune it prune. There you go. Nice So this should there we go. Yeah, it's thinking and and done, right? So, um I just love how fast it is. That's just yeah. Yeah, it'll just it'll just do it, right because one um, so this is different than um different than uh puppet or like uh, like salt stack or like or even ansible where you have to like trigger This right so you have to trigger these changes, right? Oh, you can have like a Like a cron job running cron job around to trigger the changes or force the synchronization kind of thing Yeah, yeah. So you can do that or you can just leverage the control loop the the idea of crd's And just have kubernetes do it for you. That's so much The reconciliation loop and watching the the events coming by right like that Doing things with kubernetes kubernetes native functions In kubernetes native ways is the best way to get the speed and you know performance you want from your infrastructure When you're building cloud native environments these hybrid clouds that we're talking about nowadays, right? Like this is going to extend across on-premises cloud Your desktop your laptop for development whatever it is Like this is how we touch all this. Yeah, so um So yeah So here what's what's actually really cool is I was talking about before like, you know What if I come in here and just delete the project, right? So when you delete a project Um, boy You're here. Everything's gone. Like everything underneath is gone All the secrets the config maps everything everything is gone in that namespace, right? So um So while this is terminating here this takes it takes a while for it to terminate, right? Yeah, because it's actually removing every piece and component replica set and the whole nine yards that you've put in there Yeah, so all the all the all the work we put in to deploying The application and it's now gone, right? So how does our go handle this, right? How does your your tool so? So it's not only out of sync, right? Notice the status instead of changing it says like oh, no You're missing gone. Yeah Alert manager, where's my page? Yeah, where is my pager up pager duty pager duty send the pager duty My app is down, right? Um, but if I click sync and I click synchronize If I go back to my project and type a vgd. It'll it'll come back up, right? So Um catastrophic failure you're back. Yeah, this is You know and it's it's already it's already running. Yeah, it's already up. It's already back up To the color I want to the state that would it was in so Um, so yeah, I mean and like clap your hands. Yeah, yeah, exactly. Yeah Clap on application clap off. Yeah, exactly. Clap off applications up. You can even so this is actually pretty I know me and me and eric we messed with alexa Connecting alexa to open shift. I wonder if we can do the same thing with argo and this is uh our argo sync that Alexa sync that uh sync that application for me So this is uh So this is really cool if you're doing like If you're doing in cluster, right? So there's the idea where argo can handle multiple clusters This is one of the the differentiating factors Of of between argo and and flux, right? So so flux You need to it's in cluster, right? So you need to have a flux system In each cluster that you want to manage Whereas an argo is kind of a hub and spoke design Where you can have an like a central argo Installation that manages many clusters, right? So let's um, you said you had a cluster, right? Yeah, I got a cluster. You you want to jump into it real quick. Yeah. Yeah, so send it to me on the Yeah, I got you. I got you And then we'll see how argo Handles that So, um, what what they should the yeah, well just make sure it's in the right chat um, oh shit So yeah, so I'll let you uh Well, while you're while you're looking for that. I got it. I just simply in slack Cool. Perfect. Thank you. Um So how do you handle so what if I want? To deploy this application, right bgd What if I want to deploy this application to many clusters? Yeah, so there's a there's a few things that you need to keep in mind, right? So one of the things is that that changes Is the fact that the routes going to be different Okay, so the routes going to be different, uh, you know different clusters going to have a different dns name It's going to have a different different everything. Yeah, different everything um, also What if I want to do something like like a blue green or canary deployment or something to where You know what I want one cluster to show the blue square and I want the other one to show a green square, right? Oh Yeah, like how do you do that? Right? Like how does like I have a deployment and it's hard coded in um You know, how do you do that? So the answer to that is what people commonly do is a customize, right? So they use customize and that's built into argo cd, right? Um It'll it'll do customize. So the the idea is that you have a base directory with everything that Everything that's common between all clusters, right? So in my base directory things that are common between all clusters are things like the service, right? That doesn't change, you know, I'm still listening on 80 80 In my service no matter what cluster I'm in. Um Things like the deployment, right? So other than um Other than the fact that this value is set to blue the deployment itself Is uh, it's pretty common Also, the route definition is common, but if you notice here in In my uh, my customized configuration, I actually left it empty set All right, cool. Yeah So I have base, right base meaning I have um This is common across all clusters and I have a customization yaml saying, um, you know, when you run customize Run it against these services here. I was these Manifests. Yep. Um, touch these ones. Yeah, touch these guys. And then what do I do, right? So I have overlays So in my overlays I have um I have cluster one right in my cluster one I have a deployment definition that says hey Whatever I guess so I'm using the the json, um patching I forget what it's there's like a long name for it, but um, I'm using that mechanism To say hey Patch this value to mean blue. So no matter what it's set. I want you to change it patch it before you load it into kubernetes um And in the route too, right? So in the route it says, um, if uh patch patch it within I I'd realize I have to change this but that's that's fine. We'll do it live. Um We'll do it live. So we have here, um, we'll patch the host, right? I put it have it empty set in the base Here I said, okay when you load it in patch it with this when you want to say cluster one Same idea with cluster do same idea deployment, right? I'm going to change it to green And in the the route I want to change it to this route, right? Um, so I'll actually I'll do this live or watching here Uh, I believe this is your apps route So we'll do this. This is your cluster So satchin mentioned that I think we with git ops we need to ensure that all the operators should also get deployed through git now That's possible That's basically what operator had does is it is pulling in stuff from, you know, a certified repository and, uh that image, um You can in theory deploy your operators with git and like go through this whole process yourself But that's part of the the the the power of open shift is that operator hub integration Which you can also, you know deploy that operator into your, you know, vanilla kubernetes cluster any kubernetes cluster certified kubernetes cluster Those operators should work. So there's nothing preventing you from saying Uh, git apply this operator, right? Like Or not git apply but kubectl apply this kubectl. Yeah. Yeah. Yeah And also, um, I can definitely see the operands Being in in in git, right? Absolutely. Yeah, I definitely see those definitely being in here. So I have This is my cluster and then this is yours Uh, all I guess is what you're in detroit, right in dtw. Yeah, dtw, baby. There you go So this is this is actually a really good really good example, right? I have a cluster in the west coast and have one in central Central. Yeah east coast. I mean, I don't know whatever you call Detroit. Yeah Yeah, where is this Detroit central Detroit is Detroit is central or eastern time zone um, got you and we have faster traffic to New york and chicago than we do toronto, which interesting Toronto's a little further away than chicago, but yeah, it's weird I'll say you probably drive faster there. Um, you have like when I want to go to toronto. I still have to lay over in atlanta Cool. Um, so yeah, so here, uh, if you're watching I updated the route, right? So now when Cluster two gets deployed This gets patched and then And then the first cluster This route gets patched, right and one does one to the other. So, um Cool. Oh, yeah, so, um And how to delete something in argo city you have to delete it from within argo city, right? So you just ready for within argo city? Yeah And it's actually gone here from the branch. So that's how you delete something, right? So now you have an overarching of control system here. Um Cool, so then let's Let's uh, Let's where is here? There it is. So i'm gonna um So you're gonna see me add a new cluster to argo city, right? So if I do argo city A cluster add Right, it just sees mine Because I already added it right lc cluster list Right, it's um, it'll list the The cluster I just added at lax over here Um, so it doesn't see your cluster, uh, chris. So what I have to do is I have to do an oc login Uh of that cluster, right? Uh, what is the option in secure skip tls verify? Yeah, and this is open TLC tlc-mg are and this is the standard password. Yes, it is I can never type it I always copy and paste it Yeah, I can never type it. Yeah, I probably should both copy and paste it There it is. All right. So then now if I do, um Cluster add now it has this the second option, right of Oh, look at that You can pick which one now Yep, so that cluster add this guy. So this should work Right because we fixed the The the binding. Yeah the binding. So now now that the cluster is there now if I do Argo CD, you know cluster list It'll list you there as well. So Now I am all your cluster belongs to me now. So So let me uh, should I Copy link Let's open it up here just in case. Yeah, just to give her just have a side by side. Yeah Um, oh, yeah, this is the the oauth page here And then I always have to look at the keyboard to make sure I'm typing I'm typing right. Um, so here. Yeah So I hear I've had two clusters, right? Um, what I'm going to be doing is that if I go to um The base right everything is going to deploy in the same namespace. I'm just gonna call it bgd, right? So if I go to Um, I forgot this one is mine. So if I go to this one bg is not found I go to yours bgd So clean clusters clean clusters, right? Um, so here Come on You can do it Reload There we go. It's not application itself. No apps yet. Um, so same same steps, right application name I'm gonna call this vgd k one. I guess right because we this is the first cluster and it's I'm using customize right default policy URL same same URL same Same branch. Yep So You can do it. I believe t w i g ch. Yeah, um You cheat a little bit, but I'll yeah Well, this is better than what what I do with j with purse list, right? I heal he I think I give him nightmares. Um So here what I'm saying is for the path. I'm saying, okay go to the overlays and And read cluster one, right and that'll run customize and it'll load all the right options there Um for cluster one. I want my destination to be lax, right? And in the namespace of bgd Right and then that's all I need. I click create And this will uh, it's gonna say missing out of sync and it's gonna sync up if you hit sync, right? Yep, so here we'll add another application bgd k two Um, again, this project is in the scope of uh, our go so, um And this is see if I could spell it again Yeah Nailed it first day So here I got uh cluster two, right? So here now i'm Um, just basically it's the same get repo. I'm just pointing to a different director. Um, and then in the cluster I'm saying uh, detroit And then the namespace bgd, right? So create so here, um So here I get get these tiles one of says obviously in in my re they're missing in both, right? Yep, how to sync out of sync missing for sure Missing for sure, right? So if I sync here Yeah, do the whole thing And then let's sync this guy too And let's uh, let's look at the let's look at the let's look at the cluster. So it says the tank. They're both all right Yeah, this this one's progressing this one's done Um, so let's go to the the la one as you see my um, yep, there's a project It's project here. So this is all it would be the same on both, right? If I project pod everything except the url, right? So I expect this to have the proper url Looks like it does green, right? And this is yeah yours is blue mine's green got it So let's uh, let's see what yours is still going No, it's synced. Okay, right cool. Cool. So let's go. Oh, see there it is So, uh a few seconds ago. Yeah, just yeah a few seconds ago So let's go bgd here. Um, again, all this is a standard Um, except here has this this route, right? Where has the the ttb ttb? Um, when I click here, it'll give me green Right, so it's the same. Get ups, baby. That's right Um, so the uh, so the idea is here is that I have a single source of truth But then I can deploy it multiple times and I use customize in order to have change So this doesn't have to be like environment variables or it doesn't have to be Things like routes, right could be also things like scale, right? Like I want to run You know Three in one data center and I want to run four another data center maybe because you know one data center Receives more traffic than the other and I want to be able to have different scale, right same app different scale Um, so you can pretty much use uh, customize for all for all of that and um There's there's other methods to do it, right? I think case on it is another one Yeah, there's a few out there. There's a few out there. I think customize is the most uh, the most popular one Yeah, um, it's the one it's the one I use it's the one a lot of people use You don't have to use that one. The the idea is is that you have single source of truth Yeah, and use whatever tool to um differentiate And um One thing and I'm surprised no one has asked about it, but I'll bring it up. So someone I was waiting for someone to ask Um, is how do you how do you deal with secrets? Right? Um, like why like you're not storing username and passwords on git um Right because sometimes secrets. Yeah, you definitely think matters have that. Um, so the idea is that Um, most people in um, they're using either flux or argo or use use something called That they're tooled by bit nami. What's it called sealed secrets? Oh, yeah, um Or uh Hashy corp volt. Yeah, you can use vault. You can even use um If you're using ansible like if you're using ansible as your sync tool You you have ansible uh secrets as well, right? Like you think yeah, like you you could have all of like Let's say you had your entire, you know Argo cd repo, right your vgd repo just completely blasted out with tons of clusters and everything else, right? Like you could actually use jingetoo templating Yeah, and ansible secrets inside that and you know have ansible kind of Manage that for you in the process. Yeah, it takes it take a little bit of glue, but you can make it work You can make yeah the the idea is that So the idea so the answer or the idea is yes, you do store your secrets and get you store them encrypted Very yeah, yeah, yeah, so you store the encrypted version and get and um your secret has to live elsewhere Yeah, and then it gets decrypted at runtime, right? Only at runtime. Yeah, it only a runtime it gets decrypted And that's why vault and sealed secrets and all these things exist is because it's very hard to trust That you know all of a sudden aes encryption is going to be broken or you know, right? Like you don't necessarily want all your secrets and get right like but sometimes you have to put them there um But like that's why vault exists and you know shout out to them for making you know sealed secrets a thing and their most recent version and Yeah, like making that more usable in these cloud native environments is great Yeah, so the so the the idea is that yes, you do store it and get because um You know and this problem existed before right so like this was like this is always a problem This is always been a problem. Like where do I store your passwords, right? Yeah, what do you store your passwords? Yes, that's always an organizational problem and now it's an application problem. So Exactly, you have to solve it. Yeah, you have to solve it somehow because yeah Where do we store because like, you know, you're behind a firewall. You're in a You're in a a private get repo and still feels funny And I can you have like all the rbacks is you know in place every rback every everything AES 256 encrypted secret. It still doesn't feel it doesn't feel yeah It still feels weird to put it on get like it just does it's a pair of dime It just does and that's yeah, that's why I don't I don't It's a personal preference. I I've seen people do it I have done it in the past where I put the secret up on get and it hasn't caused me any problems, but Yeah, it's still it's one of those things that's like in the back of your mind It's like all those secrets and get is like it's just waiting for something wrong to go happen Maybe yeah, I'm gonna run an ai like right like it's someone gonna dump this file and spend a whole much of amazon credits To get into my infrastructure kind of deal right like who knows right? So Yeah, if you want to use a different tool Uh, that doesn't result in putting your secrets in the repo. No one is complaining about that. Yeah Yeah, we we get it. We get it. Yeah, totally. Yeah, totally get it. Yeah, so cool Oh, man Yeah, someone someone asked about like how does this fit in? Um The life cycle pipeline. Yeah to a pipeline. Um, so this is so the the reason it's called argo cd It's because it's a delivery system right argo cd flux cd You know because you can do a lot of these a lot of these principles Using like a jankins pipeline right and just having jankins having jankins Fire everything do it do it for you, right? Um You can do it that way like there's no, you know, there's the there's no hard and fast answer But the idea is that for your ci you're still doing it either via via jankins. Everyone's doing it via jankins Tecton people are doing starting to do things with tecton now. Well, I mean, yeah Do that that that iteration and then when it's finally time to deploy it's as simple as Merging that branch to whatever argo cd is monitoring, right? Like whatever yeah, so you if if you set up the branches twitch You know any kind of forks or changes that come in get merged to twitch and then argo cd updates everything Yeah, yeah Exactly so that the the the argo cd becomes the the deployment mechanism, right? So right It becomes how you deploy now, right? So that's the That it could it's it's It's towards the end of your pipeline Right, so you're you're still doing, you know, you're you're your maven builds the way you would normally be doing Oh, yeah, you're still compiling your code. You're still put packaging everything up the way you need it You still all that stuff still happens. This is the the we're getting it out of the door now, right? I guess now, you know towards the end you're building a container and then you're like patching a You know a get repo and then you merge it and then you're magic as you see So can you there's a question in chat from jay warnica. Could you schedule pulls? Could you yes, you could create a crown job? I guess to fire argo like a kubernetes crown job. Yes Yeah, you can schedule pulls. Um, are there webhooks? Yeah, I think you can yes, there there are webhooks. Um I saw that I saw this the other day while I was playing with it But there are webhooks that you can hit in order to trigger so like instead of it pulling As soon as someone merges it it'll it'll do a it'll hit that webhook and it'll it'll initiate Yeah But that's another thing you got to keep secret is your webhooks Yeah, because you don't want someone to like, you know, span your application That crush your infrastructure. Yeah. Yeah, exactly. Exactly. There was um, I need to find it. I need to It was um, but yeah, you can do like webhooks. You can do when you're doing a um, an automatic sync policy Mm-hmm. Um, yeah, if it's automatic, it just goes it just goes. Yeah But if you can you set it to manual and then you can have webhooks hit, um the webhooks at the end so Yeah, awesome. Yeah, cool, man Any other questions from the audience out there in twitch land? Um Related back to my question about tech tom. I'm sorry. Ricardo. I didn't see your question about tech tom. I'll just go up real quick Not seeing anything we didn't enter Lots of help from the chat, which we always appreciate. Thank you so much. Yeah Oh, yeah. Yeah. Well, someone says if a continuous pool cause additional load on the api server. So, um, not So it uses so it's ready to break this out loop, right? Right, like it's not any different watching for events and when it sees an event. That's when it fires Yes, not any different than watching for pods, right? Right. Like if a pod goes down, it's just going to pick it back up, right? same premise So, yeah, exact same premise. Um, so you got the uh, So it's an additional load in that Not any different than like watching for pods Yes, correct. So, all right. Awesome. Let's uh, let's wrap this up. How about that? You cool with that? Yeah, I know that's it's it's good Okay, so like in an hour like just about an hour I'll be back on here with a large group of awesome people For uh, what's new in open shift 4 4 for developers? So if you're writing code every day on open shift, definitely join us here at the 1700 utc 1 p.m. Eastern Uh, and then later today, uh, three o'clock p.m 1900 utc we are going to break down the Open shift cluster autoscalar So, yes, that's gonna be right. Yeah, Eric's gonna go through that. That's gonna be really cool Eric's gonna go through that and i'm gonna play the humble idiot on that one because You know because to be honest with you It's a tough job, but someone has to do it, right? Right. Yeah. Yeah, because this is this is what I do on this show I play the humble idiot. Nice. Nice. Uh, yeah anyways, so thank you all for tuning in. Um, there are also, uh Our dev nation friends. I want to give a shout out to them real quick Because they are doing some really really great work if you're interested, you know and like kubernetes and how to get better at doing these things check out dev nation they are on actual like master courses is what they call them and Like our team Works very hard on packaging up this this information making it very consumable and very easy to Get in this kind of personalized format. So check out dn.dev Slash upcoming we've got some of some of our teammates Christian are gonna be doing one of those kubernetes ones here pretty soon. Cool. And then uh, yeah There's a bunch of stuff upcoming there. So definitely check. Yeah, I think landon's doing one and yeah Lange is doing one jafar. I think is in one eric's in one I want jafar to do a french one in friend in french. Yeah, I mean they do multiple language Yeah, so i'm gonna even though i won't understand it. I want to drop in just so I can see it Right like it would just sound good. It would just cool. Yeah, exactly. It would be nice to hear something different. Yeah Yeah, so oh also, uh, when in doubt like if you want to try open shift kick the tires light the fires Open shift comm slash try try you can get anything from it running on your laptop to it running on your data center out of open shift comm slash try Yeah, um, yeah, make sure you you sign up for the developer developers dot. Oh red hat.com As soon as you get that you can get access to open shift open shift comm slash try Um, and it's not on your laptop install it on aws bare metal. Whatever. So yeah Yeah, uh Sasha and cloud 99 asked is there a way to ask questions on recordings in an offline way? All the if all the sessions are recorded um Comments on the recordings. I would assume. I don't know if you can reach you can um feel free to like hit us up on twitter You can you can tweet us email. Yeah, twitter. Um, or i'm on the kubernetes slack as am I? Yeah, so you can so yeah, so i'll answer there if I have a few free cycles. So cool Yeah, definitely on the kubernetes slack both of us are definitely out on the twitter lands and emails So please feel free to uh hit us up Cool, man, you have a great rest of your day and I will uh, I will send this out All right. Thank you everyone We'll uh till next time