 introductions and yeah welcome everybody again to the webinar. Very excited that you're here. This is a topic that Rohan and I are both very passionate about so hopefully the next 40 minutes or so there's a lot to learn and yeah keep the questions coming. We'll wait till the end to go sequentially through the questions and so if you have any sort of questions or comments please please add them. So who are these two people talking at with you today? So first I'm Robbie Lochman. I'm an evangelist at Harness. In my background I've worked at a few firms. I've worked at IBM, Red Hat, Investors Fair and so distributed systems is my game. I had way too many outages, blew the error budget. Now I was told I can't touch production anymore and this is where I met my friend Rohan. Rohan, let me tell the folks a little bit about yourself. Hey guys, Rohan Gupta here. I was a software engineer at G. I worked on the CICD tool internally and then you know my journey kind of took me to Harness. We love CICD here obviously and you know I was a solution architect so I really helped you know users adopt the products and really learned about CICD and governance along the way and I kind of fell in love with this topic as we were you know exploring the space and now I'm a product manager and get to enable all of us to you know build cool things with our product and be able to deploy seamlessly. Now awesome Rohan and so what I'm going to be talking about today. So we have four main topics we're going to kind of cover. Just as a level set what is continuous delivery so we're just not talking in the ether a little bit about introduction to what even is a policy and then the genesis of OPA and what is OPA and then for the fun stuff open policy agent and continuous delivery better together. So we'll walk you through a few use cases that we've seen and also some of the art of the possible and then also we also participate in another Linux Foundation sub foundation which is part of the CNCF the continuous delivery foundation. A lot of acronyms there but also a lot of the work that we're doing in the CDF is helping everybody leverage continuous delivery and so also how you can get started and join the CDF. So continuous delivery what is continuous delivery so continuous delivery here's one of the the always a big fan of books and there'll be a couple books throughout the presentation but here's one of the most quintessential pieces about continuous delivery that is named continuous delivery by Jess Humble and David Farley and so basically continuous delivery is all about getting to production safely. The keyword there is safely in my career I've gotten to production a lot sometimes it hasn't been very safe or people will question that wasn't very repeatable or that's a very bespoke process but continuous delivery is about making sure that the path to production where your customers are so your internal or external customers is a safe and repeatable path and that's it right so it's easier said than done but we can get into how this is actually a little bit tricky that because like all of us there's no one path to software and so let me show you what I call the lifeblood of an application. I like I like doughnuts so in this case the doughnut is an application but similar to medieval times the humors or humors there's typically four humors to power an application so clearly an application needs some these resources an application will need compute resources it needs to run somewhere so at some point you don't need some sort of durable storage either how is the application binaries or data so you need storage networking it's like if a tree fell in the woods can you hear it an application that doesn't have any sort of connectivity this is an application and then also your application needs application infrastructure so take a look at these things yeah landscape there's you know thousands of cards out there that these particular projects are coming up that will help power the next generation of cloud data of applications or if you're not using cloud data that some sort of application infrastructure that will help you power this delicious doughnut and so let's talk about what it actually looks like when we call a CICD pipeline or a continuous delivery pipeline so I call this all in the pipeline right and so basically there's multiple steps just dropping a binary into production now some of the newer mindsets out there is you know what maybe when we have one particular manifest or one particular resource you have to drop but that's it there's a lot of rigor that goes into making sure our changes are sane and making sure that our changes are safe and usually there's this two right end of the radio dial on the left with these earlier stages you need to enable innovation right so there'll be a lot of iteration going on if you watch how I write software it's actually kind of dirty I try and try and try and try and try to like it you know get it right and that's probably you know a number of times but too many to get something right but also you don't want to be keep trying as you get to production your confidence needs to increase in each sequential step that hey we're building more confidence so by the time you get to production there's not a particular your change failure rate is low so going back from left to right hey you know what in dev orchestrating such things as a code analysis maybe what I've written I want to make sure that those functions are right some new unit tests and then maybe I'll pretend throughout the example that Rohan and I are in the same team which we are what I wrote works with what Rohan wrote so if I first said usually it's the trouble maker in two now kind of getting into now we're infrastructure where in that middle stage right so we've deployed it somewhere we want to make sure that we didn't do any detriment to the infrastructure we're running smoke and soap tests and performance tests and then as you get to production we're actually leveraging safe approaches such as a canary release and I'll explain what a canary release is really quickly but really core to this delivery is making sure that safety is there right so is that one and done a lot of times we're not deploying something for the first time the reason why this is so important is if I was like version one and we're rolling out version one like who cares but if it's like anything else you have to make a change making sure that your users aren't impacted that becomes a big business so really quickly about a canary deployment so basically going through the journey of what a canary is if you're unfamiliar with a canary deployment it's an incremental release and so here the user is represented by a taco love food and basically so let's say you are the user and this delicious taco through the magic of load balancing we're running incremental releases so in the application versions here they can let's say chunky otter is version 1.0 and the canary version 1.1 with the first incremental release it basically you're making sure that the canary survives right so like in this particular version we have we deployed 33% of the canary and then still stable as two chunky otters in the sequential pass when we determine everything is correct we promote the canary to 100% of the traffic and chunky otters no more but this is an oversimplified example lots of times canary itself multiple phases and there's very bespoke rules and how to deploy lastly before we get into what a policy is I like to always bring up the confidence trifecta right and this is a rationale for why do you have a policy why do you even enforce anything and so in the DevOps marker you always hear people process technology right so these three more chance but let's talk about let's break this down where are actually you the least confident well let's start with technology it's actually fairly you know I would say it's fairly objective not subjective to be confident in technology for example I have a Kubernetes cluster and there's a reason I have five worker nodes on that Kubernetes cluster because I can take two concurrent failures of a particular application right so I'm fairly confident in my distributed system design that I have a robust and resilient piece of infrastructure that is that is objective where it gets a little bit more subjective an extremely subjective is it's more the process could be subjective right so certain industries have more stringent change management processes because you know why the people we're as people were emotional and we're extremely subjective right so where do you have the most confidence and we're going to by using something like OPA we're able to blend all three so you're able to have a process that is confident you're technically confident in the process and also because of the people who are authoring it you self enforce the policy which we'll talk about a little bit but that brings us to the next word a policy so what is exactly a policy I have a Geico insurance policy on my car but these policies are just a little bit different so a textbook definition of a policy it's a deliberate systems and principles and goals to enforce a rational outcome right so a policy might be you know you have to be 21 to drink because you know you might make bad decisions if you're not 21 or in computer systems you know what your password must be 25 characters my funny story my sister started a new job and her password had to be 25 characters and I was cracking up I'm like I can't even think of a 25 character password but that is a IT security policy that she has to abide by so it really defined the goals and elements of your organization's computer systems or just principles in general is a policy so that is what a policy is let's talk about another conundrum with that a lot of us are going through right if you haven't gone through it yet you'll go through it soon enough so here I have my friend Rohan again as we ever said about Rick but Rohan is a new site reliability engineer on the team or he's a new SRE in our product team now let's take a look at our product stack this is a pretty forward-thinking product stack so we are leveraging Kubernetes for the app stack so we're leveraging Kafka we're leveraging several spring projects and we're using CockroachDB as our persistent stack and then kind of looking at the non-application stack so for our observability and monitoring and visualization we're using Prometheus we're using FluentD as aggregator and then we're displaying everything in Grafana cool right like this is a modern application stack I would say this is a very forward-thinking stack and leveraging several projects from the Linux foundation in our stack but we have a little bit of a conundrum here this is what I like to call see this is what I like to call the conundrum so let's say that we have a problem right and so Rohan actually needs more access than what he actually has and so Rohan being a new site reliability engineer like let's say you're gone forbid we have some sort of issue or outage an incident he's a more correct term and Rohan's using the team Rohan needs more access so in the grand scheme of things there's two-part combination there's authorization on an unauthentication authorization would be does this person Rohan right and so yes you know we have credentialing like Rohan has a harness email address and you know we have octa and all that good stuff so it's like yes he is who he says he is simple that's solved but what is not solved in this distributed system problem is can Rohan have elevated access that he needs to get data or to get information for us to have a recalls analysis on the systems and answers no and if we have to go and make a change for Rohan to get more data or to get higher let's say um author or higher authorization uh it's basically a redeploying to Kafka I'm redeploying to the spring stack I'm redeploying to cockroach I've redeployed to all of our monitoring observability stack and so as you can see a simple request for you know we know who Rohan is but he needs elevator privileges is a problem and there's a lot of deployments that we have to do and so this is the exact problem that OPA steps into right so enter OPA so regardless if it's harkoff with you regardless if it's you know our Prometheus stack regardless if it's the application stack or even you know the database stack uh Rohan's still here right so he's still you know he's he didn't go anywhere he's there Rohan but um but that's basically it OPA acts as a policy agent and so it focuses on the authorization piece right like what are you authorized to see it could be entitlements it could be the privileged data but basically OP OPA sits between or actually it sits above depending how you want to look at it uh these particular services because in a modern microservice architecture uh you know as brilliant as microservices are you're having more endpoints so you're having more deployments but kind of that connective tissue is actually communication right so the four main benefits a whole big benefits that OPA has is that it simplifies authorization across remark services so no matter what service you have we get a double the number of services that we had or triple it's a piece of cake for OPA because it's the central cop between those authorization calls it uses common remote protocols so HDP HPS also if using SSH or gRPC the OPA understands the particular protocols because OPA basically will make a decision on that and clearly between all of those services there's some sort of payload right like you know it's not uh you know mystery binary getting shared across or cobra actually that got removed from the java spec recently but it's not a mystery right like it's it's a payload you can make a decision on and OPA has this particular uh particular language called rego it's a declarative pausing language so you say hey based on these conditions on this payload what is the action you want to do the last big benefit before I hand it over to my brother Rohan is that this is chicken and the egg question right like it actually solves a very intrinsic question and usually the answer would be no to this uh take OPA out of the picture how often as an engineer do you have access to production probably not you know as a software engineer right unless you're like a let's say assistant engineer infrastructure engineer a cloud engineer you rarely have access to problems there's business controls to separate you away from production because technically if you write it you can't enforce it right if I if I wrote something they'll never let me see in a production because you know I know all the ins and outs and can I be trusted to enforce something but OPA actually does this right OPA actually allows the authors of the system to enforce the system and so with that let me turn it over to my friend Rohan to talk about how do you actually how can we start leveraging OPA and enforcing certain rules and uh continuous delivery thanks Ravi so one thing that we've been seeing you know with OPA and in this continuous delivery kind of spaces how do I centralize you know my pipeline deployments in the sense everyone has the ability to kind of configure pipelines and leverage pipelines but how do I know you know who is using what who is doing what role in this process because enforcing a standard pipeline across some entire organizations or teams it's hard but like you know I might have access to a dev environment my manager might have access to the prod environment we might not just because we know each other really well or you know we have a good working relationship maybe we you know hand off you know we we ease up on the restrictions a little bit how do we make sure that you know the business policies that you know our sec ops teams or operation teams are providing they are enforced they they they have some sort of structure in place where we have that kind of separation of duties we have that idea of a standard pipeline kind of design we have the ability to balance between this rigid structure and I still care about my developers freedom I want them to be able to you know build pipelines design pipelines and also deploy frequently because we also have the job of you know delivering features at a high velocity for our business so these are kind of the problems that we we we see in continuous delivery but we also see opa as a really is a supplement to this process where we can start putting some structure in place and enforce it in the lower levels and it's stronger together and we've seen that because you see as opa it's it's nice as a as a drag go essentially so you have the ability to kind of define this as code you don't have to be the best programmer to figure this out it's pretty pretty high level as you don't need to know Java or Python it's it's very much just looking through the json and kind of picking out which fields you want to enforce because the whole communication in opa is by json so it actually is really helpful for you know sec ops and compliance uh base people because they they might not need to know how to write you know Java code or Python code to build a system to kind of enforce their business policies on like a pipeline it it's very it's user friendly it reads nicely and you can kind of figure out based off reading it how what the rule is what's it going to enforce so you as a compliance or sec ops persona would definitely be able to write these rules and then you can rely on your you know dev ops team to kind of deploy and enforce these roles and that in itself enables teams to kind of deploy in a consistent manner that's not only compliant with their business but it also enables them to have a repeatable successful kind of process and as you see here it's opa is you know designed for kubernetes it was uh you know it's really good at managing the config management it's it's it keeps you know me from specifying resources that are way too big and i'm not blowing up my cluster every week or i have a bunch of teams deploying into the same namespace and it's starting to make things make a mess in that namespace so i have to clean it up later we can really kind of have structure and enforcement around how we deploy our resources and kubernetes without having such a deep technical understanding of kubernetes and the underlying you know behavior of how pods are getting scheduled and who's taking up what resources we can put guardrails in place so we don't have to deal with those kind of lower level problems and we see a lot of benefits from it we've seen users really love this kind of granular structure because i don't have to configure the r back and all these different systems i have to configure it up the kubernetes level i have to configure it at the git level i have to configure it at you know the the grafana level i have to configure it at the app d level those kind of tools that's that's just part of the equation i could have one system to define it and i can have one system to be the interface for it that's that's the the benefit of you know adopting more of like a policy driven uh enforcement kind of approach with your software systems and so some of the problem spaces we can as you saw in ravi's earlier slide around deploying to your dev environment deploying to your qa environment you saw that there was like smoke tests regression tests um there was like scans those kind of pieces are very very uh crucial in your deployment pipeline so there's two parts to this one is you know opa i feel if you're familiar with it a lot of people talk about the r back capabilities and it's a very very powerful tool for that but i think other pieces of this that opa can be applicable for is like quality decision and evidence the reason why is because when you're doing a deployment you are running your tests you're running some scans or making sure the image doesn't have anything that's vulnerable but as it progresses through the deployment you lose sight of those pieces of information because you just you're just seeing it the results and you're just continuing to let the build go through the process what we want to what we see is people want evidence that before i say hey this app is going to production i want to see that you've actually hit 90 you know percent on your unit test i want to make sure that sonar cube gives you a great grade i want to see that you know your nexus iq you did a nexus iq scan i i also want to see that you know when you deployed that canary it ran for like 50 minutes and i got metrics from it from app dynamics i want to see all that evidence before i can concretely say go to go to production with my deployment so we've seen it opa as like that rule engine where i can collect all that data and format it in a json format and send it over to my rule engine to dictate did you meet the score is this good enough and that takes the human element out of it so i don't have to chase down these things i have a system pulling it in and then i have a governing system which essentially iterates over it and says hey you failed or you passed there's no gray area you can't say you can't say uh i think this is okay 89 versus 90 that's okay no we we it's at zero one or either in or out that's just people right rohan that's the second part of it that's the difference yeah yeah so we yeah exactly and so we're passing judgment based on evidence and opa really really enables that with the users now the second piece i mentioned earlier you know separation of duties right i i might be you know a developer and ravi will be my manager this case right so i wrote some code and i build it and it triggers off my in my pipeline so it starts building we'll put it through the dev cycle the qa cycle and then hopefully promote it to prod but during that process you know i might need ravi's approval to get to production and so a lot of you know regulated businesses don't want the developer to be the one to approve to production because sometimes what we've been seeing is you know we're we're a team we're we're friends outside of this and so sometimes that kind of spills into you know our process where we say ravi might say hey rohan uh i don't have time to approve this to prod can you just hit approve and let this go through i'll be back later and you know i'm i trust you because you know we've been working together for five years and and things are i trust your judgment however that doesn't fly when i'm building apps that are like touching you know healthcare data or my financial information or you know my insurance information i don't want to i don't want people i want to make sure that the software that you know that i'm i'm i'm consuming as a user actually kind of follows some sort of process so i can and that builds trust with with the user and the consumer so the separation of duty piece is extremely important because in an internal business they definitely don't want the developer to be the one to approve their own bill to production it that's just not going to fly and a lot of you know compliance officers look at that kind of information and logs to say and the reports to say hey are you guys following us a good practice to make sure that you know the consumer is protected and so we see the separation of duties problem come up numerous times and so we think that you know OPA is an excellent you know solution to this problem because you know i can have a list of approvers stored in a kind of a third party system and when i get to the point of deployment you can automate the decision-making so ravi can't let me you know approve to prod and go to lunch you know he he's got to say hey i actually have i'm the only one who can approve Rohan's bill to prod i this bill is going to hold until i do it or i have to reject everyone else who's trying to bypass this rule so OPA is that kind of buffer to say hey you can't do this only x amount of people can do this so this might be ravi this could be my security team this could be my my like head of release someone else besides myself has to perform this task sorry i'm laughing because i've done this uh numerous times in my development days i i've i've over provisioned you know resources and i've crashed clusters i still you know even when i give demos today i create some resources and you know it crashes the demo environment and so deployment configuration sanitary is sanitary this is huge like when i deploy my kubernetes application or my ecs application uh how do i make sure that a the image is not coming from the public docker hub i need to take it from like a private s3 bucket or an artifact like an artifactory or how do i make sure that in the kubernetes like container spec i'm not taking a massive amount of cpu or memory or how do i make sure that the pod has the right labels so i can track it in my other business systems OPA you know really does a good job of being a buffer because essentially before the deployment you can catch that whole thing through gatekeeper and it will check to see hey do you meet these kind of criteria it's like you can write rules like making sure my image is not coming coming from a public repo making sure cpu is under this amount and making sure memory is under this amount and what you can see is those rules are being evaluated and you know that you know there's a guardrail in place that as we're going through the cicd pop process there's a system checking to make sure that my config actually you know it matches my criteria at an infrastructure level and even at the at the config level as well making sure the config is sanitary and my infrastructure is you know safe it's it's de-risked because of this kind of governance we definitely don't want to do you know all our best testing and production i get your aws bill rohan so definitely you're very prone to over provisioning so thanks for my points for my next vacation whenever that would be but yeah was there anything else you want to chat about about use cases before i introduce the ecotinous delivery foundation um i also wanted to talk about uh one last piece which was around the around like enforcement of in the deployment process what about like blackout periods right how do i make sure that people aren't bypassing those how do i and how do i like build a system that you know that can enforce that without me having to say hey guys you are not deploying on friday because i don't want to be on call saturday dealing with this how can i make sure that you know there's a check in place and opi actually can be leveraged for that as well as like a like a as a rule engine to say hey deployments cannot happen on friday deployments cannot happen on monday and so we we have you know a deployment configuration uh is one part of it but actually the process of deploying that at a higher level can be managed by opi as well because at runtime you can definitely send the system like a check saying hey a deployment is starting and then opi can say what's the time take like the metadata of what the time is what the date is and you could very much design like a rule to say hey enforce the blackout window you have to deploy on tuesday wednesday thursday you can't deploy on monday and friday and reject the deployment so you have that peace of mind again i talk i spoke about guardrails it's extremely valuable kind of as an interface for those guardrail policies that we have in place in our organizations today yeah that's perfect and i think we know rohan and myself we talked about there's a lot of rules when going to production right there's not just one you know past this there's there's lots of incremental confidence by going back to the technology choices or process choices even the people choices uh you know how do you enforce that an opa is a perfect rules engine for that you don't have to understand like drools or i log or some other you know fancy rules engine it is it's simple enough that more people a wider swap of skill sets can actually implement it and a lot of what we talked about today it's it's ongoing right like it is how do you bode particular rule sets rule sets that are hard coded or rule sets that have been ingrained deep in a system to something more abstract and that's a lot of the goals that we're seeing with opa open policy agent and continuous delivery but depending on where you are kind of kind of close out with a few points here as you're going along your continuous delivery journey or if you have no idea again what continuous delivery is there is a Linux foundation sub foundation to help you out uh so i want to introduce pay homage to the continuous delivery foundation aka the cdf uh which uh we're a proud part of now so basically the continuous delivery foundation here's actually a presentation in a presentation i actually gave this presentation the screenshot in the middle um called how much should you deploy and i suffer from something called photo photo fair of deploying we're actually this is fun fo d but how much should you be deploying right and a lot of being in industry a lot of what we see uh you know if you're if you read this another book uh lead enterprise uh you might see you know what i need to be deploying a hundred times a day and then you might google it say am i am i crazy how many times a day does a big software from deploy and you don't have to be deploying every 12 seconds right like if you're deploying every six months and you get that down to every three months this is the production right you might be deploying to uh you know a dev interenvironment a hundred times a day but getting into production if you're going to cut that in half you know from six months to three months you're incredibly more agile uh leaving with it's a new a neutral home for many of the continuous delivery tools and projects and so there's several engines out there which harness participates in and so if you're looking to get more into the continuous delivery space the cdf is a great place to be organized there's several special interest groups and working groups they're there so if you want to participate i participate in a few of them extremely open and valuable for you to come and learn more or come and participate in expertise but with that if you would like to get a copy of the presentation or learn a little bit more what you learned about today feel free to hit up this bit.ly or give it a qr scan and so we can actually take a look at what questions have been been asked now so let's take a quick look here in the question and answer section or in the chat so if you have any sort of questions feel free to ask them that looks like our question q is empty uh Rohan and i can answer anything about uh opa or about life in general um if you have advice uh where to eat you know we can give you some some of that too so don't be shy we're here for y'all just give it a few minutes Rohan what is your favorite coffee mine uh i've been actually playing around with the fills uh the testara one or tessara it's been pretty good i like the mid behita one i know i yeah yeah the all right we got some good ones okay how can we attend the slides or the recording um you can hit up this bit we link here i believe the linux foundation will publish not too long after uh recording um what is your favorite distribution of linux Rohan oh man that's a good question that's a good question um i i i'm a big like you know enterprise guy because you know i i started in enterprise it's very much like red hats distribution of linux that's the one that i've kind of learned on and i ride that out but yeah that's that's mine ravi do you have a favorite one i like mint i'm on the mint mojito train i like mint linux it's easy it's been something i'm born to so um i'm a big fan of a big kind of mint actually it's on my mother's laptop i gave to her start learning uh let's see so is there a demo we can do with harness opa um but we actually do have a demo we had a we have a demo but i we actually have a lot of uh blogs that kind of uh in on our harness site that actually go through how to configure it um in harness and how a deployment works uh ravi actually a a colleague tiff um wrote new a couple of them uh and the team can kind of here can kind of check it out and and see kind of how it works with harness and even outside of harness just how how you could design a rule how you can apply it to you know your kubernetes cluster and then you can even test it outside of harness just try running you know gatekeeper and running cube ctl with some sample manifests against it and you'll see kind of how the rule engine behaves yeah perfect um and also yeah we do i think her example is really a really really eloquent one or a simple one it was like uh blocking a namespace that shouldn't have been deployed to so uh makes makes perfect sense um so another question internally do you use opa just for primary deployments or do we use it for other workflows or what have you seen so internally we've been working on some cool things on around you know how we could regulate deployments but also it's it we're looking at it more more it's more than regulating kind of the cluster we're focusing more on the rules engine and kind of how we can you know make a make a judgment call based off off of it so that that's kind of where we've been uh looking more into more than just the restriction of deployments because restriction of deployments um itself opa does does it because it's you know managing the cluster and the configuration before you actually do the apply so uh the the things that we're kind of looking into are more along uh making decisions for you and for for those taking that question one step further i think um always for uh a harness implementation it's not necessarily restricted to kubernetes so you know opa's correct it it grew up on kubernetes but uh you know a lot of times you might be in between like you know you might your organization might not leverage kubernetes or you know which is very valid or you might be having partial apps right like hey like 10 percent 20 percent is on k8s and so just manually going just reiterating for what Rohan said like more it's actually very powerful rules engine so it can be running to orchestrate things and making gay decisions for pipeline decisions that are eventually the end point is a vm or another linux insurance for bare metals so yeah good question is there anything else we can answer feel free well i think we're all set looks like the questions have slowed um so yeah i just thank you everybody for your time uh today there'd be this morning this afternoon this evening uh really really glad to hear these great questions and uh yeah cheers everybody great thank you so much to ravi and rohan for their time today and thank you to all the participants who joined us as a reminder this recording will be on the linux foundation youtube page later today and we hope you're able to join us for future webinars have a wonderful day