 Welcome to another episode of Authorization in Software where we dive deep into tooling, standards, and best practices for software authorization. My name is Damien Schenkelman and today we're diving deep into the open policy authorization layer, aka OPAL, with Gabriel Manol, head of DevRel and Growth at Pervert IO. Hey Gabriel, it's great to have you here. Hey Damien, that's actually great to be here. Super excited to speak about authorization every day. Yeah, I always look forward to chatting with folks that do this, let's say, for the living. Could you give our listeners a view of your background so they learn more about who was your guest today? Definitely. So my name is Gabriel and for the last, I'm not counting years anymore. I'm doing software engineering. What I'm counting is my years in authorization, particularly authorization in the field of access control. So in 2015, I started to work at Cisco in a product called Cisco ICE, which is like a server to authenticate and authorize user to the network, which is kind of something which is rare today because today we're using zero trust, so we authenticate and authorize user in application instead of in the network. But that was a thing there. I worked there for five years, then I moved to Palo Alto Network, also work around security operation control rooms, and then most of the stuff was about broken access control. And for the last couple of months, I'm working at Permit IO, a startup that bring authorization for application developers as a service, which is a trend growing now, but now I'm doing it from the go-to-market perspective. I'm the developer there, so I have the feeling of the user, and I'm really happy to help developers implement better access control and particularly authorization into their application. How did you go from those positions where again there were hands-on to kind of like a startup and doing head-of-growth development? How did that transition happen? So let's say from 2018, when I was back in Cisco, I participated in Hackathon, and this is when we started to see everything regard zero trust. So we have like huge customers doing network auth, and they have like thousands of switches that deploys in many sites, and all the authorization, all the authentication was like with network credentials. But then they are like started to use the cloud. I mean, it sounds late, but this is how things happened in Cisco and huge enterprises. They started to use cloud and then they come and say, hey, I need to authorize my user based on cloud data, maybe even serverless stuff. And we don't have a way to do it because we like a server that sit together with network equipment. So my idea was let's create like cloud extensions for authorization. So when we do authorization decision, instead of just doing it in the local setup of the authorization server, we can go to the cloud. So I developed something like this in the Hackathon, and then it was huge success. And then I promoted to do a technical leadership in CTO officer. And this is where I started to meet users, right? So from since then, I'm not working on traditional R&D teams. I'm working more on CTO officers and, you know, like more innovation teams. And for me, the next step was like, do more of the business, do more of the PN. And that really is like the nature of place for me currently, because I can start every week a new software project. I can see tons of types of architectures and deployments and even experience code and also experience this manner of, you know, go to market and do the business stuff. That's very interesting, very interesting journey. And it all started from starting to take authorization logic away from the networking layer and back into the application layer, not relying on the network and network membership to make authorization decisions, which is a typical thing that as you say, as you move to the cloud, you start seeing, but again, some enterprises don't do it or not all components do it. And it takes a while for this thing to happen. The topic of today is OPAL, the open policy authorization layer. And I thought that again, that's a good start because kind of like it leads us in that direction, right? What's the technique that you can use to extract authorization logic from different components into a set of policies and then deploy those in your code, make decisions based on software. Before we dive deep into OPAL, I thought we can maybe do a high level overview of policies and attribute based access control. This is a topic that we discussed a lot on the podcast, but it's always good to kind of like do a quick refresher. So in your own words, what's an authorization policy? How are they related to roles to attribute based access control for people that maybe are new to this and aren't experts in the topic? Yeah, definitely. I always love to start, you know, with the basic understand what we trying to account. And we'll get to it later. But I think this is one of the most challenging for developers to try to do authorization. So there is a whole topic of access control and application, right? So we have application, we have their data, we have operation that we need to do on the data. We have a lot of stuff that can be done through our application. And then we control the access to the application. And we always like to stay in the comfort zone of authentication, only verify who our users is. But in the real world, we want also, you know, to maybe categorize the user, some users can do something like this and some user can do something else. And this is where policies come to play. The most abstract manner of policy is like a simple text sentence, like is a monkey allowed to eat a banana, right? So we try to understand who is a monkey and we try to categorize the action that they can do like eating. And then we have resources like banana. And then we have a lot of policies that we try to implement. And by the way, this is the way I recommend developers to start think about authorizations, thinking your own word, define the policies in your own word. And every application, of course, has like a different set of policies, but we need to streamline them, right? Software is a structured thing. And when we go to streamline, we go to policy models. We spoke about, you mentioned RBAC and ABAC. So RBAC, in my opinion, is the most common permission model because that the simplest one. We're taking the users or the principles that do something in the application and we categorize them by rules. The most common one is like, let's say, we have admins and we have standard users. And then we get a list of all the actions or the operation that our users can do. And then the resources, let's say documents, pictures, whatever it is. And then we streamline the policies. So now it's not is a monkey allowed to eat a banana, is an animal allowed to eat food, right? And this is a way that we streamline our policy in a way that we can have better authorization, better permission in the application. Sorry to interrupt, but I think this is an important word. When you say streamline, what do you mean? Is it in comparison to something else? So let's say when you do write policies, like let's say in free language, right? You write always the sentences or statements that sounds the same. There is a principal or user, there is action or operation and there is resource, okay? And when you implement it in software, when you want to have policy rules that actually software can use it, you need to streamline each of the principles into something that software can understand, right? So the easiest way to do it is using roles and types of resources because this is something we can configure. We can configure roles for our users. We can say this user are admins, this user are managers, this user are just users. And we can categorize resources because this is our data structure, right? We have a database as no matter if SQL, no SQL, all the data is structured in resource type, right? So we have like document. And this is why when we modeling permissions, the easiest way to track and maintain is having the simplest categorization, roles and resource type, right? And this is actually where all developers start. They do RBAC. So the term, the most common term in authorization is let implement RBAC. Let do RBAC when we will support RBAC, right? This is the most common model. In the real world, that's not enough. So let's say, for example, we have a lot of work to assign roles to the users, but then we have more aspects. So let's say we have behaviors. So our users are admins, but we have premium admins and we have standard admins. And sometimes we need to make decisions based on maybe geolocation, maybe the first letter on their name. And also in the resources, it's not enough to have only resource type because we have like sensitive document and then we have document. And if we are trying to keep strict RBAC all the way, then we need to create a whole, let's say, database table just to keep a special type of data for authorization. So then we go RBAC, attribute-based access control. Instead of streamlining our policy in roles as categories and resource type as categories, we are looking deeper on the principles. We are looking at the user attributes. We are looking at the resource attributes. And then we write our policy in attributes manner, right? So if we, again, speaking about monkey and banana, we can see animal from species and food from species by the attribute. So we can say it could be like yellow food because we know the attribute is there. Yeah, that's an interesting thing you mentioned in terms of evolution. As the application, as the users get more sophisticated, you start needing to do more granular things. And that's where something like role-based access control where you have a few roles and very coarse-grained actions that they can take on resources or even categories of resources might not be enough. And as you need more to go more granular, policies become a viable option for that granular authorization decision-making. The thing I wanted to ask is, what's the typical evolution in this path that an application follows, right? I guess not everyone starts using policies. So when do they decide to start using them? How do they take it always and adapt it to use policies? And what challenges do you see teams face when they start doing that? From what I see the first mistake happen on team when they start to kind of design authorization, they are falling into the implementation details of the particular language or framework that they are using. And specifically the term RBAC. So one of the things that I dislike in authorization, I don't want to use that word because RBAC is great and I'm probably using it for a lot of stuff. But the way that development teams, instead of implementing authorization or defining permissions, they look at the framework they use. Let's say they use FastAPI or ExpressJS and then they are look on the RBAC framework that giving them the basic functionality they need and they implementing it. And a week or a month or a year after they understand that it's just not standing. And the reason could be a lot of reasons but the main reason is bugs. And the bugs usually happen when policies start to be mixed with the code. So you started with the framework and probably our audience is developers so let's talk about the most common way of RBAC framework. It has like decorators or annotators or an API endpoint that declared the role that actually allowed to get into this endpoint. Right? This is actually really simple. But then someone needs something that considered ABAC or they need like multiple roles or they need to get a decision in the middle of the endpoint itself and then they start to create like if statement that mixed with the code and then someone get access to somewhere they not should get to. And when they look at the code they see this code usually the problem not started with the framework because the framework aims to be simple. The complex start when they are understanding that instead of writing application they are writing permissions. And then they try to decouple policies from code and there is some approaches today to decouple policies of code. One of them is the one that we are going to speak a lot about today, policy as code, declaring policy rules instead of imperative coding them inside the application, declare them out of the application and only get decisions based on them or only enforced the authorization based on them in the application. But there is also some approaches like and we can expand on it later like policies graph or maybe policy that modeled on databases which is a kind of narrow thing but we saw some implementation of it. So after they are doing like the mistake of implementation details of instead of designing permissions they are moving to policies code. Policies code the main benefit again is the decoupling of policy from code so now the company can have one place where all the policy declared and if I want to see the configuration I know that instead of just guessing what happened in the application code I can go to the policies code right to the declaration of the policy and understand what is the permission and also when I need to manage it I have one place to do it and when I need version it I can couple and decouple it from the application release cycle so in one hand they can version my policy configuration because now it's code and I can use like Git or other version control tools but in the other end I'm not depend on application release cycle so let's say if I release a new feature I not must couple the policy configuration to it and I can like change the policy just in time without deliver a new policy code that also help them then to scale for testing the policy themselves and then focusing the application only in the enforcement right so the way that we are enforcing feature on the application the way that we are not allowing user with particular attribute to access to document with particular attribute the way that we enforce it not related to the way that we actually declare it the declaration and the decision in one thing and the feature that enforce it is something that should be decoupled from it and this is where we see the exponential growth of policy as code in application so after they do the mistake after it is suffered of authorization that coupled together with application code they move to more mature models like policies code and then declaring policies and manage them separately of the application itself that makes sense I like how you went from you start with doing the authorization in your framework which typically is you get a role or something in the token like maybe it's a JWT that comes into the endpoint that you take the scopes from it and then you use some logic to match that with what you can do or not with the endpoint but over time that doesn't scale and as you say if you need to apply authorization decisions at some other points in your code base that you don't have that middle work to make that decision so you start extracting this authorization logic into policies and it makes sense because you deduplicated and it's in one place but there are a couple of things to unpack here on the one hand these policies need to be written in a language and that's kind of like a new thing for developers to learn and then this language needs a runtime or some way in which you can run programs written in that language to get to authorization decisions how does that typically work? Yeah, so the way that policies code work has two essentials components one is the language itself and the second is let's say the runtime so we need to run this language somewhere the runtime usually refers as a policy engine an engine that can get decisions the policy itself is like the configuration for this runtime so if we take like an example of ABAC policy we think that a user, a paid user it has an attribute that they are a paid user is allowed to create a document with an attribute of rich text okay, let's say that we allowed only paid users to create rich text documents and then the way that we model it traditionally we already explained why it's hard but what if we can declare it in a very strict way so let's speak for example on CDER the policy language that crafted by AWS now and released as open source so instead of modeling it in database we are declaring a statement this statement said permit okay, let's think about it like a function permit that get three arguments user, action and resource and then we can declare which of the attributes of those principles are actually permitted so we say we're declaring like a function that called permit with the relevant argument the policy we want to declare and etc and then we have the runtime the runtime know to get this declaration and then you can query the runtime with the data so I'm passing the runtime the argument and by the declaration the runtime get a decision if a permission is allowed or not allowed this way we don't need to model our data we can for example decouple the state of the data and the way that we are modeling the data in the application from what we need in the policy config I see what you're saying where again you have the language and you have this kind of like policy files and you need to version them you need to learn how to write in that language for example you mentioned CDER which means that as a developer instead of writing JavaScript in my JS application in my NodeJS application I need to figure out how to write CDER I need to also figure out a way to communicate with this runtime that's making these authorization decisions via either inter-process communication if it's available, if it's in another machine I need to do it via some networking protocol it seems that we're going from a world where the challenges the developer has are at the kind of like authorization logic level figuring out who has access to what and moving them somewhere else more at the operational level and also kind of like the knowledge level I need to learn a bunch of components and then I need to operate them in order to get a solution to work Yeah, that's correct but when looking at that operation perspective this is actually the way software goes today so what we are trying to do is having developer develop and deliver much more features that are important to our business logic and the way we see it happened everywhere is like having them worry let's call it worry and then we explain how they not really need to worry for the operations because operations happened once coupling logic is something that actually break you to deliver new business functionality so if I need to create more bugs in my code and new code mean more bugs I need to couple it to my releasing cycles that mean I don't deliver new functionality but if at one point of time I need to spin up a new service I need to set up a new service that mean for the long term it will be much easier for me to maintain the configuration correct is not the language that I'm writing code today but who writing one language today right developers are polyglot I myself coding in Golang and JavaScript all the time and all the friends that I know as developers they are not like Java developer anymore or JavaScript developers anymore right we are kind of polyglot and the worry of a new language that the less worry I'm worrying about particularly because it's very simple languages and something that everyone can implement but the benefits it brings in separating the concern of getting authorization decision and the way that you manage it in a different life cycle that help you deliver safely a business value to the product while you maintain great access control in the other end that's a benefit that in my opinion worth it I think there's a trade off here right and it's important to know that there's a trade off I do still think that it's a good trade off to make in general eventually if I was kind of like figuring out how to build one of these apps I would need to figure out how to decouple authorization from my core logic again especially of a state scale as I have more things working on things and things need to be audited and there's compliance and security needs and so on but this is important right like you are adding pieces to a potentially distributed system and you are having people learn they need expertise in a language that ultimately is going to be used to make authorization decisions which means that they need to be competent enough to know that the policies that they are putting in place actually do what they want them to do for all possible inputs and again that's definitely a challenge that's worth considering now with all that being said I know that there's a set of tools that helps with some of these challenges of managing these policies of making sure that kind of like operations and unlike the policy deployment works and diversioning I know there's Amazon Verify permissions like Cedar which you mentioned earlier that has tied us and Opal for Opal what's this category of solutions and what does it help themes building with policies do? Yeah so as we mentioned there is like the operation trade off and this is where those tools trying to get in to come into the picture and solve this operational problem so I do have the components right I do have the language and the file where I store the policy I do have the policy engines but now I left with the operational part of you know deploying those engines managing the versioning of the policy configuration and one thing that we haven't speak about before is the data itself right when I'm getting a policy decision when I get a authorization decision there is data that need to take into consideration let's say the attribute of the users the attribute of the resources etc etc you mentioned three tools and that's actually great because each one of them is kind of representing a different trend in these policies called trends Amazon verified permissions is a tool that offered by a cloud provider to be for developers that you know based their application on AWS and they want to get this deployment of policy architecture together with their application so AWS like offering a service that help you deploy and manage policy engines together with your Cedar policies that you configure in AWS itself and that's actually a great solution for AWS developers that trying to add policies code authorization into application Stara does in the other end is very very focused on the infrastructure side which we don't mention much here but authorization also has an aspect in service to service communication right which microservice can get to which microservice and why and in what conditions etc so this is where Stara does comes into the place as a solution particularly for cloud native architectures where developer want to manage that mission infrastructure stage of authorization it helped them to manage the data it helped them to manage the policy configuration and it helped them to manage the deployment of the policy engines OPAL is an open source tool that the main benefit you can get from it is the independences of policy engine of policy technology you mentioned that we have a challenge the challenge of learning new thing the challenge of learning new language may be learning new models and this challenge is actually growing because at some point some developers understand that the language that they use the policy engine that they use is actually not fit into their product and we can get to it later but policy languages are different from each other not only in the how the language look like but in what kind of policy they can handle regarding let's say latency and so on so forth so OPAL is an open source tools that help you first managing all the policy engines but you're not limited to one policy engine like open policy agent or CIDAR core or whatever policy engine in it and you also not limited with the kind of the policies coders you're using so you can use CIDAR for something you can use RIGO which is the language of open policy agent on for something else or whatever language you want so OPAL is very pluggable for everything related to policy config to policy engine but not less important it also pluggable for the data that you need to get the policy decision so looking at the other tools verified permission is very close to the data that you hold in AWS to all the architecture that you manage your data in AWS STIRADAS is more for infrastructure level application has many many data sources this is one of the things that we see growing that if we think like even five or six years earlier application has one database this is where my data was all the time one of the things before the no-SQL type was a term of one source of truth today we don't have one source of truth we have data everywhere we have data in multiple types of database of our application we have data that you consume directly from other cloud services and OPAL has a plucking system that help you manage all the data you need for policy decisions in one place so think about it like a tool when we can dive later for the architecture itself and how it works but think about it like a tool that with single lines of configuration code like Helm charts or Docker compose configuration spin up a whole authorization system that can be pluggable into any kind of policy config any kind of policy engine any kind of data that you need to get this decision so all the operation trade-off of spinning off engines connecting your data is actually happened once in a very cloud native way I get what you're saying so you're saying OPAL abstracts a couple of things on the one hand it helps abstract operations, takes care of it so that if you're in a cloud native environment in Kubernetes, the agents are available data is made available to them as well but on the other hand it also abstracts the runtime so if you're using Cedar policies or if you're using Rigo policies OPAL does not care maybe we can kind of go a bit backwards in time and start from like what's the history of OPAL how did it get started and how did it go down this road how did it make these decisions yeah, so OPAL actually started when all VICE and SAFCO and permit founders created a company so they decided to do like authorization as a service for application and the right thing to do was of course based on policies code so if you want to do like something as a service the best practice is to do what you will recommend your users so they choose back then it was like almost two years ago to choose back then to use OPAL but as I mentioned, OPAL is very oriented for infrastructure level authorization and when you need to stand in a velocity of application in the speed of syncing policy configuration in fetching data from multiple endpoint in scaling your decision point everywhere that's where OPAL lacks so they decided to create like administration layer for OPAL that will help them to sync all the policies then the data is grow and they understood that they need more types of policies so specifically they were need in implementation of Google Zanzibar we haven't mentioned Google Zanzibar yet we can go back to it later but that's actually in implementation and then they need to add to the software that they worked support in another let's say policy system and then they edit and then they saw like they build a tool that can potentially help a lot of developers out there that want to move to policies code policies graph not policy that imperative in application code to manage their authorization workload and then they decided to release OPALT B in open source instead of developing what's right for permit which everyone that develop commercial software know that limit yourself for your need it's something that can cost later so they decided to open source and see the community that already deal the part of the community that already deal with the complexity of managing policies code to add support in more engines and this is actually what we see we see the community adopted we see that like more and more contribution coming in it seems it started with the notion of eating your own dog food right that I have a problem I need to solve it I'm also in your case you were not just working on your internal way of managing this policies but also solving authorization externally what's what's the story here like how it's deployed right like maybe at permit how it started and then how can people deploy OPALT in the wild is it just for cloud native environments can I deploy it if I'm running a few machines on on GCP or EC2 like how does it work yeah so OPALT has a nice server and client architecture is not aimed to be cloud native it could see it's everywhere it could actually fit to every kind of modern application the server responsibility and we already mentioned the component of the modern policy system the server responsibility is for the policy configuration so the server is like a GitOps enabled tools that can be connected to a Git configuration to policy configuration as code and make sure it always synced all the versioning is in place it also has like architecture to connect it to API bundle so the servers think about it like the control plane part that's responsible for the policy configuration and for the let's say data configuration the client is in the other hand the data plane side so it's like containers that you can scale wherever you want so you mentioned like you have few machines on GCP you can run each client as a sidecar to each machine or you can run it as a sidecar for your pods or in Kubernetes or whatever it is the client is think about it like not a real but kind of stateless containers that wrapping the policy engines and again no matter what the policy engines it is wrapping it in a abstraction layer for getting decision and with engines we call them data features that know to keep the data always sync with the decision making and it also of course connected to the server so the server know how to scale them and the server also know to sync all the policy with those clients so for now you can actually just take OPAL deploy it on your environment and you can plug the policy engines to it so for now OPAL already support OpenPolicy agent and CIDAR agent because this is actually what we see the real need for the market currently but we really looking for the community you know to add support on their one say from permit perspective that we develop the support of the engine and release them open source as we need them this is actually the reason we lately release support in CIDAR agent I was going to ask about that we did an episode in the season season 2 of the podcast with Emina from the CIDAR team we talked about how they can play open sources this was early May 2023 why did the OPAL team decide to add CIDAR support what were you seeing so that's actually a great question because there are policy engines out there and as permit we are kind of our engine, kidding you our internal engine is add support from what we believe the market will adopt we do have our internal let's say commercial implementation of the custom thing we do with OPAL when we think of something to be released as open source is because the trend we see coming the hype we see coming when tech giant like AWS come and say to developers look people you need to stop declarative policy code in your application you need to move into a declarative policy as code way this is something we feel OPAL need to support so we want to demonstrate how OPAL as an open source tool can make developers polyglot and in our perspective this is actually like a flow of users that decided to do policy as code started with let's say Rigo because it was great for infrastructure and now they are looking for more application level policy language the nature move will be to CIDAR there are like application level policy languages out there but none of them came as a result of a company of cloud provider and this is why we feel like the nature addition to OPAL as an administration layer for application level policy particularly to support in this language that makes sense it seems it's both a decision to allow flexibility and choice but also kind of like a more I would say quote kind of like strategic bet on hey like if AWS is pushing for this a few folks might become experts and we should have support in it for the open source project and it also helps exercise that abstract layer for the brand time on the language I guess right correct so when AWS we were like partners we partnered with AWS even before they release it we get kind of early access to the code and to the product and we actually also released an open source project called CIDAR agent which is like the equivalent for CIDAR but for open policy agent so CIDAR itself is actually a Rust package and when you want to do like abstract API calls to it you need to run like piece of Rust code we created an open source project called CIDAR agent that actually let developer run the abstracted engine of CIDAR and hence OPAL can use it too makes sense and it's always nice to kind of like partner when doing some of these launches to gauge how the open source community will use some of these components and figure out like how everything works let's go kind of like at a higher level who is using OPAL today and what are you seeing it being used for so that's a good question because I have names that I'm really impressed of that use OPAL I can mention like Tesla, Microsoft, NBA, Accenture Zep here like really good software organizations that using actually OPAL and one of the things that I like them to see doing with OPAL is the use cases so you see like a large organization that decided to streamline their infrastructure policy or their IT policies which is like let's say and all the domain of policy configuration and then when they try to take it to a modern environment they tackle the scaling so they started to use OPAL or for example you can see organization that decided to go with let's say like open policy agent itself and then they understood that the data cost a lot to open policy agent and they need a better way to manage the data that open policy agent used and then they started to use OPAL so the way of using OPAL is for me and the reason I'm so enjoying OPAL is that we see that we are really solving as challenges when users go to scale so the saying that OPAL is the administration layer that helps you scale policies code is something that we see almost in every organization or developer that start using it and what challenges or maybe gotchas mistakes are you seeing from some of these teams when they start to implement OPAL maybe one or two things that the listeners can say oh I should avoid these that a question that it's to the point that we spoke before like a new learning curve that need to made for policies code one of the common gotchas is the mistakes that user do regarding control plane and data plane right so for example they are mixing configuration that should came as a pure data in the policy config and then they are getting harder to deliver policy on time right so instead of configuring abstract policies and let the data on real time being processed by the policy engine they are kind of declaring the data on the policy itself and this is kind of learning curve that in general in policy as code developers need to train for I think about policy as configuration and decision as the data plane itself that need to be done I get what you're saying could you maybe provide an example so it's clear in people head especially particularly think if someone hasn't ever written a policy and kind of like fed data to it exactly so one of the things that specifically open policy agent support is actually doing stuff during the policy running let's say do a network call ok and this actually taking a time time means the decision could take longer when you do the right thing I'm not saying that you should never do a policy call as part of your decision making but sorry the RPC call while you do decision making but the point is when you manage it right you want to take this data at the right at the point of time that is relevant right so users kind of making their policy decisions take a long time and then they come and say hey we see that the decision taking like two or three second and we want to get sub ten millisecond and then by a simple rethinking of the policy itself as the more abstraction level of policy declaration right so instead of modeling the data inside the policy or trying to get this data as part of the policy they are now separating the concern between the control plane the policy configuration itself and the data itself that happened in the data plane when I get a decision they get the right time for decision I see what you mean in that case that example would mean try to think about the policy just from a data perspective regardless of its location and don't think about how to get that data into the policy later like kind of like delegate that to another component is that what you're going for? Yeah so specifically in OPAL you can configure data features right and the data feature OPAL is a like smart mechanism how to manage the data that decision makers save right so let's say for example you have like a billing system and you want to know now that the user paid for something so instead of doing it as part of the policy configuration so configure the policy to do some call you're doing it as part of your data feature so when the policy engine need to get a decision the policy itself is very abstract is that paid users or non-paid users and then the data layer this is the layer that actually knows how to manage this slice of data how to manage this connection of data right so we see this for example that again we can get to it later but let's say graph based policies so we want to get decision based on relationship between entities right and this require a lot of data to be in the policy layer themselves because we want to know their connection the relationship between entities and then people try to declare it the policy itself okay so they say like go to some endpoint check for relationship and stuff like that and then they are losing the world of getting policy fast and keeping the right data in the other end so this way of thinking that we are now declaring and configuring code while the decision itself happen in real time and need to get the right data independently of what happened in the policy configuration that a way of thinking that we see a lot of glitches it seems the things that you're mentioning I look to the fact that authorization decisions or at least teams and users want authorization decisions to be fast because an authorization decision happens whenever you're interacting with that application it's for yeah exactly it's for whatever let's say request or whatever it might be multiple per request and like it usually happens in distributed systems it seems state unmanaging state required for these authorization decisions is a big part of it right so if I'm running a policy for example and the policy takes 2 milliseconds but before I run the policy I need to go fetch data from a database and that code takes half a second then even if the policy is very fast itself the authorization decision as a whole might be slow and this is where the notion of a fetch share comes in so let's start there what problems were you seeing in the wild and how does OPAL help to manage this state so one of the things that we see is speaking about problems and I want to start from expanding this landscape of problems okay so when we configure policies code right we have like configuration that let's speak about very complex ABAC configuration we want to get a decision based on conditions on 3 or 4 attributes of the users right so let's say the account age is larger than 100 days and they live in Europe and they are for example paid user in a particular tier and so on so forth and then we also want to have a lot of decision making on the resource itself so it should be document that's stored there and belongs to there and etc so we have like a big policy to make this a big policy complex policy to run fast we need to have let's call it small chunk of data right the more data the more data fetching the more data processing the more data handling that we will need to have when we get a decision make this policy decision longer right and you can see it actually in policy engines that based on policies code let's call it like the stateless policies code when they are getting a lot of data they start to get decision slow this is how that happened and one of the things in OPAL that actually even differentiated from other abstraction layers other administration layers of policy engines is the data feature architecture the data feature architecture is first in OPAL you can plug in any kind of data feature so think about it like fetching HTTP data fetching GRPC data fetching Postgres database data fetching whatever data that you need to fetch you can write a plugin that do the right thing for you right so you mentioned before not in this context but you mentioned before like the connectivity between stuff so you can actually have like data fetcher for one client one engine that can go to some data source and in other client that can go to another data source okay so you can plug and play this configuration of data fetching for each client that first help you to make sure that your client get the real data that they need on top of that OPAL has a nice slicing and eventually consistent mechanism to make sure that only the data that the client need only the data that you configured is getting actually into the policy engines and not only that it has as I mentioned eventually consistent mechanism that the policy engine could know that now something changed and then I have policy to decide I can say I deny everything something changed I can say I can trust on the previous configuration until something changed but for sure I know that something changed now and then OPAL actually helping you in scaling the fast of this policy decisions so if in let's say I'm saying traditionally it's not traditionally right it's all a new topic but let's say if let's say in a traditionally policy agent you need to fetch the whole data because this is the way it should work OPAL is actually with the data fetch architecture let you very sophisticated way to manage the state that your data needs to get now so you're not only configure the policy itself you not only configure the way you declare your policy you also configure the way that they are consuming data and this is one of the things that OPAL itself even it is not a policy engine helping you get policy decisions much faster it seems then that there are a couple of things here the data fetch allows you to connect to different components get data before a policy actually runs and there's probably something there like how does it know what data to fetch before the authorization decision needs to be made so and maybe we can dig into that and then there's another part which is it also has a mechanism or at least maybe you have to implement mechanisms by which you can check if data has become stale or data has changed so that policies can become aware of that maybe there's a hey is this data recent or not and give kind of like an escape hatch hey if this data has changed since the last update on kind of like I guess that any memory cache or something like that then go down this route and maybe deny decisions until we have the latest how does the fetcher know what data to fetch and what data not to fetch because ultimately I guess here there are kind of like cost reasons around like minimizing networking costs and not doing kind of like unnecessary things particularly if you're going to external data sources right like if you're running on a cloud provider and you need to go fetch data from from somewhere else that might cost you a bit that are in memory like resource reasons like otherwise you're going to have your agent take up lots of resources depending on the data structures that you use your local searches might be slower how does sort of that work Yeah so that's actually actually connected in my mind to the previous question about when speaking about data there is as you've kind of mentioned baseline data data that actually our application based on let's say resource type something that almost never happened almost never changed and there is data like say real time data and Opal is actually let you separate that so you have like baseline data which is the data that you load your clients with it but there is even a level of granularity on it because you can actually load different clients with different baseline of data right so for example if you you have like a microservice architecture you can decide that different engines on different microservices has different sets of baseline data and this way you make sure that each one get the slides of the right data that he can get the right they can get the right decisions based on on top of baseline data you have real time data and here again you have this granularity you have kind of data feature and a different let's say you can use polling you can use like pushing you can use like bidirectional RPCs everything is actually configured in the data feature level in configuration of the data feature level and in the configuration of the deployment and in this way you can make sure that the right policy engines the right clients that run policy agent for a purpose for a microservice for the whole system is run only with the right data that they need at one point of time as I hear this it makes sense I also think that it seems that as a team or as a company using opal this is what I would spend a lot of my time right like fine tuning these fetches making sure that they work the right way does opal come with some set of like default fetches or like already available fetches for generic providers or maybe like very common providers and then I might have to do some things for anything that's specific to my team my company yeah so definitely opal comes with fetches that we develop ourselves like I mentioned HTTP fetches but some community members that as you mentioned spend the time to write their own fetches that contribute them back to the community so NBA for example release a feature for Cosmas DB DB that stands for making policy decision and for example we have like a community member that contribute postgres feature since this is a pluggable system and that's correct that all of our actually are let's say large opal users are actually writing a very specific data feature from the reason I mentioned right to make there is a new mention to make the policy decisions faster but some of them in that just contributed back so I can't remember now the whole data feature that we already have but the opal repository on github as a list of all the fetches and integrations that already contributed back from the community it's nice to mention here that integrations and third party support like pluggable stuff is not only on the data feature side so we have like a gith bundles that you can implement yourself we have like a way to write plugins as we mentioned for policy engine themselves so opal is actually a very pluggable system and we really see the community that contribute back the thing that important to them in this plugin system of opal that makes sense so that's one of the benefits of the open source approach and the pluggable system right it's like you get to benefit from others one thing you I do it too earlier and you mentioned a couple of titles was this notion of like relationship based or graph based authorization decisions you talked about kind of before which I worked on a system that's based on it we had a couple of episodes about the topic last season which we're going to link on the show notes can you maybe expand of where that fits in because it seems at least we're talking about opal we're talking about state this would be a good moment to explain some of that so I'm happy that you come with that question because personally for the last couple of months we had support in one you mentioned relationship based engines to opal but I want first you know like we had an explanation on RBAC and ABAC explain what is this relation based authorization and why it's kind of different from RBAC and ABAC so relationship based authorization is something that is very common in consumer applications so let's say when you are in the social network you can see all your posts you can see posts of your friends you can see posts of groups that you are a member of and it getting very complex when you have like layers so I am for example a member of organization that is a member of organization a member of organization and by my belonging to this very low level organization I want to be able to perform operation on let's say like my cousin in other organization because we are belong to the same top organization this is what called relationship based taxes control and again it's very common if you imagine like one of example that I like to bring is like Google drive right so you can share documents in a very granular way this relationship based taxes control has a very nice protocol or standard called Google Zanzibar and also nice implementation in my opinion and you also like mentioned it here one of the best implementation for application level itself and for model application it's open FGA it's actually implementation of a very good standard for relationship based access control that has the same but the exact the same but the ideas of policy engine something that getting the decision policy that configured as code but it has also something that don't have in RBAC and ABAC it has graph database that by looking on its node you can get the decision right so relationship is something that could take time to calculate so we want to keep state we want to keep data and this is the main differentiation between RBAC and ABAC and REBAC in RBAC and ABAC in one hand you don't need a lot of data the decision itself happened based on a very specific set of data but the policy can be very complex as we mentioned we could like combine 10 attributes in the user 12 attributes in the resource and get a decision on it but again we don't need a lot of data when we look on relationship based access control the policy rules in the nature are simple if a user is belong to a kind of relationship they are allowed if not they are not allowed in the other end the state that we manage the data that we manage is much higher and need to be in much scale than a policy agent that is based on RBAC and ABAC the nice thing about OPAL is because the nature of open source and understanding that at some point and I personally already seeing it now with some users at some point policies code will be polyglot developer will be polyglot they will want for example to have one set up of open FGA or having a very granular or highly granular relationship based decision but for other need of the application they will want to declare ABAC policy to stand in very complex decisions very fast and one of the things that I like with OPAL and I really like to see the hype around the community is bringing OPAL to a phase that we are actually supporting in one hand in policy engines and policy agent that are mostly stateless use very specific slices of data and allowed complex policy in one hand and in the same setup in the same policy server that actually kept everything thing and scale the policy clients support also a relationship based policy that require a lot of data require manage the state but let you get much sophisticated authorization decision based on relationship and I think that's at this ultimately how I think about this where on the one hand there is a space and a need for attribute based access control and attribute based decisions on the other hand there are cases where you need very fast authorization decisions you don't want to deal with kind of like operational overhead of running all of these components so like in having that state in the database makes sense and that's where kind of like systems like OpenFGA or other Zansibar implementations come in a couple of things for people that aren't familiar with it so Google Zansibar released a paper sorry Google released a paper called Google Zansibar which is kind of like how Google does authorization internally that kind of like Google Drive example that you mentioned is a good one and as Gabriel was saying it's not a standard it's kind of like maybe becoming sort of a community de facto standard because a few companies have a few open source projects starting to kind of like follow in that those footsteps maybe a comparison would be much like map reduce and how to happen kind of like from an open source perspective and the other thing as you were saying is it uses the notion of a graph of these relationships between users and objects to make authorization decisions but the key thing and this is one thing that they call out in the paper and also I think that's in most of the implementations that I've seen is that it doesn't actually end up using a graph database behind it because of how the authorization model DSL domain specific language works you can actually implement those graph relationships and these very fast lookups on top of ordinary SQL databases and that makes a lot of the kind of like operation and overhead a lot less like graph databases I wouldn't say are new kind of like they were a thing in the 80s now they're back but their operational properties are very different from things that maybe if you're used to writing a closed layer sort of my SQL or like a cloud database like Dynamo or Cosmos etc and that's a big thing right you don't have to run a database that you're not familiar with in order to run some of these open source projects. Yeah exactly this is the way we're seeing it and that's truly exciting for us and we are truly eager to see how the community will adopt this idea of managing the polyglot policy setups A couple of closing thoughts I want to go back to how this all started right with Permit. Can you explain what Permit.io does and like why you folks ended up creating Opal for those that aren't familiar with Permit? Yeah so Permit is actually wrapping all the good things together but also let you everything so you mention all the operational costs right so at the end I have to deploy the services I have to maybe develop like audit and monitoring and scaling and maybe UI to edit all the policies and whatever it is Permit just bring it all in a nice cloud offer that user can subscribe and one of the things that differentiate Permit from let's say other cloud products that offer authorization of the service is the ability to scale so you can take Permit, connect it into your Git repository where you already store your policies good and let Permit scale everything for you plus get all the external services like auditing, monitoring user management data fetching, maintenance etc etc beside of being like a let's call it standard cloud service for authorization application it also support all the operational headache that you could have from maintaining authorization system yourself as a cloud service. What I take away from that is if you want to use Opal but not have to run it Permit is kind of like Opal on steroids as a service and I guess you can deploy it to your own cloud not just kind of like the Permit cloud yeah so Opal is as we said has a lot of scalability options right so we have like kind of Opal class offer which is you know the more traditional commercial open source Permit itself is more for as you say save you all the effort of scaling deploying everything and get everything around it. Yeah that makes sense really it's been a great conversation we went deep into a number of topics hopefully this gives listeners the opportunity to learn about some of the things that are happening in the space and trigger thoughts and kind of like help them figure out some things with like hey I should be using this I should be checking that out and so on it's been great to have you on the show man I hope you enjoyed it that was really great and I was happy to be here and again I'm pretty sure you can show links you can find me on Twitter LinkedIn I always really love to talk about policies code and create better access control experiences for application. Definitely yeah we will add those links your social links some of the Permit links to show notes we will also share links to everything that we've been discussing today to show notes so that people can easily find those there it's amazing to have you and again to everyone listening in if you folks have questions about Permit or Opal itself just feel free to kind of like ask Gabriel or go to some of those links that's it for today's episode of Authorization and Software thanks for tuning in and listening to us if you enjoyed the show be sure to subscribe to the podcast on your preferred platform so you never miss an episode and if you have any feedback or suggestions for future episodes feel free to reach out to us on social media we love hearing from our listeners keep building secure software and we'll catch you on the next episode of Authorization and Software