 Hello, everyone. Welcome to Cloud Native Live, where we dive into the code behind Cloud Native. I'm Annie Telvesto, and I'm a CNCF ambassador as well as leading marketing at Vision. And I will be your host tonight. So every week we bring a new set of percentage to showcase how to work with Cloud Native Technologies. They will build things, they will break things, and they will answer all of your questions. You can join us every Wednesday to watch live. So this week we have more here with us to talk about abstracting communities and setting standards with internal developer portals. Very excited for this. And as always, this is an official live stream of the CNCF and as such it is subject to the CNCF code of conduct. Please do not add anything to the chat or questions that would be in violation of that code of conduct. So basically please be respectful of all of your fellow participants as well as presenters. With that done, I'll hand it over to Mor to kick off today's presentation. Yeah. Thank you so much for that warm introduction. So hi everyone. My name is Mor. I'm a lead architect at Port. And today we'll be talking about abstracting Kubernetes and setting standards with internal developer portals. We're going to discuss developer portals in general, Kubernetes and how they connect together and help you and your developers with the developer portals and their internal developer platforms. So let's get started. So first of all, a bit about me. I'm a software engineer previously also hosted a CNCF webinar about introduction to developer experience, which is a topic highly related to developer portals and internal developer platforms. I love video games, bouldering and burgers. People from around the office always ask me about the best place to eat around the office. I always have a cheeky answer. If you're ever in the Tel Aviv area, feel free to hit me up. I'll definitely have some recommendation for you. So first of all, what are developer portals and internal developer platforms? So developer portals are meant to be a layer making it easier for developers and platform teams to coordinate and getting access to the underlying infrastructure that their applications and services run on. As you would probably know with the world of cloud native and the growing complexity of a microservice architecture and solutions such as Kubernetes, the infrastructure space has grown much more complex and making it much more difficult for developers to understand where their apps are running, what is deployed, where is their service running correctly, should they make some adjustment or change. And the idea of developer portals is to actually simplify all of that complexity and give them that one stop shop, that one place they can access to actually gain a good layer of visibility into what's happening and also who is the owner of a specific service, who is the current on-call and other relevant information, as well as gaining a layer of self-service to gain more independence and rely less on mails or Slack messages or just finding the closest DevOps engineer in the hallway and really improve delivery and making their work more convenient and easier and letting them focus on actually shipping code and quality code without really diving into the bare middle of the Kubernetes that's running their applications. So now that we know a bit about developer portals, let's talk about the core pillars and what should be inside every good developer portal for your organization and for your R&D team. So, first of all, we have the software catalog and the software catalog is meant to store all of the information to create that nice visibility layer. So, we're talking what microservices do we have, where are they running, on which cluster, which workflows, in which cloud environment and cloud provider or cluster, and so on and so forth to really give you that visibility layer that gives you, first of all, a bird's-eye view about everything that's going on. But in addition to that, the ability to dive in and really find that one piece that you're missing, maybe that documentation to the API you're consuming or maybe the information about the current performance of your cluster. In addition to that, we have the self-service layer. A lot of times developers have these rather repetitive tasks that they do not have the permissions or the deep dive knowledge to execute themselves and these actions usually require them to open a ticket or send a message or an email to DevOps engineer to ask him, hey, I need this new permission. I need this new cloud resource. Can you help me with that? And usually the DevOps already has a script to perform that action but he doesn't want to fully expose that script to the developer because he fears it might be too complex or it might break something if used incorrectly and the idea of the self-service layer is to actually expose those actions in a safe way and pave that actual golden path for developers to gain more independence when interacting with the infrastructure. Another layer we have is the workflow automation which actually lets machines, for example, our CI CD, our Argo CD and other platforms take advantage of all of the information and abilities that we have in the software catalog pillar and the self-service pillar of the developer portal to actually make decisions. So, for example, our software catalog might say that a certain service is currently locked for deployments and if someone by mistake tries to make a new deployment for that service the workflow automation layer will check that info, see, hey, this service is currently locked you can deploy it, maybe it's the holidays and you don't want to make any sudden changes to production but might break something maybe we're at a code freeze before a new release or something along those lines so this is another way that developer portals make it easier and more controlled to deliver quality software. And another pillar is the scorecards. Scorecards lets you set standards for how quality code and quality services in your infrastructure and your environment look. So, for example, you can say, hey, from my maturity standard standpoint a certain microservice is only mature if it has an X number of replicas or higher so that is a way to show the level of resilience that is required or maybe you want to enforce certain Kubernetes cluster versions and it is really up to you how to define those scorecards how does your organization look at the standards and best practices and actually take advantage of this information to drive initiatives and say to your developers and R&D teams hey, this service does not meet our required maturity levels let's push to make it better to improve the certain points that are not as good as they can be to make sure that our services are bulletproof. There was actually a quick question from the audience so there was someone who was maybe looking at how does this all place into the larger ecosystem because they're asking to kind of see would this be a Kuberna alternative or a Conveyor IOL alternative or kind of thinking about how do these pieces fit together with developer portals. Yeah, so that is a great question. The idea of developer portals is to actually integrate with all of these tools so we are not a Kuberna alternative we're actually act as an enhancement you can use the Kuberna policies to report back to the software catalog and actually see if you uphold all of the policies configured by the organization maybe your Kubernetes cluster standards are not up to par and you can view that via scorecards that are reported by Kuberna for example so the developer portal really interplays with what you already have in your infrastructure and it's meant to take that info in and give developers and other members of the R&D team to actually have that one stop shop where they can view okay these are the policies but I'm not following right now maybe I should fix it in this way maybe I should aim to solve it in X timeframe Great, and then someone was checking the same for Backstage as well So first of all Backstage is obviously a CNCF project and Backstage is the most notorious developer portal and Backstage actually does implement these core pillars so for example your software catalog layer, your self service and I know that they are currently working on plugins for scorecards and things such as role-based access control so what you see here are the pillars that comprise a good developer portal and Backstage is also aiming to do that Yeah, perfect and then someone also wants to see if the presentation will be available somewhere like the slides and whatnot Yeah, so I will share them with you later and I believe you can send them to the people who sign up Yeah, they will be sent and also as always this whole webinar live stream will be available as a recording in the CNCF YouTube so you can rewatch it as many times as you want from there and it's going to be uploaded essentially immediately after this to the CNCF YouTube live stream section so you can head over there after this to find out all those information that gets from there but for now there's no more questions so we can continue Awesome So as I was saying the developer portal also includes usually a strong role-based access control there controlling exactly who can see which information because one of our main goals with a good developer portal is to reduce the cognitive load which means we don't want to show a developer all of the information that we have from all of the infrastructure this will just overload him and make him confused what should I focus on, what is relevant to me so the idea is to first of all hide information that he doesn't really care about and won't help him in his day-to-day and also make sure that for example he can't perform any dangerous self-service actions so for example deleting a service or rolling back a service might not be something that you want every developer to have the option to do or maybe you want an additional manual approval step when performing that action so these are the kind of things that role-based access control is for and of course with all of this great information inside your developer portal you want to use it and leverage it to generate those R&D insights and reports how are my services doing, how are my environments doing, what needs some assistance what needs the eyes of the current on-call maybe a self-service action to alert someone and things like that and the idea for developer portals is to be consumed in a very convenient way so that could be a UI or an app or a web UI or some API for those workflow automations and self-service actions and maybe even a chat-ops bot allowing you to simply send a message to the portal from your Slack to gain quick insight into some service that you own or you want to gain more knowledge into okay now let's talk about how developer portals interplay with Kubernetes and why you should try to abstract away some of the Kubernetes information inside the developer portal so what we can see here are some examples for visualizations and representations of Kubernetes information in the way they are available today so for example we can see here an application deployed via Argo CD and my guess is that if you're going to go to a developer that is not super versed in Kubernetes and Argo CD and show him this diagram it will say okay I can sort of understand what's happening here but this is way too much of a drill down this is too complex for me I don't really need all of this information and I probably would like to consume it in an easier way so this is exactly the case for abstracting away Kubernetes in your developer portal and the idea is that you will take all of the information that Kubernetes gives you there's a huge host of information there that could be very valuable and very effective if exposed correctly and actually map it to different objects that you represent in your developer portal so for example you could represent a microservice and say I care about the deployment name and that is actually the image name or I want to take from the deployment labels the runtime of the environment or maybe I want to look at my running service and actually see how much CPU is it using, how much memory what is the general service health and so on and the idea is to actually make use of all of the great information that you have in your Kubernetes and it's very easy to consume via the Kubernetes API and show it to your developers in a convenient way that makes it easier for them to understand what is actually happening what should I actually focus on and care about now every good developer portal has a data model and what I'm going to show today is a data model revolving specifically about Kubernetes because this is the topic of this webinar so what I'm going to focus on is these components of the data model and an important thing to note about data models in developer portals is that they are always very specific to an organization every organization is unique, every organization cares about different things but developers have different levels of understanding of how Kubernetes works or interaction with it so the idea is to actually map the data model in a way that fits your organization but in this case we're going to talk about clusters and nodes, namespaces, workloads which is a generic name that we give for deployments and stateful sets and daemon sets and so on and so forth and pods and when we explore these components of the data model further then we can see the actual information that we care about so for example a cluster is just a parent entity that we want to use to actually consolidate together all of the information but when we're looking at a namespace we might care about when it was created and its labels when we look at a node we might care about its readiness and its label and its kubelet version and the total CPU that it has and again the idea with a data model is that it should fit your organization so the idea is to actually go and talk to your developers and talk to your DevOps and talk to your platform teams and say hey we want to bring the data from our Kubernetes into the developer portal what should we care about, what should we focus on what will make lives for the R&D team easier and like I said some more things but we're going to focus on the workloads so the containers, the strategy config what are the limits and containers and so on and so forth and the idea is to customize it really to your needs okay now that we understand what is the data model that we want to show and why should we even want to extract some of the Kubernetes information let's talk about how we actually do that so the idea is to use an ETL and JQ to extract the correct information, the information that we care about from Kubernetes now you will see here at the bottom some links those links are sent to you in the chat so you can look at those and we'll also explore those in a second now I want to show you exactly how this idea of simplification works so first of all what you see here is an example of a config that we use in Ports Kubernetes Explorer Ports Kubernetes Explorer is open source it's meant to be installed on your Kubernetes cluster and with a simple config you can say I care about these resources from my Kubernetes cluster I care about this information from every piece of my different Kubernetes resources so I can extract the information that I care about so for example here you can see that we take from the replica sets kind or object from the Kubernetes API and we can query by specific fields so for example I only care about replica sets that are not from the cube default namespace so I only care about replica sets from my namespaces and then we can actually go ahead and say well I want to map this information from my resource and that way you can select exactly what information you want to take from each resource and using the power of JQ you can actually transform those different fields and really tailor it to your use case so just to get everyone the same page I want to show you how it looks when we're talking about JQ so for that I prepared some examples so JQ is a JSON processing syntax and what it lets you do is actually take existing JSON which matches how you can output information from the Kubernetes API and transform it in different ways so for example you can see here that I have this simple JSON object and using simple JQ syntax I can concatenate the strings and create something entirely new and of course it will also work for numbers and arrays and it is a very very mature syntax that can do pretty much whatever you need so it makes it very comfortable to extract information from the Kubernetes API and when joined with the port Kubernetes exporter it makes it very easy to take the information from the Kubernetes API and send it to your developer portal now another example that I'll show around JQ is actually even more specific to Kubernetes so what you can see here is one of my deployments and you can see here the different information that I have and I wanted to create some sort of unique name for this specific deployment so what I did I just took the metadata name key I added the deployment prefix and then I added the metadata namespace and this created this output string which allows me to gain a new string that can take information from other areas of the cluster so for example I can take the namespace or the UAD or even look at the spec or strategy and really leverage the entire information that exists in the Kubernetes API along with JQ making it very easy to extract the exact pieces of information it can even do array manipulation or constructing to really tailor that information that you extract from your Kubernetes and one last thing I forgot to show is the port Kubernetes exporter repository so this repository is a fully open source and you can check it out we also have a helm chart to deploy it but it's very easy to get inside it's written in Go you can modify it however you want the JQ and Kubernetes functionality is already built in so you can just configure the source or the target that you want to send the information to but a lot of the heavy lifting is already done it can listen to any kind or CRD that you have inside your Kubernetes cluster and you can leverage that very easily to take the exact information that you care about now what I want to show you now is exactly how this looks I'm going to switch screens for a second and while we're waiting for the screen to pop up the audience if you have any questions you can send them the chat right away so we'll get to them right away but we'll likely have some time in the end for Q&A as well so stay tuned for that as well OK so what I want to show you here is that data model that we spoke about so in here you can see the data model of the iModel which is very similar to what I've shown you in the slides and I want to actually show you how this information looks once you take that and display it inside your developer portal so for example if we go to the clusters we can see hey this is my Kubernetes cluster and this is actually a Kubernetes cluster running on my local machine and if we look for example at the node we can see the different labels which I extracted using the Kubernetes Explorer and JQ and if I dive deeper by actually using the Kubernetes Explorer and mapping those different connections and actually making those connections between the different pieces of the Kubernetes cluster because everything happens using the Kubernetes Explorer and actually taking all that information in at once and we can actually gain a very high visibility view of everything that's happening in the cluster and we can even map it on a graph and look at all of the different connections that we have now let's say I want to add more information so there is absolutely no problem the Kubernetes Explorer is always querying the cluster in real time so what I'm going to do now is and you can't see it because it's on my terminal so let me just share that oh and while we wait for the terminal to pop up there's an audience question so Sebastian asks will you present about abstracting a multi-platform landscape above cluster level? so I will not be touching that specifically in this webinar but the idea of the Kubernetes Explorer is that it's not a unique to one cluster so you can use it to deploy it on multiple clusters that you have and map all of these together to your developer portal so even if you have tens or hundreds of Kubernetes clusters running thousands of workflows or workloads you can always map those to that one single developer portal and actually exposing all of that information in one place and actually exposing it to the correct teams to avoid that overload and make sure you have that one place that consolidates all of the necessary information perfect so what I'm going to do now is I'm going to add another namespace I'm just zooming so we can see it a bit better so I'm going to create another namespace we could zoom in a few more times I think it's going like small oh now it's looking better so I'm going to add another namespace and I'm also going to add another service to my cluster so we can see we have a new namespace created and we have a new service and deployment in that new namespace and now when I go back to my developer portal since the Kubernetes Explorer is always querying in real time then we will see we now have a new namespace and also if I go to the services page then I also see a new service and if I dive into that since my Kubernetes Explorer already knows how to extract information from services and pods and deployments and namespaces then all of the information is already mapped in automatically and it knows how to take that information in so even if my service just now was deployed and it's brand new then I can already see it appear on my cluster and have all of the relevant information inside Perfect, and then there's a question from the audience again, Sebastian, thank you so much for asking questions so they ask again a VW group slash audio specific query probably can I onboard at scale developers and route them to different platforms then maybe according to internal criteria automatically Yeah, so the idea of developer portal in the end is to be that place for all of your developers and maybe not only them because since you ingest so much quality information that's arriving directly from your systems then maybe you also want to expose it to your customer support team or the product managers for example to see how progress is going with certain projects for example so the idea is to really bring in as much information as much quality information as possible and using the role-based access control pillar to actually expose the correct information to the correct personas and the idea is to not create different platforms for specific teams rather you want to create that one platform and make sure that the correct personas see the correct information but even if they need that extra layer of visibility to dive a bit deeper then they have the option to request those permissions even if it's maybe just temporary or maybe give them that tailored view but gives them the exact information that they require for most of their work sounds good and if anyone wants to ask more questions everyone's always obviously welcome to elaborate on more specific questions in the chat as well oh and there's another question or should we take that one or do we want to continue a bit more further in the presentation no for now let's continue and we'll have some more time for questions moving forward okay so we're going to take your question James in a while Norris okay so in here we actually looked at mostly the software catalog layer because the idea is to use the Port Kubernetes Explorer to take the information from your Kubernetes and put it into the developer portal in a way that is easy to consume and really tailored to what your personas and your developers require and need but as we said the developer portal and developer experience journey does not stop there and the next steps are implementing scorecards which will let you look at specific Kubernetes readiness metrics and things that you care about so for example if you look at a sample scorecard for production readiness we might say okay I say that a bronze tier is if the kubelet version is X it's silver if it's Y for example or we can look at some configured checks so for example we don't want kubelet containers we must have some CPU and memory limits we don't want anything production being deployed in the default namespace and the idea of scorecards to actually configure hey this is how a super production ready up to the highest standard cluster looks and now that we have all the information inside the developer portal let's examine it and see are we up to those standards are we absolutely bulletproof and ready and if not what can we do to improve that and as we said we want to give our developers a high degree of independence so now that we have all the Kubernetes information inside the developer portal we want to give them the option to actually take advantage of that to make actionable decisions so for example I know that the action shown here is the restart cluster which is maybe a bit extreme but of course we can take that action and say this could as easily have been change the replica count or rollback a service or change the deployed image tag and the idea is to actually give your developers the ability to trigger those actions themselves because we know that usually this is the job of the DevOps and the platform team and they usually do it using maybe infrastructure's code or Argo CD or some GitOps way and we want to actually expose this functionality to the developers to make sure that they have they don't need to search for their DevOps with mails or finding them in the hallway to actually invoke those actions and that way everyone improves because the DevOps does not need to be bogged down with mundane requests and the developer doesn't need to wait for someone to simply click a button for him and everyone can move along much more quickly and deliver software faster and that's it I hope this helped you learn a bit more about developer portals and developer platforms and how you can take the information from Kubernetes and expose it in a much more convenient to consume way for your developers that might not be as well versed and it's time for questions Perfect, there's a few questions already in so James asked a question that he saw already earlier so as Kubernetes features are released what is the time lag generally for this to be supported in the port applicable? Yeah, so the Kubernetes Explorer is actually perfectly tied into the running Kubernetes cluster so basically every new type or kind or CRD that you have in the cluster is always queryable by the Explorer so even if you just upgraded to a new Kubernetes version or maybe you installed a new service or for example if you didn't have Argo CD before but you just installed it then the Kubernetes Explorer can already take advantage of that and take that information from those new Argo CD CRDs for example and without any new work needed to be done Great, and then Sebastian actually provided a bit more extra info and said thank you so much and they apparently have 30 plus platforms already so they continued our requirements would not circle around the one platform to rule them all idea rather than a central onboarding tool to distribute the demo teams to the right platform in best case mostly automatic and auto commissioned all the best guys Yeah, so okay so in that case the developer portal is still a great tool to use because you can consolidate the onboarding information into your developer portal and immediately say hey I have John from the backend team just onboarding into the company let's send him a link to the developer portal and let him go through the backend onboarding flow maybe you can store some links for the relevant repos or documentation or those actual internal platforms that he should keep his eyes on as he's working inside the company so the developer portal can also be that gateway to the different information that you want to expose and use in the onboarding flow in your organization Yeah, sounds good and then James actually continued a bit by saying you mentioned support for our Go CD for GitOps is Flux CD and Flagger also supported? Yeah, so the Kubernetes exporter will support any CRD but it's available on your cluster so if you install it had it exposed new CRDs into the cluster then the exporter can definitely take advantage of those Great and then Irving asks is there any documentation available for them to kind of dive deeper into? Yeah, for sure so first of all the repository here contains all of the code which you can dive into as well as a link to the documentation and the home chart which you can view to learn a bit more about how to deploy and how to customize it to your needs Perfect that should be a good starting point for everyone while on the topic question from my side essentially is there any good learn more resources that people should dive more into after this? Yeah, for sure so first of all there is our demo which I've written down here so this is really a way for you to get inspired and see how far a developer portal can go when you make it fully fledged and really focus and double down on getting that information into the platform and creating that data model that fits you and see how you can make all the connections between the different areas inside your platform and also Port's documentation includes a lot of introductory videos into the world of developer portals and platforms and different software pillars so I definitely recommend you go ahead and check those videos out you'll have a familiar face there because these videos also include recordings of me and I really take the time to take it from the basics so the portal will give you why should you want it what should you put inside it and how you should expose it to your different teams of developers and DevOps and platform Great so far there's no new audience questions but if anyone is typing away send them in now oh they're asking GitHub's link please I think we shared it already there we go I'm going to highlight it again there we go there's a link to the GitHub repo there you can get all of the four mentioned create content and see more of Morse face there as well good so what we see if anyone else is typing in any final questions a question from my side so developer portals are obviously kind of gaining a lot of attraction and being a very hot topic where do you see from your side the space moving into the future anything kind of that you would think would need to be added to the discussion or so forth yeah so again this is a really hot topic right now and a lot of companies are starting to realize hey this is something that we want this is something that we need we're starting to have a lot of complexity in our infrastructure and we want to simplify that and create that one stop shop to make things easier and for the other personas that they interact with and what I believe we'll see is that more and more companies start to implement or or develop either in-house developer portals or maybe taking advantage of a product such as Backstage which is the CMCF project or maybe even port our product to leverage that developer portal and integrate their tools into that and adding more layers of integration creating better visibility and really doubling down on that concept of reducing the cognitive load making it easier to consume the information that is correct for that specific persona and really making it easier to ship a higher quality code faster perfect so then there was also someone asking is there a recording of this event available yes there is so immediately pretty much after the stream you'll find it this recording from for example CMCF YouTube under the live stream section so you'll see it there so you can always kind of tune in there to listen in if you forgot or missed some part and then there's a question from Nimesh Apologies if this was already asked but can the port UI be self-hosted? so port UI is available as a software as a service offering but the port Kubernetes Explorer which I've shown here before and I've also linked the GitHub repo is open source and you can take it and customize it and install it on your Kubernetes cluster and integrate with your self-hosted Kubernetes and developer portals if you would like that great and if anyone's typing away last call for any questions but more is there anything kind of final final things do you want to say from your side? so first of all thank you all for tuning into the webinar I hope this helped you gain some insight into developer portals and how they interplay with Kubernetes I really suggest looking more into the topic because it's a very hot topic and a lot of companies are starting to see the need for a developer portal and it's definitely a good idea to be ahead of the curve and start looking into the different available resources in the space and obviously if you want to chat then feel free maybe I'll send my email here and if anyone so feel free to send me a message if you have more questions or want to learn a bit more also my Twitter appears in the slide so feel free to contact me there as well and I hope you enjoy it yeah perfect I have it here highlighted and I think Twitter will be a great or other social media will be great resources as well for everyone to reach out and there was already some tags in the audience so that's great to hear but yeah let's start wrapping up then there we have the about me section that's good so thank you everyone for joining in the latest episode of cloud native live it was great to have a session about abstracting Kubernetes and setting standards with internal developer portals really also loved the audience questions as well as interaction today always loved to see that and as always we renew the latest cloud code every Wednesday and in the coming weeks we have more great sessions coming up so tune in for those thank you for joining us today and see you all next week