 Hi Hello, I Think that we can start maybe no anyone from the organization. Can we start? Problems. Okay. No problem Living on the edge. Okay. First of all, thanks to everybody for joining us today Just for herning us talking about Keda. So Let's introduce ourselves. I'm Jorge Turrado. I work in the CRM little plus part of SBARS group as Principal SRE and I'm one of the maintainers of Keda and CNCF ambassador and I'm also Microsoft MVP and He is my assistant. No, I'm joking Everybody, it's a pleasure to be here with you all and with my dear friend and boss Jorge My name is Binyak robotic. I know the name is hard to pronounce So don't feel about it. Don't feel bad about this kind of stuff I'm also Keda maintainer. I'm with the project since the beginning So basically we started couple years ago. It was a POC. It was a POC to try to solve auto-scaling problem It's it was a good solution So we donated the project to CNCF as a sandbox project and after couple years Keda is a graduated project. So this is really great And I'm also founder and CTO at Kedify. It's a company built around Keda We try to provide enterprise auto-scaling platform for our customers. So to to solve the special needs for for enterprises Anyway, let's talk about Keda. So we have a lot of stuff to cover today. So I will try to be quick First we will do some introduction to Keda So maybe there are some people that knows Keda Maybe there are people that don't know what Keda is So we will try to do very quick intro then we will talk about some cool features and about the future So let me start actually I will try to do Keda in five minutes, which is kind of challenging. But anyway, let's start So what is Keda? What is the problem that we are trying to solve? Imagine that we have an application and the application is doing some some important tasks an important job and This application for example is consuming data from some external source. In this case, it could be Rabid MQ and Because the you know the data they are not stable, you know, it's not a constant flow of data So sometimes we need to you know process more data. Sometimes less data. Maybe sometimes there are no data at all So what we can do we can plug autoscaling in the solution. So we have the dynamicity in the application, right? So the first nice approach would be to plug HPA the building Kubernetes autoscaler It is a great tool, but it has some limitations and especially in this in this specific use case because what HPA does it monitors the CPU and memory consumption on the target workload and In certain scenarios, especially in event-driven applications sometimes the Resource consumption on the target application doesn't Correlate with the actual need because we need to scale the application based on the data that are happening outside of the system So for example in this case based on the based on the Rabid MQ queue length So HPA is not good fit for this. Luckily, we have Keda and Keda is doing exactly this kind of stuff. So It is it is scraping metrics from the external sources or Consuming like the custom metrics custom metrics from those sources and based on those metrics It does the decision to scale up the application So it is very easy easy to use and it also solved the problem with scaling to zero because HPA cannot scale to zero But with Keda we can also scale to zero our application the important Point is that Keda does not manipulate the data in the Rabid MQ. So it is secure. We just monitors the metrics about the stuff So as I mentioned, Keda is an event-driven autoscaler. We are trying to make it as simple as possible We have six deploy plus different Connectors to services AWS Prometheus Kafka, whatever and you can scale your workloads or you can schedule wrong-running jobs based on those metrics We have a big community a lot of users a lot of stuff There is a QR code for Keda user service. So if you are Keda user, please use this one We will share it also at the end of the presentation Just a quick check on the on the architecture of Keda So we are trying to build Keda on top of Kubernetes to use as much possible. So we don't reinvent the wheel So basically Keda Operates on the operator pattern. I would say so there is an operator that manages some resources is a skill job and skilled object resource where we define the scaling metadata Then it provides those metrics That it scrapes from those external services to HPA that does the scaling from one to and replicas And Keda itself manages scaling from zero to one. So it is simple stuff. No rocket science There are other components, but we don't need to cover them So when I was talking about the resources, there are two main resources. The first main resource is scaled object With this scale object, you can you can auto scale your deployments state faucet or any custom resource that expose Scale sub resource so you can scale out is this workload based on based on different scalars The scaled object has three main sections I would say the first section is scale target where you reference the application that you would like to scale the second part The middle section is very specifies several configuration options such as minimum maximum, etc And the last section is trigger section where you can specify those connectors to external services So there could be multiple triggers pointing to different rabid mqqs or prometheus prometheus is whatever the second resource is called scale job and It's very similar to scaled object instead of the reference to the workload We can put directly kubernetes jobs kubernetes job Specification there and also we have like the configuration section and the trigger section Why this is useful? This is useful especially for wrong-running Executions and processes because imagine you have some application that is doing some long-running tasks Maybe some I know some LM models these days or something like that that the better computation might take Hours or even days so if you use HPA or scaled object to actually scale out the application with this computation The HPA might decide after some time to you know to scale in again and remove the replica because it thinks that it It is about to shut down the application with scale jobs you can basically schedule this kubernetes jobs based on those external events and Job will do the staff and finish when it need to finish. So this is the resource and Yeah, so that was good in five minutes. Was it five minutes? Yeah More or less. Okay, maybe you can take a breath drink up a couple of water I can introduce the use case for the next feature if you agree and then the stage it will be yours again Okay, please pass the the slide Have you seen he's my assistant Okay scaling modifiers Obviously, he is the expert. That's why he is the CTO not me It's true the truth on the table But I'm not gonna explain the how it works is his job I want to explain the use case of this because during the years one thing that we have faced it with if Has been user saying, okay, okay, that's nice, but how can I do some aggregated the staff because Under the hood the HPA controller applies a max between all the metrics, which is enough for 100% of the population, but it's not enough for a real scenario So that's why we have introduced these awesome feature. So the state is yours. Thank you You can take a seat. Oh, no, I just want so basically as her convention HPA is doing the decision based on the maximum So imagine that you have multiple triggers specified in the scaled objects. So for example multiple rabbit MQ triggers So what HPA is doing under the hood? It basically collects the metrics for each of the of the source Calculate the desired replica count and selects the maximum value So this is useful maybe for a lot of people but for those done that wants to more logic We introduce this cool feature and it is not just you can select average or some or whatever You can specify Mathematica formula there so you can do a lot of a lot of cool stuff a lot of magic with the with the metrics What is the use case there are a lot of use cases this is actual use case from from a user when we introduced the feature and he was asking Independently this question. So we have the requirement to be always over provisioned by free ports Our pot can process free messages at time So I was like, okay, this is great great stuff for scaling modifiers because what you can do You can basically say the the trigger value, which is under the trigger variable plus nine why nine because free ports And free messages is it stuff. So now get our reports Always this amount of replicas X extra for the for the stuff of the scaling modifier. So Yeah, for sure for sure the hard work. Yeah, now is the deal. I'm for talking you are for the work That's it, please Yeah, because the name is important. Yeah You could you can say why Lannister because we don't lie. We always pay we always pay How much do you pay me as a assistant should I send a voice or whatever? We don't see anything. I will guide you from here Okay, please My annual review will be terrible. I guess. Okay, if you see there, we have to cron cron cron triggers They are just a do me triggers for generating one metric request one metric or one as the metric resource So if we deploy that the scaler that the scale object, which basically as I said is a simple Scale object and we go through the cluster Although both a scale a box is a scalars in the scale object requires one metric The sadly true is that max is applied. So we have one although one plus one could be two Yeah, I'm not mathematical. So I'm mathematicians. So maybe I'm wrong. You're right Appreciate it Now let's gonna comment. This is more complicated. You should try to mirror the screen. It will be probably faster Don't try to invent It could fail More dramatically if you check we have just give a name in the line 53 for one trigger scaler underscore a the other one has a scalar underscore b trust me trust me and Then on top we have a new section called scaling modify scaling modifiers modifiers with the formula section When where we can just use the scaler as a metric sources We think a formula. So in that case we are saying take the value from the trigger a take the value from the trigger b Zoom them and then multiply for two. So if we have one plus one for two We should see for instances when I apply This is scale object. So go ahead finger crossed and now oh I have noticed that I didn't show lens before right? Yes Nice my annual review will be terrible. Oh my god Now you can see that it has passed from one to four Why because although I had two triggers requesting one instance per trigger HPA controller just execute a max one Max one each one now. I'm taking one plus one for two my formula So this brings a lot of power to us because we can use the all the scalars that Kedda provides as metric sources for our own custom formula, which is nice because No, you you cannot need a super advanced scenario for this if you are using sqs pops up from a Google service bus some System of cues like that and you have three three three cues Maybe you can you can scale based on this Sunday average or the sumo of all of them instead of the match of all of them now You can do it without the requirement of greeting your own code Have you rest? Yes, I'm ready. I So we saw the demo it was nice. Thank you for her and Now about at another hub days and features so last couple of months we spent improving the Monitoring stack that we have so right now we expose couple of prometheus metrics about the the stuff that's happening under the hood because you know Auto-scaling is one part but Having the knowledge about the system is also important So you can react errors you can modify or set things based on the values, etc So so we expose bunch of different metrics from prometheus about about errors about how about the actual value from the matrix, etc Also, we expose the very similar metrics through open telemetry so you can use your open telemetry collector to to digest the same same stuff from open telemetry and Last last thing the new addition is cloud events cloud events are cool There are more more. Let's say event events. So basically we are adding these capabilities to emit events on specific Occasions that are happening in the in Kedas. So for example scaling has started. We will emit that event So then you can plug this into your system and maybe do some do some additional stuff around around this area We started with just couple of couple of things and we are improving this area a lot So the prometheus and open telemetry is already stable, but the cloud events are the the new stuff that's coming. Okay, please More work. Yeah more work for you Less for me. I have to earn my salary. It's true Okay, okay, okay, okay, so We can see the scale object that I have you deployed. Let me share my screen because otherwise it could be too much complicated Excuse me. I have to apologize system displays Where is it? Maybe computers, right? horrible or main display Extended display mirror mirror mirror The third option the first option. No, no, no, no the third option. You don't see the same Mirror. Yes. Nice. Yes. Yeah. Great job. Thanks. I Appreciate it your help. Please clap Thanks. I appreciate it. It takes the knowledge to be CTO, right? So yeah Yeah, you have the experience. It's true. I Trust in you. Okay. Once again, I have my scale object blah, blah, blah, two instances blah, blah, blah, blah Okay, and now we have the metrics as spinning at all we have our own Metrics exposed by prometheus or by Just open telemetry for this quite demo. I just have deployed a Single let me show you Where is it where is it? In manager here we go. Here we go. Do you need help again? I Think that I can do it. I'm not totally sure but I will do my best trust me too Can't you see the screen? Maybe it's too small. Okay. Don't worry. He's asking because we are here for talking about Kedah Not Prometheus operator. So don't worry. He's just a service monitor for Prometheus Where you can see that when I go through the metrics of the Prometheus metrics in Kedah I am appending them extra tack sources Prometheus and I do the same when I go through open telemetry But adding the collector just for having different levels checking Prometheus because I'm gonna use Prometheus, of course If we check them here for instance, oh, it's terrible. It's terrible Come on plus plus plus plus nice Yep Source source source source source. Where is it here? We can see that we are just sending the metrics through Prometheus. So all us through through collector. So Why or what are we earning with this? We don't need Prometheus. There are Plenty of use case where you can say I don't need Prometheus anymore I'm scrapping all the metrics or I'm sending them directly from open telemetry And we are we really try to make the scaling that simple not having to take into account Okay for the scale for deploying Kedah. I require Prometheus now with something that probably you are using but we want to not enforce it and apart from that I have my plan B and Of course if we go through the collector and we check the logs Command plus plus plus I Learned quite fast. Good. Yeah, you can say that Kedah is sending the metrics. So the metrics are there It's nice because we have all the information and we can just really repair resend it wherever we want Rafa and another Prometheus to Azure cloud watch gcp wherever you want So I think that is a good improvement on the last but not least the cloud events Obviously, I have created a huge and terrible mock for it And don't don't punish me. I'm not developer. So I hope that you can Apologize me or or say accept my apologize. Sorry, but in this case We have the events Scale object is ready for scaling. So thanks to this we can just send wherever we want Currently only by HTTP, but we are working on other relays We can send all the events that they are happening in your cluster to a centralized way Why this is useful because maybe you have 20 Kedah installation yesterday If I remember an awesome girl talk about their awesome use case. Thanks Appreciate it and they have more than one cluster. So instead of having to collect the metrics and so on they can just So the use case they are not using I'm not selling a feature that they didn't present But it's just an idea you can manage multiple cluster from a single place just through cloud events I'm okay. Yes, okay. Perfect. Nice So, let me check again Okay, I'm more features Authentication this has been the year of the security for us. I could say no, it's true That is not true But it's something that we have been working hard because maybe if you joined us last you joined us last year But I think that is quite important for us is that Kedah must be secure as default Must be safe. Why because Kedah for working needs more than the usual privileges because for instance if you want to check the amount of messages on a Rabbit and Q you need to list them and it's a it's a permission that usually applications Don't have is more my management permissions or for us the securities the most important thing That we have taken to account Due to this we have migrated to AWS v2 SDK We migrated six months ago if I'm not wrong. We also support we also have supported a New pot authentication. Do you know what means pot authentication? It's quite different depending on the provider in AWS is here's a role assumption in GCP and Azure means we're not identity federation But identity we need to standardize the name. Sorry. We are lazy for remember more than one But we have also added support for Bolt Kedah now can't goes through AWS secret manager or GCP secret manager and pull the secrets directly this in combination With the previous step with the pot identities. It's the most secure approach that we can provide if they should be the previous service doesn't support that Role assumption or that identity why because you can go to take this the secret from the secret bolt that you have and Sorry Use it and the secret is in memory You don't need to pass the secret to Kubernetes API, which usually is not it's not more more and even more You are tired You want faster be clear, please terrible orders Okay, go faster As you're perfectly we have integrated these approach these manage identity support on several scalers Such as Prometheus that now support pod identity in different clouds and also we are migrating to the new Azure SDK Awesome. Thank you. This is a great great stuff No, let's talk about performance just a real quick. I Think that they have discovered who is the technician, right? Yes. Yeah, I know it Okay about performance last year graph and a lapse kudos to graph and a lapse offer to CNCF projects Free a free account for executing performance and monitoring and so on and we integrated Rafa na K6 that which is a nice tool. I suggest to give a try where is it open? Open skip verification. I trust on my own link. Oh my god I think we can go ahead and yeah, yeah, yeah Okay, you are fired. I know it and the worst part is that I have it already prepared. I'm terrible Okay, basically I will suggest take a look to this repository this repository is public It's in KEDA org is built on open source technologies And there you have the end the performance test the load test that we are currently running We are testing KEDA with 1000 scale object and one metric right now We are working on improving the performance checks, but We want to present or introduce this repository just for giving the choice and giving information For running in your own infrastructure check KEDA at the scale in your own infrastructure And let's continue. Yeah. Yeah, I speak too much. It's true Maybe I have to ask for a salesman role awesome So let's talk about the future. We don't have much time, but let's let's do it So what is next for KEDA? We have covered the auto scaling stuff. We schedule Kubernetes jobs We can do all the all the good stuff. So the next Cool feature could be actually predictive scaling. We are talking about is for ages, but I think we should do it So basically what is the what is the what is the idea behind this? We have all these metrics coming to KEDA So we know the state of the system Let's say and we can we can learn some model that basically about like the about the stuff that's happened And in the past and maybe predict the scaling a little bit So let's say that we know that every every second Friday There is a huge huge spike coming to our application. So maybe we can pre-scale a little bit a little bit faster So we can handle load even even faster than be the traditional scaling the other other other interesting stuff could be story-scaling because you know, we would like to also scale the scourge and we are actually discussing with the data ankle burn at this community to No, yeah to to implement this kind of stuff and also cloud events we talk about cloud events So we would like to extend on these capabilities because so far the The current implementations is just limited But we would like to expose events about basically anything that's happening in in KEDA And now let's talk about the last thing on the roadmap Which is the HTTP add-on? Yeah, because maybe this project is not new It's something that we are have we have been trying to put during the last year KEDA works really nice. I don't say it users and customers say it customers not they don't pay me But users think that in general or the experience that I have said I have had but what for what can we use for it? TDP traffic because it's TDP is quite different It's not an event or it can be but the synchronous event You must be there if you are not there because you have a scale to zero and you can't hold the request You are you are gone the the request will be lost in this case the HTTP add-on Allows us to hold the request thanks to the interceptor in the middle as we are a bit out of time I will be quite fast. I am around so you can ping me. You can grab me you can grab me from my head No problem. The architecture is almost like this There the red Boxes is the add-on the HTTP add-on you can just route the traffic through the body directly from the load balancer through the interceptor and the interceptor will Manage the traffic will handle the traffic will hold the request if there is an any abiable instance behind and will trigger the scaling out for preparing the instance and then their regular flow of any network in traffic and Yeah, go I know that should be any paparazzi or something so because they told me they sent me a stolen picture But it was funny. So I use it for my demo And if we check the name of space Where is it? HTTP demos I can see that there is any Pod deployment is scale to zero. Yeah, now I'm sharing the correct screen good for me. Yes. Yeah, and let me go to the demo It's loading call the start. Okay. They took a picture of us It's from yesterday. We are we were yesterday Yeah, check it's been a key stroller than me And what happened in I was the pot click it again. It has been instantly Why because the post world where it has been already there the post the but the pot is there Why because the first request has triggered the scaling out and then the pot is abiable So we can hold the request for an specific time there is a timeout that you can configure but we are Supporting in this case They're scaling to zero also for HTTP. This add-on is on beta. We have an issue For trying to draw their roadmap their roadmap until going to GA So I invite you to go there and put your use case give a piece by give us feedback open PRs Yeah, please open PRs The the left guy will be angry if you don't do it. Yes, and he's quite a strong You are just on the picture just And I think that we are almost done The last one, you know, there's a way to open the current slide, right? I have to say that and this is hundred percent true. I had no idea. I haven't idea about that And you know teach you that and you are thanks appreciated Thanks for joining us. Do you have any question? So please write the session and this is also the survey for cat eye users So if you are cad user, please fill the survey. It's useful for us. Thank you. Any questions? Yeah, we have some questions So what is the difference between? Keda and custom metrics and is there advantage of using that? So Keda is build-on custom metrics, but you know custom metrics you mean the the adapter, right? Custom metrics adapter. We are using custom metrics under No worries. I can answer The main difference is that custom metrics is another API if you are using Prometheus adapter is a really good option but Prometheus adapter Don't doesn't support the scaling to zero as far as I remember and you must scrape all the metrics into your Prometheus So if you want to use AWS metrics, you can do it Prepare your wallet because as far as I know you will pay too much in cloud watch But it's something that you can do the good part of Keda is or in my opinion the the strong Aspect of Keda is the scaling to zero, but also the the scalar catalog built in without any change and another important Thing for Keda in my opinion is that is more developer focus the focused on developers Because we try to make the things simple for them in Prometheus adapter You need to reconfigure as an admin all the relabels for making the metrics available with this Keda You can deploy in a scale object and it's built or everything is built in you Just don't need deploy the scale object and you don't need to reconfigure anything at any level neither in Prometheus Okay, I question Hi, I like to know if you use this HTTP Scanning to zero to you preload your docker images to the nodes. So you don't have to pull them first If the request arrives, that's an interesting stuff. No, we haven't implemented is over This is a interesting stuff demons that was always right a question regarding your Don't I Also scaling your interceptor Not at the moment, I suppose You have interceptor which is accepting the traffic as a forest of deployment Sorry, sorry home the interceptor is also scaled based on their own metrics through Keda Out of the box. Yeah, you can configure that target a request for scaling But it's something already a scale because otherwise the interceptor could be a problem So for interceptor does all the request start coming through interceptor then Is it like up the ingress all the request goes to interceptor then goes to all the services. Yes We are working on isolated interceptors, but right now or I mean interceptors on demand But right now there is you have to in the to use the same interceptor But you in the opposite of Keda that is cluster scope due to some limitation in the metrics server API You can deploy the HTTP add-on name spacing. So it's not The best way, but you can have a tenancy isolation right now Thanks in your first in your first example when you were scaling up to four replicas there was two time ranges like 24 hours and for me personally crown is the Program that produces event each minute and instead of that your Time range was like 24 hours every second. It was this was just for them a purposes. So basically When we design the cron scalar, we need to find a way how to basically Enable users to specify some specific time, right? So yes, we have the start and end there with the crons my question was But what's the reasoning behind the name selection whites doesn't name this clock or like time or time range for something? Yeah, probably that's probably better, but It's a good question a really good question. There is Yeah, there is an there is an issue because we have discovered you are not the first one Yeah, put all these to us. You are totally right We are working on a new scaler like time window or something No, I don't remember and the pre-k the cron scalar but cron scalar has been there since the beginning And it's quite legacy right now. I know a lot of a lot of users are using it So it's hard to the pre-k the stuff. Yeah, we need to carry on with the name. Yeah, sorry for that Thanks for the feedback. We are working on it. So we are out of the time. I suppose Yes, so thank you if you have any questions feel free to reach out the world. Thank you. Thank you everybody