 So, welcome to the, oh, that's the wrong slide. Let's start from the beginning. Welcome to the Prometheus Functional Update for August. I'm Ben Kochi, I'm the Prometheus Lead. Joshua is our product manager and we have engineers on team Paul, Kevin, Julius and Mike. Mike just joined us recently to work more on the front end. So, straight into 9.5, so in 9.5 we added some nice new automatic monitoring for the idea to production and automatic deploys. So, this is really great for demos and at the end of this, Joshua will actually do a little demo of this. It should be nice and easy for everybody to now see automatic Prometheus metrics and starting to get some actual customer facing metrics into the four GitLab users. Also, in 9.5 we've improved the GitLab Rails Ruby metrics. We have done a bunch of cleanup and added a few more metrics that has been helping out in production to help instrument the GitLab install and the gitlab.com install. And we've done a little bit of cleanup to improve the metric names to meet our guidelines. We also added a simple web server into the Sidekick job runner. That will allow us to expose worker metrics and finally get much better information from the workers to find out what they're doing. And we'll be expanding on this in later releases. We also continue to do some minor improvements. We've did some UI bug fixes and improvements and we also had a bunch of bugs that were discovered in the Prometheus multi-process library which is the backend code that instruments the Ruby code. And that was causing some problems in production we had to turn it off. But those bugs are all fixed now and we've been doing some nice performance improvements so that monitoring does not get in the way of actually serving users. So coming up in 10.0, we're gonna have a big focus on getting parity, feature parity with the influx DB metrics that have been in the system for a while. Influx DB is used in a little bit of a different way. It sends event streams instead of metric streams. So then what that means is for every event that happens we send a metric or we send a sample event to the influx DB. This is not something that is possible with Prometheus. So we're going to be translating that and improving on things with Prometheus we'll be actually be able to have live real-time histogram data instead of having to do post-processing on event streams. And this will allow us to eventually remove the influx DB metrics from the GitLab application. We're also gonna be working on another major feature we're gonna be trying to improve the application performance metrics so that users will be able to put their own metrics and be able to see their own metrics coming from their applications instead of just side metrics like the CPU and memory usage of their running applications. We're also going to start working on adding canary support to the Prometheus we've been wanting to do this for a couple of releases but it's been we've run out of time with all the various other work going on and we'll finally be able to we're finally going to put some effort into getting the canary features added to the enterprise edition. Prometheus in the gitlab.com production we've been working on the influx DB to Prometheus gateway so that the existing influx DB metrics are available to the Prometheus server so that we can do more real-time monitoring that influx DB hasn't been able to do because of the way we ingest that data. There's a little bit of blockers because of some inconsistency in the labels but that's being worked on and hopefully should be cleared up soon. We've also spent some time to help production with their alert manager and alerting so that we can reduce the alert fatigue for production and improve the alerting visibility for the production environment. And we've also been working on trying to improve the git host alerting because they've also been drowning in alerts and need to have a little bit of sanity for their monitoring. And now I'll leave it to Joshua to do a quick demo of the idea to production. Cool, thanks Ben. And I want to do a full ITP demo. We'll just kind of focus on the monitoring section here. I'll look at the changes folded into the sales script. Also while I'm getting set up here I just wanted to thank Jose. He wasn't on the team list there for some reason but he's done the vast majority of all the front end code here for all the meetings items. So I definitely want to thank him and all his contributions and make sure other folks know that he's been quite busy helping out. So cool. So this is again, a center ITP demo has been deployed by our Helm charts. And if you deploy with our Helm charts like we do with ITP, this is all very, very automatic. So I'll go ahead and create a new project and just take a new URL. Just take preview of nine five in case you haven't seen it. This is running RSC four which is like half an hour old, maybe an hour. Got a public project. Nope, I've already had the same name. So let me just do that. Put back in our URL here and create project. All right, cool. All right, so we need to make a couple of minor changes to the example app here. Nothing major, basically just add an option to interest a little latency because before it was responding so close it would always be zero and so a zero latency chart is not that interesting. And then we also introduce an error in point if folks want to use it that they can then generate some errors to show the metrics. Of course, as you all know, click on auto deploy, click on Kubernetes or we can even do Kubernetes with Canary here. Just all I have to do now is enter in our domain name which I think is this one, let me just make sure. Yep, we're good, I just couldn't see it. We'll scroll down, commit this into master and we'll of course have our new pipeline that is running. Take a couple of seconds here to get things into staging. While that's happening, I should mention that this is not just for people who deploy their Helm charts or deploy with IDP. What we actually do is we can pull these metrics off of the nginx ingress which is probably the most popular ingress for Kubernetes. And so if you're pulling there and if you're using nginx ingress you could simply set one flag on that nginx ingress and you'll get the permuteus metrics. And then once you have that, all the rest is done. The only thing with our Helm chart is it automatically does that for you. It uses the nginx ingress and then sets that flag. But so it's really not a lot of config either way even if you don't use it. It's just that it's not totally 100% automatic here. So let's go ahead and open up our page. Here we have Hello World. You'll see it takes just like a quarter of a second to load here, which is a bit different than before but not too long. We'll give it a couple seconds to get some of the monitoring hooked up. We'll give it some traffic to take a look at. We can also hit the end point here as well just to give you a sense of what that looks like. There we go. And if we head back to our page here and click on monitoring we'll see some metrics coming through. And again, it takes a couple of seconds here for the metrics to get picked up. But you'll see here we'll have the error rates which is how many errors or HB500 errors they have in per second, the latency and then also the remember request per second as well. So if we just hit refresh here we'll start to see data pop up in just maybe another 15 or 20 seconds. We can also just continue with some more traffic to keep things interesting while we're waiting. And of course during the normal demo you can talk through or go through other areas. But there you go. You can see we got latency popping up for our first data collection point. And we'll see a couple more seconds here. So you'll see throughput coming through and you'll see HB errors coming through. Again, since we just deployed like half a minute ago takes a couple of seconds for things to cycle through. There we go. So, and again as of course you generate more traffic and as we get more data points here you'll have a little more of a semblance on the chart. But you can see really there how easy it is to get some pretty important metrics. You know how many errors you're generating what's your customer latency and so you get 17 milliseconds for example here. And of course also on the error on the throughput side you know not too many requests per second but we're doing all right for an application that just launched a couple of seconds ago. So pretty cool stuff and of course you have your CPU utilization here as well being pulled from Kubernetes. So we do have a page to load Sesame for data which is when you don't have any data whatsoever and we're kind of pulling it from Prometheus. You'll see it kind of pop up there for a second. It's harder for us to know like for a certain chart if there is a data point or not but we can start to look into it and see if we can improve it, have like a little chart here saying we don't have enough data points or something like that but that's a good suggestion Clement, thanks. Any other questions? No? Okay. Well, thanks for everyone's time. We'll give back 20 minutes and maybe then if you wanna wrap up or we can sign off for now. Yeah, no, if nobody else has any questions that's just absolutely amazing to see the useful metrics for customers. And with that, yeah, 20 minutes back. Thanks everybody. Thanks everyone.