 Okay, next up is Talker from Grafana. The plan for the talk is that it takes roughly 20 minutes, so that we have 10 minutes for extended Q&A session after that. So please, if the talk is over, if you could please, stay seated and remain quiet, because otherwise nobody would understand the questions and the answers. So, welcome, Talker. Thank you, thank you. Yeah, I hope to talk a little bit about Grafana, specifically the upcoming version of Grafana. And if I have time, I'll show some tips and tricks in the demo as well. But these are all type of questions, because there's usually a ton of questions when I do these talks. My name is Porte Genni, I'm the co-founder at Grafana Labs, the company behind the project that's supporting the project and also trying to build services and products around Grafana. Grafana is an open source monitoring and data visualization tool licensed under Apache 2.0. Of course, it's found on Grafana. It looks something like this. You build, or maybe you prepare the light theme or the enterprise theme, where people are scared of dark device. I actually might use this in the demo, because it looks better in this room because of all the lights. Yeah, you build the beautiful dashboards of Graf and other panels. The graph is kind of the cornerstone of Grafana, the powerhouse panel. I just want to get a show of hands in here and grab a photo. Who in here actually at some point can use Grafana? Can you raise your hand? Okay, that's good to know. I was hoping for as well. Anyway, you might not know some of this. I'm just going to have a few slides on the history and the go for the project. This is a starting as a whole project of mine four years ago now. Almost four years and a week or so since the first release. At the time, I was really in love with graphite and time series application metrics. That's the end of life. It was really fulfilling that the dashboarding solution for those tools were lacking. I was also really enjoying using Qibana, which had a really nice solution for dashboarding for logs. So I took inspiration from that and started working on Grafana. Initially, we also supported graphite, but over time, we added support for other data sources like InfoxDB, OpenTestDB, Prometheus, which has become used to popular lately, and Elasticsearch. Lately, we also added support for MySQL and Postpress, so I can query your SQL data directly from Grafana. We also have, these are the core data sources that are from Bonn. Actually, we've got CloudWatch, which is also a core data source, but it's commercial success. Anyway, we, at least, we're not suitable for this conference. Anyway, we also have a ton of data sources on the front lines, onions and paddles, apps, and a community-powered repository of the readymade dashboard. As I said, this project has now existed for four years, and during that time, we're getting a really incredible amount of growth. Starting from version two, we added some very basic server reporting so we can sort of see what versions are running. And so the last three years, we kind of seen how many Grafana servers are actually running, not just track downloads, but actually see how many are actually using the software. And of course, the real numbers, so right now, there are 130,000 reporting servers, not servers, but of course, it's a lot more reporting, because of what people turn off to this reporting. But it's interesting to see how this trend has progressed in the last four years, that the trend is also increasing in velocity and kind of exponential growth. And part of that is of course that Grafana is amazing, and it solves this kind of niche domain where there's a lot of competitors, but I also think that the need for observability and good sort of monitoring tool has also increased, with that of more elastic infrastructure and microservices and more complex application and production environments, the need for good observability tools and monitoring tools has also increased. So if I was going to write in that way as well. There are just sort of pure installations that we are almost seeing a tremendous growth in the public source project, in terms of sort of interest from outside contributors of the companies as well. And you're also growing the 14, just in the last couple of months, we've grown more than double the team from three people to seven people. We also launched a community site to help users and developers be successful with Grafana. And it's actually a lot more to support this community site because it supports so many data sources. And most of the questions are related to a particular data source, how you query this and how you query that. So it can last in the community to also help with this because it's causing a lot of work for us in the core team to actually answer the SQL questions, how you query this in SQL and for CBO, for me, this. Anyway, pretty amazing growth for that project. It's also really great, but Grafana can be used in so many different domains and circumstances. And most of the companies that use it also are really cool and they use it for different things. A couple of years ago, we saw that SpaceX was using it in their launch control centre when they landed the first top-9 rocket. When I saw that, it made my day. But it's just cool. Of course, I want to use Grafana for application monitoring and infrastructure monitoring and metrics and application analytics was also being used in agriculture where they put sensors in the ground and look at the fertility of the soil or use it for industrial sensors or home-motivation systems. I mean, a couple of guys used it to monitor the availability of t-shirts in the exhibit hall. Let's see. Now to the real interesting part. Jump over the intro to Grafana since most of you are familiar with it. Most of it, as all of the demos I'm going to do is in V5. A lot of things have changed in V5 in terms of just looking at feel and we'll move your sheets a little bit so don't be mad. But hopefully, all of the changes are going to be for the better. As I said, Grafana is all about building dashboards composed of these sort of panels and that hasn't changed. I mean, nothing has changed beyond just some one of your exchanges. And for those of you who are unfamiliar with the basics of Grafana, you add these panels, you jump into edit mode, and you are taken to the metrics tab. The metrics tab looks different for each type of data source. So in this case, I'm using graphite and I get a graphite query editor that allows me to explore the metrics available in graphite. And once I sort of found a metric I can apply transformations to that to the graphite transformation functions and other, of course, different options for the graph panel. But, I mean, this is nothing new particularly for the V5 release. If it was a while since you used Grafana or if you hadn't seen Grafana before each panel is this isolated thing that you write your metric queries in and show data. But there are ways to sort of build more dynamic dashboards as well that use something that's called template variables. That allows you to build more dynamic dashboards where you get these kind of drop tiles at the top. That allows you to quickly change the scope of the data for multiple panels. Anyway, back to what I really wanted to show and talk about was V5. The biggest sort of thing that there are a couple of big things but the biggest thing that I've worked on and I'm really excited about is the new grid system and what it kind of enables. Before Grafana it was kind of clunky to move panels around, create layouts and all that felt very clunky. So now in Grafana V5 you have kind of more modern layout system where panels move out of their way when you resize them and it's a lot easier to move panels around. But the most exciting thing is that it's going to enable more layout types so you can have layouts like this that wasn't possible in Grafana V4 because everything had to be laid out in rows where each with a height of each panel needed to match so you couldn't have really high vertical panels on the side. Anyway, so that's really exciting but I've fucked up this layout though. But I'm really excited about that aspect not only is it more easier to use and build that support but it's going to enable more types of layouts and also new types of panels because now we can actually have panels that stretch vertically like this so the next thing that I want to do is build a new multi-stack panel that allows you to sort of stack a lot of metrics vertically which is something you can do with single stack but it's very inefficient because each single stack panel requires separate queries. So that was the new grid system. The next big thing that I want to touch on is the UX changes so we've changed the side menu a little bit and also all of the pages have a kind of new look to them with this halved look. Nothing super interesting here. Community users might be so excited that they have a search in their data source view because they have a ton of data source instances but otherwise it's mostly a cosmetic change but also it's so you can navigate around the interface easier and it looks a lot better but a more meaningful UX change is that the dashboard settings views has been now kind of combined into a unified view so you don't have to sort of switch these views to a dropdown anymore so you can now access annotations the section that we've previously called company is now called variables version history where you can restore different all the versions of the dashboard. So that's new grid and new UX in terms of a new capability the biggest thing that we've been working on for it feels like ages is folders and dashboard sort of dashboard folders because that's been one of the top rated feature requests sort of new other ways to organize the large profile installations and group dashboards along team lines or application lines so previously to do that with tags, but tags people didn't feel that what sort of tags was enough of a tool to sort of organize a dashboard. So that's why we can introduce folders which allow you to expand and collapse these regions or sections, we also added starred and recent dashboards to the search and the folders can also be managed, or dashboards in general can be managed through these two new large dashboard pages which allow you to do some quick administration of dashboards and the action move multiple dashboards or delete multiple dashboards, and it's also from these views that you can administrate the folder permissions which is also a completely new thing in front of v5 in that you can access permissions on folders and on dashboards and of course if you set the permissions on the folder they will be inherited by the dashboards that they contain so can sort of review that quickly so the permission system is pretty easy you can add permissions on teams which is also a new thing in v5 can show that but you can add permissions on the user particularly the user as well and you can add permissions on the team and if I go to dashboard that is contained so any dashboards has permissions as well and then they get their permissions inherited and you can override the folder permissions but you can override the default role-based permissions as I said teams is a completely new sort of thing that we are adding in v5 mostly now they're only there for the permission system to make it easier to manage the permissions right now we only have permissions on folders and dashboards but we hope to add them to data sources as well so lastly there's one feature that I want to mention it's hard to demo it because it's a back end feature but I think it's kind of so we appreciate it and it's interesting in a way a provisioning system previously in Grafana when you created new data sources or new dashboards it had to be done through the UI or through the HTTP API which kind of requires that Grafana is running and it's a bit tricky to do from a configuration management perspective or sort of automation perspective people have been managing to do that tricky so now Grafana has some data sources and configuration files so you have Grafana and have your data sources and your dashboards over there it's also active soon so if you change them on disk it's going to be easy you don't have to restart Grafana for that to change to be picked up so that was kind of the v5 feature highlights and I will have a few features that I want to highlight as well that one is kind of aware of it's quite recently added as well so annotations is a very old feature in Grafana that allows you to overlay which event data I want to show so annotations is an old feature in Grafana that allows you to overlay these events on graphs which could be deploy events marketing, knowledges, anything but before in Grafana you had to set up an annotation query that affects these events from a data source like Elasticsearch it was BBE to meet use, Grafana anything that could contain event data so what you couldn't do was create these events from within the UI and that's actually something you can do starting from v4 or 5 I think it was when we introduced this feature so just by holding down command on Mac or control woman or some Linux you can open that add annotation view and describe an event and of course this can also be automated and done to the game stage to create these and these are stored in Grafana some database so you don't have to sort of set up a data source to store these what else you can do is actually do the same thing but mark a region to create a region annotation and write a description and the tags here are kind of important in that it will allow you to show this event on other dashboards and find some subcharter filter for it now I'll say this if I now go back to this new grid dashboard I can add an annotation query called outages and say I want to look at all annotations that are created in the Grafana database with the tag outage and hopefully if I zoom out you should see there it is if I tag it I can then create the annotation query on other dashboards and actually show them on a conditional basis so I can tell them that annotation on and off another I mean I touched on templating briefly before what you can do beyond just sort of using template variables to create dynamic dashboards is you can also have a repeat paddle and what's new in v5 is that you can have a paddle repeated in the vertical direction not just horizontally so in this case this single-statt paddle is being repeated for each value on the slider and you can have there's also a concept of rows in Grafana which is still there in v5 it's slightly different which value you select in a template variable so the last tips and tricks is showing how you can use style overrides in the Graf to really customize how your graphs look in this case I'm showing necropraffic in and out and I'm showing out metrics on the bottom from a negative axis and the way I'm doing that is using a display override saying all series with a name out in its name should be a separate stacked group and should also be transformed and visualized in a negative y-axis there's other use cases for style overrides in that you can really customize how each series should be displayed you can also say this first should be displayed with a line fill or this should have points so I could say this should not have any lines on it so you can really customize the look of your graph using style overrides there's also a cool style override called fill below to to fill the region between two series which can be useful if you have a linear max in average more of a region that pretty much covers most of the sort of my demos and wants to be some time for questions I'm sure there are okay, please stay seated and quiet and time for questions oh yeah, sorry I should have mentioned this so Vera on Monday we have written it down for Thursday but since we're going to go to Boston and travel yesterday we kind of pushed it to Monday so the plan is the plan is to reach it on Monday and stable we hope to review in three weeks time at the Grapana Club so Grapana Labs have revised post-agmetric services which is kind of made out of poster services for both Grapana and Metrics and that's kind of been made by the driver and we also have support services and training and other sources of revenue as well different services and you changed a lot of the old test points is it possible to use them or have you designed them or so the question is sort of with this new grid system that is introduced in v5 how will all dashboards work with that new system because positioning is completely different so the answer is that your old dashboards will work just fine and they will Grapana will try to migrate them to the new positioning system so it should look close identical with slight differences in panel height because in Grapana v4 and going back you could set panel height and row height to a sort of per pixel level but in the new system there is a kind of snap so you cannot set the height of panels on a per pixel level you have to set it a little bit more coarse grade so then it's going to be slight differences I also recommended you try the beta and you have to update the plugins because panel plugins could have some issues with the grid system we tried to test with most of them but a few might need an update the biggest thing that you might notice is that if you change you say the dashboard in v5 and kind of import it in v4 it's not going to work but you can use the word downgrade you can use the virtual history system to downgrade a dashboard you say so you are adding to your guitars so provisioning of dashboards and folders are those features compatible with each other so can you provision folders of dashboards yes you can provision folders as well I think the way you specify folders is that the way you provision dashboards is to specify a specific file where the dashboard can be found on this and then you can specify a folder name as well for where the UI works like if I can create a folder for all of that Jason or something no, well the Jason is slightly different it's sort of because the panel's position model has changed and it's no longer structured in rows it's more structured in a sort of a collection of panels and their position so the structure of the Jason is slightly different but it's mostly sort of laying them out in rows in a flat array now yeah, everything is starting across Databricks there is one big change that hasn't been touched on because it's kind of the IDs for the dashboard have changed and the URL has changed and this is something we work with as last minute related to provisioning systems and also before before going back the slug is the unique ID for a dashboard and the slug is based on the title so if you change the name of the dashboard you break all the links to that dashboard so if you have all the alert emails or links in Slack those links are not going to work anymore if you rename the dashboard and we thought that was a really bad experience and also kind of ties it to the provisioning so we wanted a unique ID and something in the ID something in the URL where we can tie the dashboard to even if you change the dashboard name so that's kind of why the dashboard URL has changed in the file as well would you make system or the H2P API? it is exposed to the H2P API but it's not supposed to we can't provision users and teams through the provisioning system yet it's something we sort of we have talked about but we're not sure about but it's definitely something that we might want to add so you can add specific teams or users and provide some rules there but nothing has done right now how would you deal with providers or commissions or how I would provide it to other authentication providers? with commissions how would you deal with commissions or other kind of people like this and using users? currently we don't have any ways we haven't written any way to integrate the permission system with outside permission systems to help that or all these commissions are something that's kind of only living upon right now but we will probably look into some way to sync these with outside sources as well but currently this is something that only lives in the corner so I was just going to ask you about dashboards yes and the reason for that is to provide some features anything in the corner is kind of tied to that to the dashboard in the database like if you start a dashboard if you have a learns in the dashboard the learns is kind of tied to the database ID so there's a bunch of features that require the database ID but what's cool with the provisioning system both the data sources and the dashboard is that you can lock down anything from the UI so you can control that so this data source the file and the provisioning system you don't accidentally change it in the UI you can control that the region annotations are there ways to get those problem data sources not coming from the database