 Hello, hello, this is Room C. Welcome to Room C. We are going to go ahead and get started with our first speaker who is talking about agile data visualization with Dash Builder. Can I ask you to start by introducing yourself? Hello, good morning, Buenos dias. My name is William. I'm from Brazil. I'm a software engineer working at Red Hat. So that's basically it. I love open source and open data. So I'm really glad to be here and thank you for being here as well. So yeah, so today what I wanted to share with you is my experience going from big open data system to small and agile data systems with these two that I work with called Dash Builder. So to get started, let me give you some context. We have an open data group in my city where we try to do as some of you and probably do as well, which is basically go to the government sites and grab data and put it in a format that machines can read. And then we create visualizations or systems using this data. And at the beginning we have, you know, big systems like we have back end, so have applications and it was relying a lot of code. So it was, since we are a small group, it was hard for us to maintain the code because we have no funding. We do everything in our free time. So yeah, so it was really great. We created some big things. For example, this system, it's a big system that scans the government, the federal government, money that's given to the cities and we built some observations to show how much the city is depending on the federal government. And but it was a very big system and hard to maintain. And we had this system on the right here which is, it was a easy way to search laws in our city. And we use NLP to put some tags on the laws to make it easier to find some law. Yeah, it was all great, but it was really hard to maintain. So our problem was that, and it used it to take like one to two weeks to put on air. So it was very slow to deliver things to the citizen. So today we change our approach. So today, we deliver faster dashboards, small visualizations or small applications directed to the what people ask us. For example, last month we had a neighborhood in my city that were suffering with flooding and they wanted to know if the local government was doing something to help them and we scanned the data and quickly created a dashboard so they could see everything that was done for their neighborhood in the last few years. So it took like one to two hours to build the dashboard and put online and send to them to see this information. And how we did that, so we use a dash builder. I know that there are other tools in the market that probably could achieve the same goal, but with dash builder, it's a tool that I also work at Red Hat. I am one of the developers of Dash Builder. But one great thing about Dash Builder is that it's a really free open source, free, it's software-free. So it's very good, you do not have to pay and the license is very good. You can put in your application, do not have, you know, to give information to anyone to use. You just pull the package and start using. And all the dashboard definitions is using pure text. So we don't have to go to some proper format. It was also something that we wanted. And Ben text-based, it is to reuse. Because for example, if you want to do some data visualization, you can recall that you did it before so you can copy the code and using another realization with different data sets. And it's also easy to consume in JSON file, CSV, and also the metrics format. Because, you know, Prometheus is a time in series database and to read data, it reads from a metrics format and this format is supported by Dash Builder as well. And another good thing is that to transform the data is very easy. So you just have to declare if you want to group, if you want to filter, or if you want to slice the data. It's just like two to four lines of code, YML code, and then you have the format that you want. We don't have to install anything and it's all client-side. So all that thing that I said about backend is not anymore here. You just have to put the data somewhere that's accessible and we can build data visualization with that. So how we do this? Basically, we start creating our data set. The data set is the core concept of Dash Builder and data sets can come, as I said, from JSON or CSVs and we can also declare inside the YML. So we have here, I don't have to do this, so I have this sophisticated stuff here. So we have here a new data set. I can put the content here and then I can declare the visual part. So here I have A2, B3, if I want to visualize in bar chart, I just say that I want to put a bar chart here and I say where is the data set and it creates visualization. And as I said, any accessible URL will work and we can cache the data. So this is very important. So if you have a big JSON, you can load the entire in the memory and then Dash Builder will work entirely offline. So it needs one request to get the data and it works in the browser. Another good thing is that we can transform JSON. So in this site here, Economia Popular, they are putting together a lot of metrics, economic metrics about Brazil and but their format is not compatible with Dash Builder. Let me just go back again. So Dash Builder accepts data in a 2G JSON array. So the data must be in this format here. But as you know, most of the time, the data will be in another format. So Dash Builder supports this great tool, JSONata, where we can transform the JSON and grab the format that we need in Dash Builder. So then using points to that URL, we put the expression. In this case here, I had to change this column, the first column, the year because it was understanding the color as a number. So I had to make sure that it was a label. Then I can visualize it any way I want. So when you are building visualization, we'll talk more about this. Basically, you just declare what kind of visualization you want to do some configuration and then put the data set and everything usually works. Sometimes it doesn't work, but easily it works. So having the data set, then you can display the data any way you want, as I said. So Dash Builder provides you things like the full charts and the way that these charts are put in the page is by having this concept of the player, which is a block of the page that visualize a part of the data set. So imagine that you have the data set. It can be a big data set, but you want, for example, to group by some column. You can build what we call lookup, which is just basically an operation over the data set. And then this player will render the data set on the page. We have a lot of displays that are really for use. And usually they are compatible with the data sets. You just provide a category and a value, and then everything works. So I said page, so we can have multiple pages and have a way to navigate between pages. So I can have really big YMLs and then I have a lot of pages and then organize them with navigation. And one other good thing about Dash Builder is that if you don't like the visualization, if you have some tool that you use in your company or any other Sharks library, you can create your own visualization. This case here, this part of this small dashboard is all we did with JavaScript. So we used Dash Builder, built-in components, but we could add some of our components here. And if you need a backend, Dash Builder has a Quarkus extension. Is anyone here a Java programmer? No? It's hard to find Java programming in data conference. I'm used to that because usually people like Python. So I'm always the exception because I love Java and we have this tool called Quarkus. You can use Kotlin. Do you know Kotlin? Okay, so it's a little better than Java, at least. So you can use Kotlin with Quarkus and put your dashboards with a backend. So it's really easy to use a backend when you integrate with this tool. And how do you do that to develop it? So you can start with N-text editor, write your YML, and then put with Dash Builder. It's all in our documentation, dashbuilder.org. But then, or you can just go to this URL here and start developing our dashboard. And then when you finish your dashboard, you click here and it will be an open shift, but you don't have to do that. You can use any stacked provider to put your dashboards. So for example, GitHub page. How many of you have used a GitHub page? That's very good, right? Because you develop, you commit, and then everything's online. So Dash Builder can be used with GitHub page. And the data set, as I said, you declare, sorry, the dashboard, you just declare the data set. You can declare the columns. If you don't do this, Dash Builder try to guess what are the columns of your data set. Then you create the visualization. In this case, I put here a bar chart, but you can also put components that will filter other components. So for example, here I put a selector. And then when I click on this button here, this chart is filtered. So it's all for free. So this was something that we wanted. We didn't want to go over all again to do, you know, the JavaScript and Java development to grab data and go row by row and filter and then update visualization. We didn't want to deal with this from time to time. We just want this to work. And Dash Builder provided this because it's all built in. And as I said, you can provide the dashboard using OpenShift. Just click on that button that I show, and it will require you to log in into online OpenShift. But then you don't have to be with that. You can use GitHub Pages. I have a video here where I go through all the process. But in this post here is all described as well. All you can just use this NPN package, bring the Dash Builder into your application, and then unpack, and then use that as part of your application. So if you want to get started, you don't have to read a manual. We have a lot of samples to get started. So you can, for example, grab one of these dashboards, just change some things and it will work. As I said, we use a lot of code. So sometimes when we have some requirement, we just copy all dashboards and change some things and it's working. So I have a lot of open data samples here. I hope to go through some of them. And just to finish, so one great thing about Dash Builder, you don't have to install anything. But this is one of the great strengths of using Dash Builder. And everything that you probably need is there. So you don't have to do any development to build small observations, simple observations. It really is to reuse dashboards in YML format. And it's great to consume different JSON formats. So don't have to adapt your data of source to the dashboard. Usually you do not have to touch your source of data. Because one hard part of building open data is that usually you have to go through scrapping and then format the data to put in the format with Dash Builder that was really easier for us. And it's to deploy. So usually you just have to click and everything will be ready. For example, the site with open data will be important to that online tool, read, save, and then it's published. So it's all in the browser. So let me move on to some coding. I forgot to. So these are our dashboards all YML-based. Do you know what happened in Brazil on January 8th? So at that time, a lot of people invaded the Congress and was really sad day for us. But basically they published it. Some people were arrested. And they published the data in the news about the age of the people that are arrested. So one thing that we did was to find the median age. So look at this. The median age is not to know crazy teenagers invaded the Congress. So they were mostly senior people that in anyway. So it was very interesting because we built this. It came to the news. We built like 10 minutes and posted in some groups, WhatsApp groups, and also in the internet. And this became well known that the age was different from what we are used to do. There was one woman arrested with 73 years. And this was done. We didn't have to create the data. We just cop the data and everything. We just cop inside the YML and build the visualization. We didn't have to put the data somewhere. It was all inside the YML. So other thing that we had in Brazil was this thing called Secret Budget, where the money from the federal government was going to the cities. But we didn't know how much money was going in or what was happening and who was giving the money to the cities. And it was really bad because we found the transparency. We couldn't find what was going on. So a friend of mine did all the scrapping of data. And we created this visualization in one to two hours. It is a big visualization. So we have a lot of dimensions and integration with other dashboards. But a good thing is that some journalists asked us to modify this to gather information that they wanted. And we could modify in GitHub safe. And it was ready for them in two to three minutes. So it was really good to do a development close to who was interested in the data. So let me go back to the slides. So that's it. So if you have any questions, we'll be around. I don't know if we have time. So we have five minutes for questions. And that's basically about Dash Builder. Thank you so much, William, for that great talk. I'd love to see live coding or some dashboards. We have time for questions. So we could do questions in English or Spanish. Do you want to have a question? Let's use this, please. What does the back end do if you mentioned there was a Java-based back end? What would that be for? So let's say that you want to access some specific system that is not accessible from the browser. Usually, cars issue. Because when you are running the browser, it's hard to reach certain URLs. You may have some authentication that's not supported by the browser. For example, we'll be the dashboard where we had to get the token from a serviceee. And we could do this in the browser, so we put it back end. So back end is usually to strange authentication schemas or if you have to access a very specific system, like a mainframe or something like that. Any other questions? I have a question. Sure. You gave two examples where you talked about how quickly you were able to build these dashboards. What is the fastest one that you've ever built? I can show you now. Yeah, please. So as I said, you go to this URL. I think one open here. So this is our online editor. So you can develop on the left side. You do the YML here. And then you click on this little I. You can visualize what you just did. And this is a simple application that explores all the Dash Builder components. So let me create a new one. So you go to the home page, click on New Dashboard. So you have one sample for free, but that's cheat. So let me start to inform to crash. So here's the data set. So that's basically it. So are you good? Very nice, very nice.