 Hello, everybody. Thank you for being here. I am, as I said, Guillem Durand-Ballasté, and I will be talking about interactive data going through with Shaolin. These are my Twitter accounts and my GitHub accounts in case you want to check me out. And, well, I'm explaining what I'll be talking about today. Shaolin is a new Python library that allows for easily adding a layer of interactivity to a Python object when working in the Jupyter notebook. This presentation will be no but a tutorial, but an introduction to some of the main capabilities of Shaolin, alongside some real examples in which Shaolin has been used to build a graphic user interface for some actual models. And, of course, the final goal of this talk is just to convince you to use Shaolin and maybe try to build a community around it to make it grow. How am I presenting it? Well, we will basically talk about data going through. There's philosophy behind Shaolin and its features, inspired by science, martial arts, hacking and teaching, combining all those different concepts and merging them in order to build interactive dashboards. Shaolin stands for Structure Helper for Dashboard Linking. But, of course, is also called this way because it allows you to perform some nice data control. A term which can be translated as mastering data through hard work and practice. So, what is it? This is just how we work at my company. It all started as a way to let my boss play with all the code I was developing without having him... well, letting him play without having to actually learn Python. He's a really good programmer, but he's working like in Delphi, Pascal Object Oriental Language, and he just didn't want to know Python. So, when I tried to do something about it, I honestly had no way of doing it. So, when I asked him for help, he told me something like, okay, if you've got a problem, you don't know how to do, just step aside and ask yourself, what would Feynman do? So, I tried to turn the peculiar way that Feynman had of looking at the world into features of my software, which I will explain in the next four chapters of my tale about data control. You also find some of the Feynman quotes that inspire me to build these features in case you get bored of my weird English accent and just are here for the video. So, if you wanna ignore me and just read them, it's fine by me. So, let's get serious. Chapter one, science. As a math scientist in my work, I should really be doing something to honor that term. So, let's talk about science. The core of science is about thinking, asking questions, doubting, and experimenting with nature in order to build knowledge. A knowledge that we tried to embed into something we called models. We've been using language plus reasoning, which is also called math, to communicate our models for the last few centuries we've had success. But actually nowadays, we found Python a simpler way to explain the process of language mal reasoning. Because if you write your models using code, when you try to explain them to another person, you will find yourself evolving from a blackboard into a dashboard. So, science is also all simplifying. Shaolin aims to bring your code closer to the Jupyter notebook by shortening the process of writing complex interfaces using the API widgets package. Although its syntax is a bit complex, it allows to really reduce the amount of code needed to write a graphic user interface. And thanks to the web capabilities provided by the notebook and the API widgets package, you don't even have to know Python to use it. So please, join me in converting Excel models into something more visually pleasing. How we can do that? Well, here's where Shaolin comes into play. Shaolin achieves this by merging object-oriented programming with the API widgets package when using the Jupyter notebook as a backend. Here, for example, we can see a Python object which we transform into a dashboard. By just inheriting from the dashboard class and adding the lines of code that you see inside the redboards. How this weird structure works? Well, first of all, you need to define your widgets. Shaolin widgets are defined by strings formatted in the following way. First, you write the kind of widget that you're going to use, a dollar sign, and then you state its parameter values as showing the picture. You can find all the weird syntax rules that this package has in the example folder of our Jupyter notebook. And, well, just go online and check how it works. Because once you have your widgets defined, it's time to define your layout of your dashboard layout. So, well, here you can see a nested list structure in which the widgets are stacked just to establish how the parent-child relationship works. It's like when you are working with a widget package and you want to stack widget boxes. Well, it's an analogous process, but just with some leads. You don't have to write children equals and all your children's into the dashboard. If you print this stuff, you get this. This is how a dashboard is defined, shall I? You can see that in the first line, the C means that our dashboard will be a column comprised of two rows. The first one will be an in-range slider, which is defined in a syntax mimicking the one provided by the interact function of the API widgets package. And the second one will be a row comprised of two widgets, a dropdown and a checkbox. And how does it look when coded in the notebook? Wait a minute. This is just the example I've shown you. And this is just an object that is in charge of processing an ampere array and scaling it, applying it and applying an arbitrary function. This is not really interesting on its own, but it serves as a good example in order to illustrate how a different dashboard can be combined into building something bigger. Here, for example, we have another one that just takes a map-loaded plot and another dashboard and converts its output into a new one, so you can see what you're doing while you are formatting your data. Of course, this example on its own is not something really great. So, if you are looking to building something more complex, you can also... you shall not forget that. Some of the functionalities that are embedded on the software. In case you want to build plots or you want to do something quite complex, you may find a lot of dashboards in our library that can save you a lot of time when stacking different dashboards. For example, here you have a color palette which has embedded all the seaborn functions on the map-loaded color maps, so you don't have to worry about about choosing your color palettes again. In this case, this dashboard, also it's using cufflinks in order to plot an arbitrary data frame and you can actually select what kind of plot do you like and which columns do you want to plot for your data frame. And also, I think I added finboards among different cufflink things for making your plots. This is what Shaolin is about and what we are trying to build here. So, well, let's have a quick recap. Viral models using Python objects and then you can sort them into interactive dashboards just by adding a nested list of strings or dashboards. And now let's go to chapter 2. This has been influenced by me learning martial arts for the last few years and I talk about what I call judo. Well, it's called... That means literally gentle or flexible way. Judos and martial arts, that it's basically based on two principles. The one, be flexible, of course. It's all about flexibility. And the second one is about taking advantage of your opponent's strength. As I'm kind of tired of showing your slides, let's do a quick demo to show you what I mean. Here's one we're talking about flexibility. As I just show, all the dashboards can be stacked and merged into each other in order to complete something bigger that way, intended originally. Here you can see how I built dashboards in order to map some data, actually an arbitrary data frame, or a pandas panel or even a panel 4D, to visual data in the form of a scattered plot. It's basically... you have different dashboards, all combined. Every color means different dashboards that have been stacked into each other. That allows you for selecting almost any visual property of a boke plot. Now we'll see how it works. You basically have here your dashboard with your boke plot and you can choose any visual property you like. For example, now we're... This is the well-known IG data set and we want to do something new with this plot. Well, I think it can be done anything new because everyone knows it, but I'll try to have some fun with it. You can, of course, tune every default parameter interactively, like here the size of the marker type or even the alpha colors if you want to check for overlapping in your data. But, of course, you also can define mappings based on data. In this case, we are using a seborn color palette in order to distinguish among the three different species of iris. You just have to select with your color mapping picker and then apply your mapping. And voilà! You can also map, scale, and apply any function before converting your data into visual properties. So it's actually really flexible. It even allows you to do something like, okay, I would like to just plot two types of iris, color them with color gradient and the species that has not been colored, I want it to be purple and to map their size between 20 and 50 according to its petal width. I don't know, you can do really weird plots with that. You just have to select what kind of data you want to map and the one that is not mapping will get the default value. You can actually rank your data and scale it. You can actually do anything you like. And of course, you still have automated tooltip information. You can choose which columns you want to plot in your bokeh chart and it works just like this. So, now the recap. You can use Shaolin to make dashboards and stack dashboards with other dashboards so you can get real weird and cool stuff. All the dashboards I programmed can be instantiated with some kind of data but you can change the data it is mapping on the fly so you don't have to worry about getting any weird crash or trailer errors or that stuff. Well, now it's time to get serious because it's time to be a hacker. Yeah. What does it mean to be a hacker? First of all, a hacker should be able to find bugs and know the limitation of its own world. So, I talk about why Shaolin meant for. First of all, if you are trying to please your boss and get some nice dashboard that will make your investors really happy in a short time, please don't use it. Take Caravell or take something flash-based and go do your business easily. This is an oratory at the framework meant for science. And in the other hand, if you are just writing some quick function and you want to add quick interactivity without worrying about making complicated layouts or scaling your functions or your models, you can just actually interact. It's really great, it works nice, and everything that's being built in this demo has also been built using just the IPI Widget Package, no Shaolin, and it's possible to do that. It will take you about 50% more lines of code, but there's actually no problem with that. But if you want to make something able to snowball, I mean snowball by starting to make some small dashboards and making it grow in order to stack them to make some more complicated plots, you should be using Shaolin. Interact is meant for functions. Shaolin is actually meant for Python, object-oriented programming. What's next? Information should be free. Of course, Shaolin is free software. You can do with it anything you want. Of course, the final goal of this software would be to make the Jupyter Notebook something like the ultimate science-sharing platform. If you collaborate with me, it would be really great to map all the PyData libraries so we could use them without having to know Python. So if you do anything, just show me the code and I'll be glad to interact with you. Anything you want to find about it, you can find it at the github repository of our company. And finally, if you want to be a hacker, you should try to use stuff differently than it was intended when it was made. Let's see what I mean with that. Now, it's time to feel like a hacker. Here you can find a dashboard that I coded a while ago, mainly as a joke. Of course, you can see that it is possible to fully customize any CSS of your dashboards and make it look actually like pretty anything you want. And, well, it also includes premade dashboards for wrangling with IPs and ports. And, well, if you know Spanish, you can see that this is actually a joke that I made. I was trying to implement the proof of concept of some port scanning that worked while loading malicious HTML code into images in order to perform port scanning fast firewalls. It was really complex and it took me a lot of bad written scripts to do it, so I tried to embed everything into a dashboard. And once I discovered that the malicious HTML code could be loaded from the same HTML widgets where you need to display your outputs, the Jupyter Notebook became something like a port scanner. So, well, you see here that you can play and do a lot of stuff with it. Here you can see the snowball effect where by just defining three new widgets, I can also add everything that I had programmed before in order to get beautiful outputs and customizable plots. Here, well, I'm just testing this, starting my scans and sending all my phone requests in order to get information about the delays of my forged request from my malicious HTML code. Here, well, I've used the barplot from the formal example in order to make some customized port to match all the other CSS. This is basically done by mapping all the plotly interface with a dashboard and merging it with another one that is just about making a cathlinx plot with a custom-defined layout. Cathlinx is a library that binds the Pandas Data Library to plotly plots. So, if you want to do, I don't know, any weird plot and fully customize the same thing, you can choose actually any property you like. Let's continue. Be a hack and recap. Just remember that this is nerd-oriented software. Don't use it to play your investors. They will be actually disappointed. They don't want you to waste time coding every functionality of the layout in plotly. And, of course, free software, and it's mainly focused on trying to create a snowball effect when writing your data analysis tools. And now, well, let's get into our final chapter. Nothing more to add. Just teach and play. I call it teach because show the names to make teaching and learning science easier. I call it play because for doing RayScience, you should actually be having fun. So now, I have some fun by showing you how is it possible to turn an entire PiData talk into a dashboard. I honestly hate how this video. Here is my graph plot dashboard. This is kind of tribute to Miguel Bath i his talk about finance network mixed python, which was, I think, a bunch of months ago, like made in PiData, in which we talk about representing correlation matrices as graphs. Here, we have joined everything that we made in order to fit this plot with a time series data frame, calculate a lot of correlations and metrics, and combine everything together into a graph. Here, you can see this graph representing, like, seven time series of forex data. And every link in its, every edge in its graph represents a value for miscorrelation matrix. We're trying to represent all the information, like a snapshot of how the market behaves during a given period of time. So, first of all, we just define a mapping for any, like, for every of our time series, and now we are defining a mapping for our correlation matrix. As we are choosing to be a color palette of just five values, all of the correlation, all of the correlation values of our matrix are represented in five different bins, so you can, just with a glance, no know what's happening on the market. Here, actually, you can see that the two correlations that are higher, either one between AUSD and a billion dollars, and the lower ones are the bluer. I choose five bins because now the correlation that are almost zero gets indistinguishable, almost indistinguishable. Of course, you can choose this mapping to do anything you like, like coloring your different time series by the properties of the graph they are defining. For example, here you can color them by its betweenness, its betweenness and trality, which is a measure of how this graph is structured internally. So, we define a new color mapping, more pleasing, but the greener it is, the more bigger it is between its centrality, and the more brown it is, the lower it is. It's between its centrality. Here, we have also mapped almost all the capabilities of the network software libraries, so you can have different layouts, change how your graph is filtered. In this case, we have filtered the correlation matrix, analgúering, which is called minimum spanning tree, and you can also use another algorithm that's called planetary maximum filter graph. Just in case you have a really big correlation matrix, you can filter the values in order to get only the bigger or the lower ones. If you want to get the bigger, just do it, and if you want to get the lower ones, just click on invert distance and you get your matrix filtered by its most relevant values of negative correlation. You can also choose any layout you like, which is implemented in the network's library, and, well, it's actually kind of fun to play with this. You should see here how it's displayed. You can filter not only by your correlation matrix, but also you have any other graph, which is structurally corresponding to an adjacency matrix. It can be also used to plot your graphs. So, if you get Miguel Vaz, video of his talk in PyData, and you get just another screen with this open side-by-side, you will be able to follow everything he's saying in the talk without having to worry about much about how is it going, and all the examples that he's showing, they are actually really complex to understand if you are just coding it. Oh, okay. Now, before summing up, I will show you another video I have here, as we would have thought, no? Perfect. Also, I'll show you how you can style any dashboard you like just by using its state manager attribute. Here, you will automatically load a bunch of widgets that are in charge of defining how your CSS layout will be. It has all the features that the IPI Widgets Library has, and it allows you to set all the CSS units by just using widgets. So, we can see here, in this example, is it really easy to put the customize all the dashboards you are making just by using widgets. If you need to change the width inside or something, this can make it actually really easy to do it. So, well, finally, I'm just saying just one more thing about having fun and all, or not more than a minute. Finally, I would just say try to be accessible. Here, to try to prove myself wrong, to get some feedback, and I don't know, listen for your comments if you find some better way to do what I do. So, thank you very much for coming here. Thank you, everybody, for listening to my first talk, Eva. And, well, I hope you're asking questions, because there's still five more minutes to go. Thank you very much. Thank you very much, Guillem, for this amazing talk. And now the floor is open for...