 How do you approach utilizations? Since it's a short talk, I don't know how much time I will take to go into the course. But what we can do is, as I looked at one or two examples of how to generate graphs, mainly in Python, I'm going to use MaxRotlib to actually show that one of the different kinds of graphs that we can draw and how do you actually figure out what's going on in the data. So in my talk in the morning, for those of you who attended to me, actually can use a good for any of the big data systems to actually pull data out from and pull and aggregate data and get it to a level that we can visualize it. But there's so much information that is inherent in the data, inherent in the data. I think Anand is here. He gave an extra graph in the after-missile. I will go a little more deep into some of these things and especially in the emphasis on Python. So let's start. So we all know that data resolution, everybody, everybody we have, we are confronted with data. There's financial data, there's data from social networks. So the first step is struggling to make sense of the data. And though we have the tools, and though we have the tools, and for actually taking and analyzing the data, what we don't have is how do we actually find out more than just the magnitude of certain damage. We also probably want to look at the relationship in the data. And that is what visualization is going towards. It's a fairly interesting field, and it's kind of a picking up speed. And you can see some of the, I will go into some of the details like how we have to take out the data and start visualizing it. So this is the catalog data. This talks about, there are four different countries. India, Greece, Germany, and US. And this is carrying the month-on-month inflation. And this is representing a tableau format. So if you read this particular slide, maybe you can figure out if it has been in the month of April, inflation pattern is quite high in Greece. It's typically low, but there are certain months. For example, in March, and in September, it has increased a lot. So it still takes a lot of time for humans to actually look at the tableau format and make sense of the data. So what is probably better to do is probably to start working this data. This is an example of bad graph. But I've probably put one in so that we can find out what's the difference between a good visualization path. Probably I think the colors is not very good. But at least, even if you look from a trend perspective, you can figure out those two points kind of stand out in Greece. US has been fairly consistent. There has been a big jump here. Then if you look at, again, India, I mean, there was a peak and then the trend is upward again. So this was drawn using Excel, which is a fairly decent tool for actually doing visualization. A lot of people do not look Excel as a way of actually representing and analyzing and actually plotting data. But that is a fairly decent tool. So that should be used visualization. What are the different parameters along which visualization can make a good difference? I'm going to cover examples of each of these. But I mean, everybody who has seen DVDs and has grasped on this, people actually use the wrong charts all the time. They use pie charts when you don't need to have an explanation of composition. People use maybe line charts to sometimes show distribution. But what visualization should do is actually along this time because what are you trying to do with, what are you trying to show with that? Are you trying to show a comparison? Then maybe the best way to do it is maybe use a line chart and maybe not something like a pie chart. But we see that a lot of people do keep on mixing each of them. The other thing that you want to find out with data is what is the relationship between two sets of data. So if the visualization actually shows the relationship between two sets of data, then it becomes fairly easy to actually discern what's going on in the background rather than just looking at a complex text. Third is distribution, over a certain axis. Typically it's type or other certain other dimensions. You can actually use a histogram to show the distribution of data along a certain axis. And finally, composition. Example that composition could be extremely useful is we constantly as humans try to categorize stuff. For example, in the stock market, we have different kinds of stocks. We have tech stocks, we have FMCG stocks, we have maybe like the shipping sector, the industrial sector, so we have different. And under each of these, you have different companies. So if you want to find examples of the relationship between each of these companies and what sector they take, then maybe a composition based graph makes sense. So we'll go into an example in each of these. So this is like a line chart. So this was done using a very simple example. Very easy to draw. This is the code for it. So actually all the code is up on GitHub. You can actually, it's already uploaded, you can go to GitHub.com, you can go to GitHub.com, there's a message and then under the top, it's just closed and this top is completely updated. So this is the code that has generated it. I have given a range and then I have taken random functions and different kinds of random functions. And typically in Mac, you just have to import a pipeline under which there are a lot of libraries. You can go and check on the web that 2D libraries, the 3D libraries for which you can do interactive visualizations. So you take all of these parameters, maybe manage the data and then eventually just plot using each of these functions. There are the ways you can actually set the title, the labels, and there is a interactive mode as well. So currently, so it's a very simple example. You can actually plot this even in a, and the map one is actually just static blocks. But you can also embed it in Django and other Python related stuff to actually drop block. So this is an example of comparisons that you can use. Then if you want to see relationships, especially in data where it is not very apparent, maybe on two or three access, scatter blocks can not necessarily be 2D, but you can do this even. Now you have to use a random kind of generator to get a plot. But typically, if you, for example, find the, want to find a distance between, say, two people, what you can see is you can actually use, you can see that there will be clusters of data and some things that are easily correlated. So that cannot be easily seen when data is in a tableau format or a textual format. So when you want to actually explore the relationship between different things, then copy, or what simple way of doing charting that everybody, every library has is to use a scatter chart. One of the things is when you use PyLab, which is like an easy, interactive way to do, you can either use the same thing with chandrets or you can use the show function. You can see that I have commented on things that hold most of the show function because it brings up the, it brings up the data blocks. Same thing with distribution, I'll skip over this. And composition, which you find PyChart, you can use a standard for train composition. Typically, the example that I gave, like different types of industries and components, in each of these industries can easily show much capture. PyChart is useful when you want to see the relationship only on one axis. So it actually reduces the damage value. But it is very useful for saying the percentage composition that is there in each of these. So these are like the basics, composition, relationship, and all the other parameters that I talked about. This is how you compare some relationship, distribution, and composition. So these are the four different things that you look at when you're actually going to show any data. And then, like a very basic stuff that these are the charts that you can use. But what do you want to go beyond that, right? I mean, what do you want to, how do you show data that is your multidimensional? Is this 2D charts, in some sense, or two-dimensional, not just in terms of axis, but even in terms of other aspects, how do you bring out other aspects which show the relationship? So, typically use the S and Y axis. You can use the Z axis for plotting surfaces. For example, the function over on a surface over a period of time. You can see what is the maxima, the minima, very easy, especially if you're doing machine learning or any of these things, you want to see how the surface looks and what to do on gradient descent, for example. You can see how the surface looks. You can also use color. Color is a different setup. Typically, you don't use the whole palette. You can still use the whole palette, but typically what you see in most places is that you use the gradient to actually show the higher intensity for mean something and the lower intensity for mean something else. Signs of the elements also can be an indicator. I'll give an example of this further down the, further down the, and also the compositions, because you can actually show, like for example, using a dream app, how the elements are correlated to each other and how much of that part we, for example, if you want to figure out how much of the tech industry is, how much of the tech industry that we want to, which we want to use in the US, how Google wants to use the US, in terms of market cap. You can use the composition and use a box and then on top of it, you can actually use the different elements. And finally, you can use animations. I don't have examples of animations in any of my slides, but yesterday I found out there's an extremely good library called Chaco, C-H-A-C-O, which can be used for visualization, track visualization. So on the web, I have come across Infobis and Protobis, now I think Infobis is for D3. It's an excellent library for doing data visualization on the web. We actually used a string graph from D3 to do something during the hacking of the app. So the library is very good, especially if you want to look at hierarchical data sets, or you want to look at graph structures, that is very good. If you want to look at relationships and networks, then for visualization, there's JT, but I won't go into that right now because I'm talking about Python, what's available. Stream graphs. So stream graphs is a concept that is based on the work that was done by Leigh Byron. So one of the axis, the base axis is typically type. And what this graph could do, is what is the intensity that can be used here. The other thing that is typically used here is the composition, right? So if you look at it closely, it's actually a stack graph. And you can see the stack. So what can something like this represent? Now you look at a listening history, or someone. Last but not the best, I don't know how many of you use last but not the best. Last but not the best has a streaming graph. You can look at your own listening history. And depending on either genre, like maybe rock or pop or world music, you can see what kind of genres you're missing, but how did they, what are you missing, what did you miss in over a period of time? So that is something that you can use. Other thing that can be used, that can be used very successfully by the New York Times is, you can see the movie collections and what is the genre of the movie. You can actually plot it over a period of time. And each of these can actually represent the movie. So streaming graph is extremely useful when you're looking at a period of data and then you have different components of it. Color line, different types of color, and composition can be shown by the standard kind of characteristics there. This is quite an intent diagram. So in machine learning, typically what you do is you have two matrices. And typically they're positive and negative elements. And they kind of vary in time. So this is like a 20 by 20 matrix. And the size, the size of the individual elements, tells us what is the magnitude of that element. And black indicates negative and white indicates positive. So you can actually visualize the whole matrix and see if that clusters or how does the space kind of pack out. So it's very easy to see from a graph like this. But what we can actually show you the, and these are the elements of a tree have a certain weight or could have multiple weights. And you can show the composition by what parent it belongs to. And also you can show it by the magnitude you can show by the color weight. So it's actually a tree that's represented, unfortunately, for each of the graph it is not showing up. Speed match is another thing that, which is very similar to some of the other things that you have seen before. It's like we can actually have multiple axes. And on two axes you can show different kinds of data. I mean you can use composition as well as the gradient in this. Example where we have been using this, I want to take more pictures. So for example, on one side we have set up publishers. On the other side we have the category that they belong to. And we can actually, the color can actually show how much revenue we made from it, how many clicks we got, how many add books we got. So we can have a three key to match. Side by side we can see what was the ratio which gave. And what are the areas that are kind of hot? I mean, are giving us more revenue or less revenue or more number of clicks based on certain, for other metrics, based on certain parameters that we have. So this also can be done using macro click. The old is also fairly easy for doing something like this. So it's kind of built out of the box. This is a polar plot, typically used in scientific computing but it's not necessarily, you can actually use the circular to actually show the revenue over that as well as. So you can use space as one of the dimensions and you can also use this, the radial distance as one of the intensities. Again, you can use gradient arrows. We are showing like, we have four different parameters on the same, it's easily possible to do multi dimension analysis doing something like a flow level. Finally, I'm coming to contour maps. You again want to plot a function. This one looks like a sombrero. And this is one we are showing it. It can also be shown in a pudding manner. There are ways in which you can interact with it also so that you can see from the top, how it looks from the side, how it looks. You have maps like this. So you can see on one side, you're seeing the characteristics of this, which is the distribution, on the other side, you're seeing the histogram. Seeing that how are they concentrated on two different axes. So you can use projections or you can use something like a surface also to look at the city side of things. We can actually look at three or four dimensions. Again, you can see the gradient is used to show the intensity, which anyway is shown, but it kind of accentuates and emphasizes based on the color. That's it. Unfortunately, I can't show much more, but you can, most of the code is kind of self-explanatory and not only is extremely good for being static plots. It also has very good integration. You can go to Macroflip. There's a whole gallery of visualizations that you can do using Macroflip.