 And so today what we're going to focus on is visualization. So we are starting from the assumption that you guys already do know how to do the basics of bringing a plot up. So I'm not going to cover the very, very raw basics of bringing a plot up. Did everyone get the announcement from BSpace about what you had to download? So the URL is right there. For those of you who are already nicely using Git, you can grab it. You can clone it from Git. And that way you can get real-time updates. Those of you who are not using Git, you can download this just by clicking on the Download button. And then it'll give you a zip file where you can grab the current state of the repository. And what we have here is a bunch of notebooks. And we're going to go through several of them today. There's probably more material here than we can reasonably cover in three hours, but we'll do our best. And the rest of it will be reference material for you guys. So hopefully this will work for everyone. And if not, let me restart my notebooks. So here I have a terminal, which is in a checkout of that directory. And what I am going to run is the same thing that all of you are going to run, which is to type ipython notebook. That's all you need to type. And that should bring up a web browser. It may take a couple of seconds to come up. In my case, it shouldn't be too much, because I had it open already. So when you type ipython notebook, you should get a message like this, and it tries to open a browser. It does its best to open a browser. I hope that most of you will follow along and type with me. All of these documents that are up there are meant for you guys to work along. If it doesn't work at all for you, you can view them and Paul was going to put up static PDF copies of them that you could use to copy and paste from if this doesn't work at all for you. But ideally, hopefully throughout the semester, you will have your setup so that this is as smooth as it is there, or you can just type this and bring up a browser. So the browser that has to come up has to be either, we do our best to detect, but if it doesn't work completely, I want to point this out, it has to be either Chrome or Safari, or Firefox 7 or newer. So any of those three will work. Internet Explorer is a no-go, doesn't work at all. And Opera has problems, and Firefox older than 7 can be made to work, but it's a little cludgy. So is there anyone in the room for whom one of those is not enough, who has none of those? No Safari, no Chrome, and no Firefox newer than 7. 7 or newer. OK, so we seem to be covered browser-wise. And if you guys all have the end thought distribution installed, the current version, you should see something like this. And so before we dig, let me close the ones that I had opened beforehand, so that we start fresh. Oops, that I didn't mean to close. That was documentation. OK, so we're going to start by looking at this one, the structure D-types introduction. This isn't strictly visualization, but it's a little bit of an explanation that we need to see for everyone to understand what we're going to do later. And it's a piece of numpy that may have gone by a little too fast during the boot camp or the introduction you've had so far. So it's important to at least review the basics. Is the font size big enough for the people in the back, or do I need to crank it up a little bit? OK, so we'll leave it that way so that we can fit a little bit more on the screen. And I'm going to full screen my browser so that I can see everything. So this is a very quick introduction to D-types, to what are called structured D-types. So a numpy array can be thought of as a linear chunk of memory with boxes in it. Hey, Paul. Did you put the static PDFs up somewhere? Oh, you did. There's a zip on the resources. On the resources on B-space? OK. So if anybody doesn't have, can't bring these things up to work for you, on B-space, Paul, just uploaded PDFs, these same things, but on PDFs, so you can read and you can copy and paste into a normal IPython session if this doesn't work for you. So a numpy array can be thought of as a linear chunk of memory where in each box you have an element. And let's say that there are six of these. And then there's a little bit of metadata somewhere that says, think of these six numbers as a 3 by 2 array, for example. So think of these as having two rows of three elements each. So there's a part of the array that tells numpy how to interpret the geometry of this. But very importantly, there's a piece that tells it what's on the inside. What does each cell contain? Does it have an integer? Does it have a floating point number? So the basic arrays that you've seen so far have regular numbers, regular scalars and integers, floating point number, complex numbers. But it turns out that numpy allows you to specify the type of what goes in there. So you can actually say, look, in here every single one of these elements is actually itself something that has structure. And I have a floating point number here, another floating point number here, and I actually have a vector of two floating point numbers, for example. So you can have a 3 by 2 array, but where every element is actually has internal structure. Why would you want to do that? Well, imagine, for example, you're doing a fluid simulation, and you're trying to represent, and let's keep it 2D just for simplicity of shape, you're trying to represent the pressure, the velocity, and the temperature of a fluid in a 2D flow. So you're going to discretize your domain as a grid, but at each point in here, at each point which represents the physical space, this thing has pressure, temperature, and velocity. Pressure, temperature, or velocity would be described by this kind of structure, a floating point number, a floating point number, and velocities in the x and y directions. So it would be nice if you could represent in your array, if you could say, my array has these many x by these many y grid points, and at each point I have pressure, temperature, and velocity. And numpy allows you to do that, precisely that. So what we're doing here is we're starting pylab support so that iPython imports, preloads, numpy, and matplotlib so that we can do arrays and plotting automatically. So we're going to type this. And once we switch gears into matplotlib, I'll explain what that inline word means. For now you can just type that. How do you execute cells in this notebook? You type shift enter. If you simply type enter, each of these cells, each of these things right here, is effectively like a mini script where you can keep on adding text. When you want to run all of it, you type shift enter and it executes the entire cell. So this is how we specify that we want an array whose internal structure consists of pressure, temperature, and velocity. We make an object called the numpy dtype. A dtype is a specific type of Python object that is used by numpy precisely for the purposes of describing the memory layout of data. So you say, I have a dtype which contains the names p, t, and v, and the formats float, float, and a two element float array. So this is the syntax. Now there are several ways of creating dtypes. The syntax is actually very flexible. I am not going to go over all of them. Right here there's a link to the documentation on how, let me open this in a new tab, the numpy documentation on how to create these dtypes. There's multiple syntaxes. It's fairly elaborate. We could spend two hours on this alone. All I'm going to do is show you this much. So once you create this dtype that describes exactly the example I gave here, now you can say, give me a 3 by 3. Let's imagine our grid is only 3 by 3 for simplicity. Give me a 3 by 3 grid of that dtype. And now you can access, for example, you can say, the pressure, I'm going to initialize the way you access these is like a dictionary. Square brackets and the name of the field, we said there was a field p as a string. So I want my pressure to be Gaussian distributed numbers with a mean of 10 and standard deviation of 3. So you're going to initialize your simulation with random pressures around a mean of 10 millibars, say, and this sets all of the pressure field in all of the cells. Then you can say, you can ask for the pressure, and that's exactly a 3 by 3 array. You can then initialize the temperature to be at 300 Kelvin, for example. And then you can initialize, how do you handle these that have more complicated structure? How do you say the velocities? I want them to have a mean of 5 as standard deviation of 2. But now, what kind of array do I have to create? Because this is a 3 by 3 array, but in each cell, the velocity itself is a 2 element array. So the shape of the velocity has to be 3, 3, 2, because I have to have the 2 element arrays, the vx and vy components of the velocity at each point. So now if I print the velocity, this is the velocity. There's a 2 component vector at each of the 9 locations in my 3 by 3 grid. So there, you've been able to describe the fluid array. If you now type what is fluid, oops, I haven't run this code. I'm sorry. Let me just run the whole thing. So the fluid array is that. The entire array, obviously, it's kind of complicated to look at it, because you're looking at all of the data for that. But here, you've seen what each individual cell has. So you can see how powerful this idea is. Imagine that Josh does a lot of this. They get a lot of data from instruments, cameras that actually pump binary data raw on the wire. You have an instrument that is doing a data collection. And all you know is that it pumps data with a certain layout and memory, but that data might have more structure than simply be raw numbers. You might want to group it in this way. All you have to do is tell NumPy what the layout of that data is in memory. And then you can absorb a raw binary chunk of data from an instrument from a data file, and represent it as a structured array. So this is very, very useful. You'll have many chances to use this, and we're going to have an exercise on this very shortly. Are there any questions on this? OK. So this is. You're talking about the distinction between this and the rack arrays? Yes. So you can, because it's slightly inconvenient to type this square brackets v, NumPy allows you. So let's do fluid R be fluid. And you can view this as what's called a rack array. What is the difference between the normal one and the rack array? The only important difference is that the, whoops, in fluid, if you want to access v, the velocity, you have to do square brackets and quotes. In fluid R, you can do simply dot. Well, let me access the pressure because it's shorter so that you see the printout. So this is what happens. Fluid p doesn't, fluid dot p doesn't work. You have to access it as square brackets and quotes. Fluid R dot p does work and gives you the whole sub array in one shot. Now there is one slight catch where there's a couple of caveats with these. First, normal NumPy arrays already have a bunch of dot names after them. It's actually quite a long list of things that they have. So you can't have a name in your names that is any of the names of things that already exist in NumPy because you'll have a conflict. And NumPy will win. So your name will basically not be available. Now, you can always still access with the square bracket syntax. You're not losing that. So this is only a convenience. The other caveat is that when you access, if you access one element of your array, for example, this is one element of the array, individual elements don't have the dot syntax anymore. So even though the overall array has the dot syntax, individual elements don't. What's that? Fluid R. OK, sorry. Still the same error. Yes. Sure, sure. So even the fluid R, even though fluid R dot v has that fluid R, you can't access an individual element. Now, you can get from dot v the 0, 0 element. This you can do. But you have to get the entire fluid dot v and then access at 0, 0 element, which may be an inefficient way to go about it in a loop, for example. You're creating the whole view just to get one of its elements. So in my mind, all of these problems that I've just explained, we actually intend to fix them in NumPy. They're all solvable. But this is the current state of NumPy. So the record view, this, by the way, doesn't copy the data. You're simply creating another object that has slightly more convenient syntax. I personally use the record arrays very often, just because I prefer the dot and typing is just much more convenient. But you have to be aware of these couple caveats they have so that you don't go, why does that not work? Especially this one, the absence of this, the fact that this doesn't work, I think is really, the fact that this doesn't work really bothers me. Because I really think that should work. And it's kind of, Stefan, do you know why that is? Just we haven't bothered to do it? No? You get a void because it's the opaque container for a structured element. So you don't get, the problem is there's no such thing as a record array scalar, is the issue. Yes, question? No, two. Yeah. Output eight. No, it's displaying a three by three matrix, but writing it out. So it's the nine elements, the grid is three by three. The spatial grid is three by three. At each point, there are two elements, vx and vy, if you will. We specified the shape of v is two elements. It's right here. And so there is shape plus two because we have an extra internal degree of freedom. For those of you with a physics background, you can think of these as internal degrees of freedom. And this as the spatial degrees of freedom. This is how you would view it in classical physics parlance. So in here, what you have is these nine elements just laid out one after the other. If you count braces, you have the same number of braces on the outside as here plus an extra pair of braces because now instead of a single number at each location, you have a vector. And you can nest this to death. You can have a matrix. You can have within d types, within d types. You can go crazy to the point of making things completely incomprehensible. But for simple cases like this, it's a very handy data structure to have. And so what you get with this are effectively things like you can think of these like Excel spreadsheets with named fields, but with n dimensional. Instead of only being rows, you can have arbitrary numbers of dimensions. Any other questions? So this is one of the foundational data types in the NumPy, SciPy world. And it builds, as I said, on top of standard NumPy. So now, after this brief review, and there's a link in there, there is a link to the page in the documentation that explains all of the syntax, now we're going to switch into Matplotlib beyond the basics. So I'm assuming that you guys already know how to do the basic Matplotlib plotting, right? So now we're going to dig a little bit deeper into things that are not necessarily completely obvious in Matplotlib out of the gate. And we're going to try to understand things about Matplotlib. So the reason why I asked you not to initialize the notebook with dash dash pi lab was because I want to turn pi lab support on a notebook by notebook basis differently. Because I want to show you that there's two ways you can use Matplotlib, you can get figures when you're using the notebook. You can have the floating windows that you've had before, or you can have these inline figures that stay and they're kind of pasted into the document. And the inline figures are very nice because they stay with your document. The problem is that they're not interactive. You can't pan and zoom. If you want to get an inline figure centered around a particular point, you have to manually say, I want my x limits to be this and my y limits to be this manually. Whereas with the floating ones, you can just grab the mouse and zoom into the part of the data that you want to see. So eventually we will enable in IPython the ability to toggle back and forth. Right now we can't. We haven't finished that. It's a little bit tricky to get it to work right. We haven't had the time to do it. So right now it's a choice that you have to make at start-up. If you start the notebook with dash dash pi lab, you make the choice once and for all for every notebook in that session. If you don't do dash dash pi lab, then you have the freedom of turning that on for each notebook separately. And so if you simply say percent pi lab, yes, question. So the question is for the web test. The question is whether pi lab only did import num pi and sci pi things and matplotlib things or actually did something with the figures. The answer is it does both. Pi lab is actually doing two things. First it's doing a set of it imports all of num pi, it imports all of matplotlib, it imports piplot as plt and num pi as np. Those are the things it does. But in addition to that, it selects which matplotlib backhanded to use. The architecture of matplotlib uses the concept of backends, the library renders to a generic API and then a specific renderer generates actual images. And so we have to decide, do we render to an interactive window or do we render to an image in a buffer that we send to the browser? And that choice has to be made when initializing the library. And so the pi lab command does both, actually. So the question is, if you initialize with dash dash pi lab only and don't say inline, what happens is you get floating windows. Yes, you get all of pi lab but with floating windows. If you say pi lab inline, you get inline windows. And as I said, I have actually a patch that allows toggling back and forth. But I wrote it a while ago and I haven't updated it. And it's kind of bit rotted. So I just need an afternoon to sit down and clean it up. Because ideally what we want is to be able to go back and forth between the two. So for now, we are going to use, well, actually, let's do it in this mode for a second so that I can explain something. So I'm starting it in pi lab mode. And in here, it told me that it's using the QT back end. Right here, it says I've started with the QT back end, which is the one that happens to be my default, it may be different for you guys. And when I run the next cell, I get two floating windows. So this is what happens when you don't say inline, I'm sorry, you get floating windows. So first of all, before we go back to the text, I want to show, for those of you who are not at the bootcamp, very quickly, I want to show you the basic usage of these interactive windows so that you know how they work. So these are the MATLAB live interactive windows. And they have a few buttons. And depending on which operating system you're on, these buttons could be at the top or at the bottom. They're always the same. It doesn't matter. So the buttons allow you to do the following thing. First, let's concentrate on this one. This one allows you to, if you click on that, you go into pan and zoom mode. You get a little hand that allows you to simply pan around your data. They're useful. This one allows you to draw a rectangle and zoom into it. So you can draw rectangles and zoom into parts of your data. Once you've done a few of these operations that change the view of your data, then these three buttons begin to come important. These are inspired by the metaphor of a browser. In a browser, what do the arrows do? They take you back in the pages you've seen and forward in the pages you've seen. Well, think of each render of your data as a page. So these take you back. If I go back, I go back in my view history of my data. So if you've been zooming in, each of these will take you back one, and the forward button goes forward again until there's no more history. And home takes you to whatever that window started as. Whichever view of the data that window began as, that's what home does. So that's what these three buttons do. Zoom to rectangle. And now that you've seen that, let's return to the pan and zoom button. Because with the left mouse button and on max, I don't know what you have to do to get a right click. But with the normal button does pan, the right button does something interesting. It zooms, but keeping the point of origin fixed. So what it does is it allows you to zoom by keeping one position fixed, and it allows you to control the zoom rate on the x and y axis separately. So if you move the mouse right and left as you zoom, it only zooms the x axis. If you move the mouse up and down only, it only zooms the y axis. And if you move diagonally, it zooms both. And the ratio of zoom depends on the angle at which you move the mouse. It sounds funky, but once you get the hang of it, it's super handy. It's a great way of quickly zooming into your data. And because it keeps the point of origin fixed, it's a very easy way if you want to look at something you click there, and you start zooming, and you're basically going into that part of the data very quickly. And finally, this one allows you to save your images, so you can save a static shot of the image to file. You can save in multiple formats. You can save as PNG, PostScript. Probably for you, the most important formats will be PNGs for web display and PostScript or PDF for papers depending on which journal you're sending it to. So this is all I want to say about these. There's a couple of other buttons for fine adjustments. Let's not spend more time on them, because we have a lot of ground to cover. If there are any questions, I'll be happy to return. But these are the basics of using these floating windows. So they're very handy for interactive data exploration. And you can create static figures from them at any point. Now, even if you are using the interactive windows, you can in the notebook still get static snapshots of these figures. There's a command called display that allows you to display any figure. So MatBloodlib has a concept of getting a specific figure, get current figure. So MatBloodlib has a concept of getting the current figure. And if you say the GCF command gets the current figure. So if you say display and you get that figure, display GCF, it will paste that figure in here. In this case, it does it at a fairly high resolution. And if you have made more than one figure and you have specific variables holding the figure, you can actually display whichever one you want. Or you can say, oh, I wanted figure two. You can say figure two, which makes the figure two be the current figure and display GCF. And that will paste the second figure. So the fact that you're using the floating windows doesn't prevent you from having static images of your figures. Yes, Stefan? Question? Can you just try out again? Yes. So if instead of doing this, let me close them all. So if instead, if I had saved these figure handles, let's say that this is figure two, and I make explicitly a figure, so I get the exact same thing as before, but now I have two variables, F1 and F2, that represent my figures. I can just say F1 and display it, just as if it was a variable. So the Python notebook, if you have assigned the figure object to a variable, then you can simply hit the variable and just as if it was a number, it would print. If F1 was the value three, it would print three there. It prints the figure. What it does is it grabs the data for the figure and embeds it. So I just want to explain this because you have both options of using the rendering figures, and they both have their uses. So it's important that you know how to use both. So yes? We would actually get this to do WX after this pile up, and so this pile up, then that works on my Mac. But just pile up by itself. Oh, really? Yeah. I had to tell it the back engine. I had to tell it which back engine to use. So I had a space WX there, and that's what I needed to do. Okay. And that's because it'll, it defaults to not a floating one, but whatever is in your map of RSC. Yes, so the question was how to get the floating windows, and depending on how your map of lib is configured, just typing pile up might actually not default to the floating window one, depending on how you have your configuration. So you can explicitly ask by saying pile up WX to get a WX back end, or pile up QT. I would do QT because that's the one I use by default. On the Mac, there actually is a Mac. There is a native Mac back end called OSX. I can't type that. This is not a Mac. But those of you on Macs, you can type pile up OSX, and you will get the native Mac back end, which I think works very nicely. Okay. So what if you've been using, right now as I said, there's no way to toggle between the two at runtime. But you can toggle without having to completely kill your notebook. The reason is that you can restart a notebook kernel, and this will kill the Python process and open a fresh new one, which has no pile up yet. So if I restart my kernel, which obviously deletes all my variables and closes all my windows, then I can rerun it again. And now the next time I can do, I can make a different choice of pile up. So from now on, unless I need floating windows, I'm going to do inline just because it sees. So unless I'm demonstrating something specific to floating windows, I'm going to use inline windows, because they're a little bit easier for demonstration purposes. So the first example showed a very, very simple plot. And the reason why I wanted to look at this example after having explained the whole business with floating and inline windows is that this is how many of you probably so far have seen how to use matplotlib. You have some data in x and y. You make a plot. You give it a legend. You give it a title. You give it some labels. And you make all these calls using plt.whatever. You're not creating figures by hand. Maybe at most I made a figure here manually, but this would have worked actually even without that first call. And later, I can add a line to that plot. Basically, I keep making plot commands, and they just keep getting added. This API is very familiar to anyone coming from a matplot background. How many of you have used matplot in the past? OK, roughly half the class. So this is modeled straight up after matplot. So it's going to be very familiar for those of you who come from a matplot background. But effectively, what it uses is a very, very stateful entity that is keeping track of the concept of, I know what my active figure is. All the commands go to my active figure. If you want to do something in a different figure, you have to make that figure the active figure. It's not a very good way of writing robust code, because it means that things depend on the order in which you execute them a lot. If you make the active figure the wrong one, and you move this piece of code over here, now things go to the wrong figure. So matplotlib actually has a different way of building objects, which only slightly takes only a small amount more code than this. And I find to be my preferred form of working by and large, because it's much more maintainable and leads to cleaner code in the long run. So this is fine for quick and dirty stuff. You want to just open a data file, or you have a function that you want to plug quickly to look at it. This is perfectly OK. But the object-oriented API, now I'm going to do the exact same two plots with the other interface. So this object-oriented API, I make a figure and what's called an axis object. And instead of calling figure, I call a command called subplots. So the subplots command, let me open here again. That's just that you're proud of subplots. Yeah, OK, I usually give Fernando a hard time because you were in subplots. Because it was annoying to use beforehand. So what the subplots command does is it lets you create a grid of n by m subfigures, and it returns both the figure and what's called the axis. And why do you want the axis? Now we have to understand how MatBloodLib is structured a little bit. And that was the whole point of this lecture. I'm going to be a racer. If I sit on the parentheses for like 100 milliseconds, I think I don't remember what threshold is. For a little bit under a second, then eventually it pops up. In master, tab is broken, so I couldn't press tab. But yes, if you press tab, it will also come. Tab out of parentheses will bring up the call tip. Unfortunately, in master, tab is broken right now, so I can't use it. So in MatBloodLib, when you make a new figure, you get what's called a figure object. You get a figure object. And then, but the figure is not actually the thing that makes plots. The figure is just a container for plots. Plots are made by what are called axis objects. So MatBloodLib, if you want to plot, the figure itself doesn't know how to plot. It has no axes on its own. You have to add a set of axes. This is how the library was designed. And this, an axis object, this is the guy who knows how to make plots. An axis object has methods to make log plots, semi-log plots, error bar plots, logarithm, scatter plots, polar plots. And you can create axes of many different kinds. So if you want to make a grid of plots, say you want to make three plots, you can do this would have three axes objects. And so you could call this axis 1, axis 2, and axis 3. And with these variables, now each of these entities is capable of doing its plot. So x3.plot, blah, blah, blah, will plot something in here. So the top-level plt.plot command that we used before, plt.plot is shorthand for make me a figure, make me an axis, and call plot on that axis all in one shot. Very convenient for one of things, but not very fine control if you need to really specify where you want things to go. So the subplots command, what it does is it returns, you can have an n by m grid of plots. And it gives you both the figure container and each of the individual axes. And you can grab them one at a time, or you can grab all of them as a numpy array of objects so that you can then say axis 0, 3, for example, and call plot on that. And the reason you may want to do that is because if you're writing complicated code where you're creating a 4 by 4 panel for statistical analysis of plots, for example, and then you want to iterate, and now you're writing code basically algorithmically, where you're moving over a grid of plots to access pieces of your data, you don't want to be doing that by manually counting things. You want to be able to index that array of plots with actual variables. And what it will return is precisely the figure in the plots on the n side. So what is the difference between how you access plot and how you access, how you do things with the axis versus how you do them globally? So before we had done, let me delete this cell. So before we had done this, plot, legend, title, x label, y label, that's it. Now the syntax is a little bit different because now the syntax has to do those things like the titles and the labels are done with set commands. Why? Well, that's how the code was written. I probably would have written it slightly differently, but so be it. So now I've made, if I simply call subplots with no arguments, the default is a one by one grid so that it's as similar to just calling figure as before, you get the figure and you get a single axis. On that axis, you can call plot. You can call legend. And then if you want the same thing, title, you have to say set title. So you have to say set title harmonic, set x label, and set y label. Now here, if I make a new figure and a new plot, the advantage of this is that I now have a handle on both the first axis and the second axis. So ax is the first axis, ax1 is my second axis. So I can go back and plot cosine on my first figure even if this code is later in the code. Before, I had to be very careful up here in that I did everything for the first figure, and then I'm done. Because the moment I make a new figure, anything that I type here goes to the second figure. And there's a way of manually toggling back and forth, but it's a state machine. So it has a notion of who I'm sending these things to, and if you get it wrong, things end up in the wrong place. Here, you have explicit control. You have your first axis. You have your second axis. And you can tell later on, you can say, I want on my first axis also cosine. And when I run this code, I get this. I end up with my sine and my cosine up here. And I end up with, now, observe, the legend only shows sine here. And that's because I added this after I made my legend on my first axis. So if I want, and that's because when the legend was constructed, it the plot only had the sine line. So it doesn't mean that there is no sense of order in these things, but at least you can control where things go. Any questions? Paul. In defense of the set way, at least this way you get to know what are the things you can set on. True. That's completely opaque. True. And to let's illustrate that point, if you do set and have complete, at least you see, as Paul was saying, these are all the things that you can actually set. So it makes for a much more discoverable interface, because you type set underscore, and then you can kind of look at that list and figure out what it is that you need to do. Any other questions? So the subplots command is equivalent. If you find yourself on a system that is very old, you might not have the subplots command. You can make the figure manually. And then you can start calling add subplot with a funky syntax that is copied from MATLAB, which is an integer, or you can use comma, you can comma separate it too. But this syntax 111 means one plot total, first row, first column. So if you're going to make a 2 by 3 grid, you would say 6, 1, 1, 6, 1, 2, 6, 1, 3. So you see a lot of code written doing basically index arithmetic to get this mess right. And it's very easy to get it wrong. But that's the syntax. And you have to add them one by one. So under the hood, subplots is doing all that logic for you. So we get it right once, and then we reuse it. But it's important to know that that exists in case you need it. So this, as I said, is the simplest case. But you can actually very equally easily say, I need a 1 by 2 grid of plots. So now what comes back is going to have two axes. And you can immediately, if it's only a couple, you can name those axes as variables immediately by using this syntax. Python allows you to name things explicitly in this manner. So if you say subplots 1, 2, you know that the first thing you get is always the figure. And then you're going to get a 1 by 2 array. This is called tuple unpacking. You can unpack. And this is going to be the first element and the second element. And you name them right away, which is convenient when you only have two or three. Using the indexing array syntax is necessary if you have a complex grid. You're not going to do x1, x2, x3, x1, a bunch of variables. But if you only have one or two or maybe three, it's actually handy to have just standalone names for them. Choose whatever works best for you. Now, what about this share y argument? So for that, I need to switch gears again. I'm going to restart my kernel because I need to switch gears into. This is something that only makes sense in floating windows. So if I say share y equals true, what does that mean? So this is a simple plot. But this is one where I said 1 by 2, share y. What share y means is that the y axis of these two plots is actually shared. They're linked. And so the advantage of that is that if I decide to zoom in this part of my data, the other one also zooms. If I decide to pan in the horizontal direction, they're not linked. The x-axis are not linked. But if I pan in the vertical direction, they're linked. If I zoom in the vertical direction, the two axes are linked. So if you're analyzing data and you have quantities that are related to each other, you can decide maybe they're in different units. So the y-axis don't make sense to be shared. But they're a bunch of time series computed at the same time from the same phenomenon. You want the x-axis to be shared. So you zoom in 1. And all of your time series zoom together to the same spot. That's what you can do with share x and share y. Question? Could you do it? No. I don't know that there's an automatic transformation for rescaling, for rescaling and sort of rescale on the fly. So the question was, could you do this kind of axis sharing? But if there is something like a unit transformation or a relationship between the two, and so you want to inject a transform, as you do the changing, I believe the answer is no. Stefan? So you can change the way the ticks are related. And then, so you have the same numeric scale of what's involved of the x-axis for the y-axis. True. So you can add dates on the one. Good point. Yes, that's a trick. So the answer that Stefan gave was, you could have two different axes that share, say, the x-axis. But where each one of them has a different tick format. It turns out that in that blood lib, the things that put these ticks, most of the time you just let them be, they come out whatever it is based on your data. But you actually have very fine control over who puts ticks in here. And there are examples in the map of the gallery about how to do that. So you could specify two axes that are sharing, but that are using different tick formators. And the tick formators apply the transformation that you need. I've never actually tried it, but that's a good idea. I mean, effectively what you have to do is transform your data in one plot to match the range of the other one. And then you let them share, but then you show some sort of transformation on that data, so you undo that transformation, that's fine for you. Yeah, yeah, but it works. For your final project, you could submit the math plot with a project. Support for that. Yeah, often people will do, for example, lock plots in that way. Instead of transforming the data, they do a normal plot, but they only change their tick. So it works. Question? If you want to unpack it manually, so let's do a couple of examples. Let me show you. So the question was, what would the axis tuple for a 2 by 2 look like? So let me first not do the tuple, first to show you what it is. If I simply say sub plots, and I say 2 by 2, and I don't unpack, I just grab ax, I want to show you what it is, first of all. So I get a 2 by 2, and so ax is what I get is an array. I get a 2 by 2 array of axes objects. So I can grab ax, if I ask for what is its shape. It's a numpy 2 by 2 array. I can access ax 0 comma 1. And I can say plot 100 random numbers on that one. And if we look at the figure, oh, this is important. Why didn't anything happen? So when you're doing floating windows, if you call a method on it, it doesn't update the figure right away. In this case, you have to manually say draw. Why? The reason is because the object-oriented interface is meant to accumulate commands and not render them until you say so for performance reasons. Because otherwise, the actual action of drawing is expensive. And so if you're building some very complicated figure in a loop where you're doing lots and lots of things, you don't want to wait on each little sub-command for MatLovLib to re-render the whole figure. It would take forever. So when you're using the low-level interface, MatLovLib will not draw until you tell it to. If you're using the notebook inline plots, what happens is we have to draw to send the figure. So it seems to work the same. But when you're using the floating windows, you see this fact that you have to manually tell it draw before drawing actually happens. Actually, yeah, I don't know how to control that. I think draw is everything in the current figure. Paul, do you know if it's any different? So the question was, is it possible to control the scope of the draw so that the draw only applies to a specific? Well, we can ask. OK, here. We ask, whenever you have a question like this in Python, you ask Python, show me the code. And so you type draw double question mark, and you see the code. This is the code for draw. And it turns out that it actually does get the whole canvas. So it applies draw on the entire canvas. There is no such thing as partial draws, it seems like. Maybe there's some low-level. With these questions, if you actually talk to John or one of those guys, there's always some low-level, weird way of doing what you have in mind. It appears for an updated cell every time, and only update a specific part of it. True. Yeah, it's kind of a different route. So show, the difference is draw is meant to update a figure. Show is a harder command, a more expensive command that is meant to also bring windows to the forefront and possibly create windows if they didn't exist. So for example, show is the command that you put at the end of a script. If you write a standalone script that is meant not to run an IPython, but just have the command line as a normal script, at the very end, if you don't put a show at the very end, all the figures that you created never will pop up as windows. So show basically does drawing, but does even more than drawing. It does window management. Draw doesn't do window management. I already had created the window was already there. All I was doing was updating the data in one of the panels. So this was a roundabout way to answer your question, because first I wanted to show you what you got in this case. Now the question is, can I grab these guys all by themselves? I believe you can, but I'm not sure. I think you can do this. Yes, you can. So the question was, how can I unpack these four acts? Well, it's 2 by 2. So if you write a nested structure which has a 2 by 2 nesting, then Python will correctly unpack those four elements in a 2 by 2 unpacking. At that point, I probably would just use acts and index with indices. This is probably the limit. I tend to use the unpacking when I have two or three. True. If you do only this, because a 2 by 2 array is a 2 array, has rows first. If you do this, this also works. And in this case, acts1 is the first row of arrays of plots, and acts2 is the second row of subplots. This is just how Python works. This is nothing specific to Math.lib. Now if you have a really obnoxiously complicated case, let me restart my kernel so that I'm back to plotting. If you have an obnoxiously complicated layout need, you can use subplot to grid, which is a more complicated command. I won't go into the full details of it. But this is a command, this is a little example that allows you to do things like this. You cannot do this with subplots. Subplots is only m by n. That's it. Rows and columns, as many as you want, but that's it. This kind of overlapping grids, you have to use the subplot to grid. And there is something even worse called the axis grid toolkit, which allows you to do really, really exotic and complex grid layouts with plots. So with Mathplotlib, the answer to how do I do something in Mathplotlib is very rarely you can't. It's mostly a matter of how much time are you willing to put into it. But there are tools to do really, really esoteric things with it. I don't actually know how to use the axis grid toolkit. I've never needed to do something as exotic as what it does. But while we're on the subject, so the one thing that you all, and I showed this at the bootcamp, but I'm going to briefly show it again for the benefit of those of you who are not at the bootcamp. The first thing you do when you're trying to do something in Mathplotlib is you go to the gallery. The gallery is a page that is full of thumbnails of all kinds of stuff in Mathplotlib. And you browse through the gallery. And you visually scan the thing that looks closest to whatever it is that you're trying to do. And you click on that. So if you're trying to do something with, for example, contours and images and color bars, OK, that looks like a good starting point. So you click on that. And then what you get is a bigger version of the figure. You can actually get high resolution versions of it. But more importantly, you get all of the source code that does that. And so, but you don't even have to copy and paste. You can copy the link of the source code. And in your notebook, let's go back to where I was, in your notebook, you can type an IPython command called loadpy. And if you give it the URL of that path, IPython will fetch the script from the internet and put it in the Excel so that you can just go ahead and run it. So it's a very easy way to find something that you need, grab it, view it, see that it does the same thing. So indeed, this does the same thing that the gallery was showing. OK, yeah, it's pretty much the same figure, more or less. And so now you can begin changing it and tweaking it until you get what you need. So loadpy is very useful. The loadpy command, what it does, is it grabs either a local file, you can give it a path on your disk or a URL, fetches the data, and puts it into the next cell so that you can begin playing with it. OK, so this is about as much as I'm going to show you about subplot to grade and complicated layouts. Beyond here, go to the gallery. I mean, at this point, you go to the gallery and you look for the examples. The gallery is an unstructured view of the entire set of examples. If you want them categorized, the same examples are here sorted by topic. So you can also see them sorted by topic and by name. And sometimes this may be an easier way to find the one you're after than visually. But Matplotlib has lots of examples. Unfortunately, they're not all as well commented as they should. But it does have a lot of examples. And the other thing you recognize is that some of them are kind of the staple way, where they're just trying to show you how you disorient, and they're not saying big one, and they're not saying an axis. So I actually find the gallery is pretty confusing for learning how to make your own figures in Matplotlib. They're really just to show you how to do that one sort of thing. And so the point that Josh was raising for the webcast is that the gallery uses a lot of very stateful PLT top level commands, which is not really the best way to work when you're trying to build a complicated plot in for a paper or something more long term. And I think that the gallery is just in big need of a big cleanup. There's also a fair amount of redundancy. It has kind of overgrown over the years. And so there's a fair amount of redundancy. It's still very useful. And obviously, it's better than nothing. But there is a fair amount of redundancy there. And I think it could use with some trimming and fewer examples, but examples that were better explained, that were kind of fewer of them with more explanation and detail, would go a long way. There's been a little bit of whoever sends an example goes in. And then over the years, you just end up with a fair amount of, and some of them unfortunately have even no comments at all. So OK, there's a bunch of code, not a single comment, and a figure. Well, that's actually not always the most useful thing. But still, even with these caveats, it is a very useful resource. And I still do use it a lot. OK, so how are we doing on time, Paul? We have this, we're going to finish this one. OK, no, no, but we do want to do that. And I'm not doing all of this notebook, because this had some old materials that was redundant with yours. I ported it, so I would have it. So how do you control now properties? So for example, if you want, many common properties can be set directly in your normal call. So for example, if you want a dashed red line, you say line style equals dash dash, color equals r. Those of you who come from MATLAB again will find a syntax familiar. The plot documentation explains what all of the styles and colors are, and it works this way. But if you grab the line object, just as much as we saw how the axis object had all of these set x label and set y label and so on and so forth, the line objects that you get also. So in here, when we plotted this, the output was this. Was a list with a single matlotlib.lines line2d object. Well, that line object is the object that is this red line right here. So you can actually get a hold of the line object and later manipulate it. So if I do, if I grab the line object, so this syntax simply means get the list and unpack it. It's a single element list, so I unpack it in one shot. And then I type tab, and here I've just copied and pasted so you would see all of the tab completion in one shot. These are all the things that you can set about a line. So you can set its colors. You can set the data that it represents. You can set many, many, many things on it. I don't know what many of them are, but some of them are fairly obvious and fairly useful, such as color. The set data is very useful because this is the actual array of numbers that the line has. So if you need to update a plot without redrawing the whole plot, as Josh was asking earlier, what you do is what Paul pointed out. You don't try to focus the draw operation, which is global. What you do is you grab the object and you reset the data on that object. And so every time you draw, only that object will have changed. And so set data is one of these properties. You can set the dashing. So the only reason why I want to tell you this is so that you know this way of working exists. And when you need to do something with fine detail, you can begin to manipulate it. And finally, MatLive also has a command called setP. I want to mention it in case you see it in tutorials. I typically just operate in this way. But you can also say set property and on the line. You can call line style. The advantage of setP is that if you simply call it, it prints back what the possible values are. So here, you have to look at the doc string, whereas here, it will tell you, oh, these are all the things you can set. So if you say setP on line, it will print. These are all the things you can set, and these are all the values you can use. So basically, it's a way of discovering, OK, I need to change this. What can I change on this thing? SetP on that object, and I'll print out, OK, these are all the things you can change here. And setP can manipulate multiple lines at a time. So for example, you want to change the thickness of every line in your plot all in one shot. So this is an example where I've made a plot with two different lines. I made x1, y1, and x2, y2. So one of them is sine of x, another one is sine of 2x. And then if I later simply say setP on lines, and I change the line width and the color, then this automatically applies the transformation to all of the lines in one single shot. So this is a little bit of the machinery. And you can call just like line has, as we saw, a bunch of things you can set, it also has a generic just set, which allows you to set basically all of those in a single call. So set line width, and color, and line style, and so on and so forth, you can do all those in a single shot. So as you can see, you have very fine control. It's not just that the top-level call has to be the end of what you do. So in simple plots, this is what matplotlib will return, line objects. And line objects, as we saw, you can manipulate. Oh, and here I had an example. So here's a good example of changing the data on a line. So now once you change the data, if you replot, so here what we're doing is we're making a plot, and initially the plot had only 1, 2, and 3. But later I'm changing the range to be from 0 to 1. And if I reset the data to be now this data from 0 to 1, the original range will not be correct. So now I have to manually tell it to set the limits on the axis. And we can do that manually, or we can grab the access object and ask it to do it automatically. And then as we said before, we have to do the drawing manually. So this is the kind of thing that you end up having to do if you're manipulating the structure of your plot by hand, which does end up being necessary when you're doing things like animation, for example. Then you have to basically now get a little bit dirtier, because all of the automatic machinery that tries to guess for you is not available anymore. Now the other component, as I said before, is the axis, which we saw here. And then finally, you have the figure object, which is the container of all of them. And the figure, these things are intertwined. So the figure object has a member called axes, which is the actual list of all the axes that it has. So if we make, for example, if I make a figure and I print what are its axes, I get an empty list. So the figure always has the list of axes that it contains if I do f.ax equals subplots 2 by 2. Here, I get my figure with 2 by 2 plots, but the figure axis is just the naked list of the actual axes instances. So you can always get these things. We actually debated when we made subplots, whether to only return the figure, since you can get the axes. But because it's a little bit awkward, and because of the desire to have 2D indexing, we decided to actually return both. And similarly, yes? Because it was written as a list. Basically, when they first wrote map.lib, the axis was made a list that just gets new axes appended. So also because you don't have to use, as we saw in the axes grid example, they don't necessarily have to be included. True. So in general, you can add an axis anywhere. True. Yeah, that's actually the right answer. So the question was for the webcast, why is this not an array but a list? And really, the answer is what Paul said. You can have a figure, and you can have an axis here, an axis here, an axis here that overlaps if you want. You can have axes that have an arbitrary geometry. When you create new axes, you can tell the coordinates of the axes wherever you want. So it doesn't necessarily fit a model of a 2-dimensional index. For subplots, it has to because we made subplots be m by n. It's restricted to that case. Yes, but internally, dot axes is the internal map.lib storage, which has to accommodate all cases. You could append to this list another axis that you create by some other means. After the fact. And then redraw. And then those axes know where they want to be thrown. We can put an axis on top here in the middle of these four, and that will be perfectly happy doing that, which is in fact how you do inset plots, which is a very common thing. You have something. You're drawing some data here, and you want an inset plot here. You have a big axis, and then you put a little axis here. And the inset plots show some other property. That's perfectly OK. So one of the things to recognize is that you can delete axes just as if you're popping items out of a list. You can also remove lines, because lines that are part of axes are just objects, and you can get rid of those. Another thing which you'll need for the homework is to know something called set visible. So you can actually set the lines to be visible or not. So you don't have to actually kill all the data and actually kill the line. Like if you wanted to show something on and off, you could actually just keep toggling the set visible. So this is an example of set visible. So in here, we made this funky grid and the call to subplot to grid. But we don't want tick marks on all of these things. Imagine how ugly that would look if there were tick numbers everywhere. So what we did is we made this complicate, this call to subplot to grid to create the whole layout. And then we walk. So this is an example where we're walking the list of axes. And then we're saying for T in all of the tick labels in X plus all of the tick labels in Y. So I want all the tick labels X and Y of each axis. I want to call set visible to false. And that is exactly what Josh was saying. It turns the visibility of that object to zero. And then map of lib simply. Map of lib, as it's drawing, it checks anything it's about to draw. It has a visibility flag. And if it's false, it doesn't draw it. So this allows you to make things visible and invisible on the fly, which also means that you can actually make interactive figures as Paul is going to show a little bit later with event handling, where if you click on something, data becomes visible and invisible. So you can overlay data, turn it on and off, without deleting it. You don't have to re-render. You simply change its visibility. And so now that we've sort of seen here, there is an example that is mostly for you guys to study so that you see sort of a common, slightly non-trivial plot. And later, we're going to see a much more complicated one, but a common non-trivial plot, where you have a few lines. You want a thick horizontal axis. You want a thick vertical axis. You also want a grid at the tick marks. And you want a legend. And you want a title and labels. This is an example that does it all in a sort of a reasonably common type of plot, where you say, I'm plotting three data sets. And I want them all to have a certain thickness. I also want a grid. And I also want a legend. And I want my legend in the lower right. And I also want two thick axis lines, one horizontal and one vertical, to represent my x and y axes. And then I want to set my label. So this is kind of something that is a little bit beyond the simple plot command. And it's fairly common of a reasonably realistic data plot that you would say put in a paper or a poster for a conference or something. So how do you understand this is how all plots in Matlot Live are structured. This notion of figures that contain axes and axes that contain lines and polygon patches and other internal geometric primitives. This is how Matlot Live achieves all of its plotting. So what are some of, now I want to quickly show you some common plots that you may actually need to use in practice. For example, an error plot. How do you make an error plot? And in this case, I made it just with the standard, the standard, the top level syntax. Keep in mind, these are always the actual drawing is always done via the axis. So just as I made this in here, I could have made my usual f.ax and then these would be ax.errorbar, ax.setTitle. So in errorbar, you can specify the values of the errorbars in x and y explicitly or you can pass arrays with the actual values if the errorbars in your data are different. So making an error plot is very straightforward in Matlot Live. This is one slightly more complicated example where you want to do something like that. You want, oops, I'm not going to go through a real line. These, you have these notebooks. These are for your benefit. What we were trying to do is to give you a set of examples that was a little bit easier to navigate than the gallery for kind of common, commonly useful things. Okay, so this is an example where you have errorbars that are different for every point and in some cases you only have errorbars in the extra y direction and in this particular case you also want them. You want this to be a lock plot and you want the errorbars to be in a different color. So for example in here the errorbars are in a different color. They're green, which is different from the color of the lines themselves. You can study the calls. The idea is for you guys to play with this. You can make, this is an example of making two plots in the same figure. One with a normal linear axis but the other one to be a semi lock plot with a semi lock y plot which is the x axis is linear the y axis is a logarithmic axis. So by now we're used to this. We do two by one. So two by one means two rows, one column and I call plot on the first one and I call semi log y on the second and that's it. I have my semi lock plot in my bottom axis. By the way, something that I want to show you. Let me just make this in a new, oops. So two other things that are useful to know about the floating windows. These grids that you were seeing. Oh, okay. If you say G, the grid pops on and off. So the same thing as toggling the grid with a function call can be made interactively. And if you type L, you will change into a logarithmic axis, whichever axis is closest to the pointer. So this will toggle the vertical axis to be a log axis and this will make, oh, okay, they changed that. And they didn't ask us for permission. So yes, it's K. So L is the y-axis becomes log and K is the x-axis becomes log. Do you have a question back? No. Oh, okay. So these are things that are, these are handy to know when you're working interactively, especially for example, if you're plotting numerical error in an approximation and you want to see it, often you want to see the error both in a linear plot and in a log plot to look at numbers of digits of precision, for example. You can do, I do this a lot or toggling the grid. So this is obviously something that in your, if you want these in inline plots, you have to explicitly request semi-log, your plots to be log. So when are you there? No, so there's, the question is when are we going to make the inline plots more interactive? There's two answers to that question. The easy one is we're going to allow toggling between interactive and floating. That's relatively straightforward. I even had the patch working a while ago. It just has bit-rotted, it's a matter of, that's an afternoon's worth of work. The harder but obviously much more interesting question is when are we going to make these plots themselves be interactive? That is what everybody wants. It's not trivial at all to accomplish because, and we are looking into how to do it. There are JavaScript libraries that are capable of doing pretty good rendering. The question is how to hook up the map.lib back end with JavaScript. This weekend, somebody showed me, yes. It's GML5 back end. What's that? We've also looked at, so the HTML5 back end is a project actually from South Africa, from some colleagues of Stefan, from the Square Kilometer Array in South Africa. And Stefan, maybe you can update us a little bit on that. You met with them this over Christmas, right? Yeah, so what was the question I was asking? Oh, the status of the HTML5 back end. Right, so we spoke a little bit about how to interact with the Python notebook. They basically use it for visualization of the end, but I can run any other server that I want to do that's not a problem with the iPython. So yeah, the problem is that they have their own web server. So the HTML5 back end basically spins up a web server, whose only job is to produce plots. And so it has its own port. It's an entire separate web service for generating plots. Whereas obviously that, we're not going to have two web servers on the inside of one another. So the architectural design is not quite what we need, but it's possible that we could repurpose some of that. So I suspect what's gonna happen is things are probably gonna go in several directions at the same time. The JavaScript, excuse me, the JavaScript solution I think is gonna get done pretty quickly. This weekend somebody showed me, unfortunately, in private. It's not something which is available to the public. So the development at a company, they showed me they have wired up pieces of the iPython notebook with one of the modern JavaScript visualization libraries that are out there. And they're beginning to develop a prototype on that. So, and these people did it over the course of a couple of weekends, I think, using our code. So that I think is gonna get done fairly soon. The problem with that is that you get a few simple plots. You don't get the full map of lib machinery. Ultimately what we want is something where map.lib itself can send data to JavaScript front end, which renders nicely in the browsers. You can pan, you can zoom, but whenever you need new data from map.lib, it can call back into Python. And that is the long-term project. If anybody here likes JavaScript webby stuff and wants to talk to us, this is a really, really, really interesting project, but it's not trivial. It's not a trivial amount of work. Does that answer your question? Oh, yeah. Okay. The answer to a question like that is always, it'll be faster if you help us. Yes, yeah. If you roll your sleeves up, we will get done quicker. So here's a more complex plot showing, showing more complicated log plots that we don't need to go too deep into. Bar plots are another type of plot that is fairly common. A lot of people need to do bar plots with error bars. This is fairly common in many fields. So here's a quick example that summarizes how to do a bar plot that has error bars on all the data and the error bars themselves are of different colors for each individual subgroup that you have. Nothing particularly difficult about it. This one I want you to concentrate on because I'm about to stop speaking and give you a problem. So actually wake up and pay attention for a second. So here I'm plotting, I'm using a different kind of plot which is called a scatter plot. So you can plot, I can make a plot with, this is a plot of a hundred random numbers using circle O, using circular markers. This is made using the plot call. But Matlotlet has a different type of plot called a scatter plot which is the scatter call. And the difference between plot with circles and scatter is that in scatter each individual circle can be controlled. When you do plot with circles you're simply saying you can choose the color but it's all of them. You can choose the size but it's for all of them. They're all the same. Whereas in scatter you individually have control both over the size and the color of your individual markers. So this is useful for representing data and in this case it's with circle but you can choose the polygon the marker that you put in here and the argument to scatter are the x and y axes as always the x and y data but the next argument s is the size of that point. If you give a constant they're all the same size but if you give an array then that's the size in area. Keep in mind this is the area. So it's the area in pixels of that it's an estimate of the area of that marker and you can also give it a different, you can give it a color map. So you can actually encode with color in your data some other variable. So now I want you guys to spend 15 minutes or so doing a small breakout problem. Where the breakout problem is imagine that you have data that has this format. This is real data, there's a file called stations.txt in the data directory that you downloaded and the data contains four letter codes and then it contains three numbers latitude, longitude, and elevation and what we want you to do is first of all two things and there will be a later part with van handling. Make a scatter plot of these where both you encode the elevation with both color and the size of the marker then add a color bar on the right that shows the color elevation map and also label each station by its four letter code so that when you see a map you have the four letter code and if you have the basemap map.lib toolkit installed, whoops if you have the basemap toolkit which I don't know if EPD includes can somebody try typing the following import basemap? No it's not that, it's from MPL from MPL toolkits import basemap does that work on EPD or not? I don't know if they ship, it does? Okay so you guys all have the basemap toolkit then in addition to the basic plot do another one that actually uses latitude and longitude and you're gonna have to look up a little bit about basemap to finish this and that also uses the the basemap toolkit to actually draw proper latitude and longitude grids, draw rivers and country boundaries in this region of the earth and put the satellite image of the earth as a background. So it's actually fairly straightforward and you have, well 15 minutes let's see how far you get I mean the whole point of these is that there's a little bit more work than you can actually finish but give it a try for 15 minutes ask us if you have any questions and I'm going to put up what the results look like to give you, where's my dashboard? I'll give you this, mapping seismic stations. So this is what the basic plot should look like we're encoding both the elevation the elevation is represented both in the size of the circle and in the color this is the elevation in kilometers and these are the labels of each of those stations and this turns out to actually be a part of the Himalayas so if you use basemap you get the same thing but now this is on top of the NASA blue marble image so you can actually see this is the snow these are the snows on the Himalayas this is Tibet up here and this is Nepal down here and you have the rivers in Sian and the country boundaries in yellow so this is actually not too much code I mean that's it, that's the code that does this it's 15 lines of code and you get a very decent realistic looking plot so give it a try, you also have the solution but try to at least, you may not be able to get all of this working though you could cheat a little bit they're actually just very simple calls at the very least you should be able to get this one done because you have the data sitting on your system and if you get stuck raise your hand I'll come over and we'll give it 15 minutes all the way to green so did anyone get the actual earth image to come up? not quite? okay so let's have a look at the first solution so the first solution was really basically one line of two lines of code so okay no first we have to load the data so let me backtrack a little bit so you guys all have the mapping seismic notebook so I load everything oops so let me start in line I load numpy and map up lib so I'm going to define my D type as we saw at the beginning I'm going to have stations, latitude, longitude and elevation and this is the syntax that I showed you where you define a dict with names and formats you can also define it in this way a list of pairs either they're completely equivalent there's actually more syntaxes in that document that I told you and then once you have that you can say load txt which is the numpy just like I made zeros with a D type you can load with a D type so if you call the load txt numpy routine that reads data from a text file and you give it that D type which we just made then we load our data file and as Josh had said we make it a record just for the convenience of being able to type dot, lat, dot, long so at first I'm printing in here so we load the data and at first I'm printing just a few simple things for example, if I print tab.station tab is my table so tab.station it prints these are all my station names these are all my elevations the first station is tab zero which is bera at these coordinates and 120 meters basically at sea level the mean latitude each one of these things is a numpy array so you can ask what is the mean latitude of all these things well just call dot mean on that array and you get the mean latitude so this is just to show you how quickly you can summarize your data once you have it loaded in this record array form but now the meat of the exercise which was the plot so we make a subplot I made it a little bit bigger and now I simply say on my axis I call scatter longitude goes first on the x-axis latitude on the y-axis for the sizes we want to encode the elevation and I just played with numbers until I found a number that was roughly good I basically multiplied the elevations by 40 and added one so I wouldn't have very small numbers at zero this is purely for visual purposes and then for color c allows me to give it the color I use the elevation then I want a color bar so the color bar is an attribute of the figure because a color bar is actually a special type of axis the way MatBloodLib draws this color bar right here it's actually making an axis and in that axis it's actually painting colors so it's not an attribute of this axis it is an attribute of the figure but you have to tell it what data it's mapping you have to tell it the color bar what is it going to represent so you pass it the output of the scatter call I don't expect you to have known all of this but the whole point was to get you trying a little bit playing with the gallery finding examples we make the title and then we want to label it MatBloodLib has any axis object and put text at arbitrary locations with a text method so we simply and in this case I'm using the bracket syntax so you see that they both work so now for every record in my table I'm going to put text at the longitude and latitude coordinates slightly offset so I'm not slapping the label on top of the point but a little bit offset and then I'm going to make them bold just so that they are a little bit easier to read and that gives me this puppy and now I'm going to repeat effectively the same thing but now using base map to make a map of the earth and as you see it's actually pretty straightforward first of all I want my base map to contain a part of the earth that goes a little bit outside the boundaries of just my points because otherwise base map will draw the box really just on the points so I made I found the bounding box by calling min and max on my latitude and longitude and make it a little bit before and a little bit after so these are my latitude and longitude boundaries my geographic grid is going to be my parallels and my meridians are going to be these I want to draw a grid with five of them just so I have something that looks nice so I can use just lint space this is a tip that I gave you when you make base map you pass a flag called resolution this is the resolution of the geographic data structures that it contains things like rivers and country boundaries and stuff like that the F flag for fine resolution makes this call take forever on this laptop I don't know how long it would take on my desktop at home which is a very fast machine it took like a minute to render that map with fine resolution so intermediate is fine and then this is how base map works you make a plot the same thing and then you create an object called the base map object attached to an axis so what happens is you have your axis and you say treat it as a geographic map base map is a really fancy library that has knows about all the projections that geographers use out there and it allows you to do transformations between data and geographic coordinates and data in plot coordinates very easily so this object now the M base map object knows about earth it knows about earth within these boundaries at this level of the resolution and it will draw in these axis so at that point it's actually illustrated forward you say draw countries and you give it a color yellow, okay draw rivers blue marble and that slaps in the background the NASA blue marble image draw parallels and you give it the parallels and you say which parts of the map you want labeled because in mapping people often put labels on like on all sides of the figure so you can say which parts of the figure you want labels on drummeridians and then this guy knows how to do scatter it's like an axis but look in here I'm not calling axis on I'm not calling scatter on the axis itself I'm calling scatter on this guy which understands and does all of the reason why you want to do that is because in this case the figure that we're getting is a square axis so it doesn't appear to be that big of a deal but imagine you're representing a big patch of earth what happens with a big patch of earth what do you need to take into account curvature exactly coordinate transformations there are and there are many projections and there are projections that are that are conformal projections that respect angles but distort areas and there are equi area projections that preserve area information at the cost of distorting angles you can't have both because the sphere has a different topology than the plane so the reason why you call scatter and plot and these things on the method is because it was instantiated with a specific projection and it will apply the appropriate coordinate transformations so that you pass it data in latitude and longitude and it knows where it has to go in the right coordinate system in the resulting projection and that's it and after that I made the labels the same as before but in this case I call them in yellow and this is an extra flag there's a couple of extra flags in here that you haven't seen before one is this z-order the z-order is the depth on the figure you can imagine a figure as being made of different things being drawn on top of each other and sometimes the order in which Matt Lottlet draws things is not what you want and you may end up with the rivers covering the dots and you wanted it the other way around so by passing the z-order for a specific call you can control in height and the higher that it doesn't matter the actual values don't really matter all that they define is an order a stacking of things so by passing z-order larger I just make sure that these guys end up on top of the image and the rivers and the country boundaries and then I also was able to control the transparency in this case I said make these guys slightly opaque so instead of being alpha alpha is a number between 0 and 1 0 is fully transparent you don't see the object 1 is fully opaque and anything in between blends it with the background it allows me to basically see a little bit of the background behind this so I can see the rivers behind these points but only faintly so this is only a few lines of code and this is the kind of plot that you could perfectly well put in an actual paper and if you save it to a PDF you have a publication quality plot for a paper on a on a reason and this is a real data set okay any questions on this yes it should it should um let me very quickly I don't want to take if it doesn't work right away we'll look at it um we'll look at it later I'm not because I want Paul to have time for his section but I need to load the data that's interesting what's that oh sizes not defined that's why because sizes was defined where are the sizes defined ah here okay so that was that the title is not defined Jesus I did have to run the whole thing okay so this one is working fine the one with the without the map and now let's run the one with the map nope it's it's fine so yeah there is no reason why it should are you running it from my notebook or yours? it didn't just try it from the command try it from just type I type on in the terminal and then see if I can see it okay but we'll look into it at the end of the class I'll come and have a look but now I want to leave it for Paul so here you don't want to do Instagram oh yes you're right yes we had a little bit more so two other very common types of plots that that you may need to do and this is here I'm simply showing you how what this that map of it has a built-in histograming function which is very handy you give it the number of bins you can control what how it's how the histogram is normalized the what color to put it in you can you can as always look at the help string but importantly what it returns is the actual edges in the bins and the geometric objects that compose the actual rectangles of the patches so it's kind of an unusual return value but it's it's useful to know because you may for example especially the bins you may need to use those for some other things so you you've been your data you do it your computer histogram but you often may end up doing other calculations and you want to know the edges of the bins that were used so this allows you to simply say however many bins you want and you get the actual bin boundaries back so that you don't have to recompute them by hand and importantly look at what I did here map of it has full-blown tech support so in here we put at coordinate sixty zero twenty five right here it's a little faint I should have made the the font size a little larger mu equals a hundred sigma sigma equals fifteen and I'm using dollar signs and backslashes because map of it has full-blown lot of support and it you don't have to have a lot of installed it actually has a lot of it layout algorithms of tech and the tech fonts are built into matlotlib so you can pass any arbitrary math expression to matlotlib you enclose it within dollar signs uh... whoops that was me uh... you enclose it with dollar signs at both ends and because lotik has so many backslashes you typically want to make that what's called a raw string which means that backslashes don't require extra additional escapes uh... in python and then you can put you can put anything that in matlotlib accepts text accepts lotik uh... all of these commands that uh... that accept uh... that accept text accept lotik so here is one final example that is really straightforward and that the only point of it is that you have it for reference here where it shows the various types of text annotations that you may want to do so you have this this is the what's called a soup title the super title at the top for the whole figure the reason why it's different is because if you have say a figure and two axes like this if you say access set title each of these will set its title right here whereas the the figure title is a single one centered at the top of the whole figure obviously with a single axis they're kind of they're almost redundant but there is a difference labels for each you can put text and you can express text you can put text either in coordinates relative to the axes or in meaning between zero and one so they're fixed position on the screen no matter what the data is doing or you can position text in data coordinates so that if pan and zoom the text label moves with the data because it's attached to the data you can put tech and you and it has the notion of what are called annotations which have arrow support and if you look at the gallery there's all kinds of arrow styles that you can choose so that you can put text and point an arrow to a specific part on your data so this example is really mostly for for your reference access sharing we actually covered this at the beginning and so I'm pretty much done because event handling is going to be covered by paul with a separate presentation for event handling and fancier plots and then we'll come back to image images in that plot and a little bit of 3d if we have time for it pointer, mouse, cool yes it is I have my mug somewhere alright so hey guys so I'm a little sick and a little doped up so please excuse my slowness we're going to start with the I'm so sick and dope y'all you don't even know let's see I'm so doped up I don't even remember what I called my MPL events that must be it okay so this is going to be short this part of it so Fernando already stole my thunder so I'm going to load up this notebook in a Pi lab mode and I'm going to show you first well that's not right okay cool so Fernando showed you what some of the buttons that you can press when you have an access showing and here I'm showing you all the buttons you can press and I just want to put this up here to also point out the fact that you can this is something called an RC params okay and RC params are there are some default values but you can also set them in your dot map plot lib slash map plot lib RC so if you just look up with RC params for map plot lib you can actually set these so that you can change these keys to whatever it is that you like if you don't like L to be you know toggling the logarithmic thing you can you can change that if you like and we'll look into a little bit how you can add more events but this is this is where you would do it and let me show you now that we have that so let me just do a PLT plot sorry range one to ten so this popped up I wrote loaded up Pi lab mode and so now I can if I press the L key as Fernanda already showed that this is logarithmic view of the same the K will do it for the x-axis and so what are some of the other ones the ones that I always use are to O so I don't like to go up here and use the mouse to just press keys or press these buttons effectively so I just press the O button and that's the equivalent of the zoom whoop sorry so O is zoom so now I can do this zoom thing and so so I've just zoomed in and I could go up here and click the back button but there's actually already a built-in backspace will do it should do it when it works alright how about C okay so C and so C and V allow you to go back and forth and again you can set these so it's the it's I'm doing the same thing as this so anytime I press V to the which is to the right of C on the keyboard turns out on all keyboards regardless of the layout you'll go that way if if you press C it'll go back the other one that's that's good to know so let's do a couple more so that zoomed in what if I want to go back to the beginning of where it is that the plot started with which is this home button right you just press H and so that gets you back in there that's useful I use that all the time so uh... next to O you have the P key which is the panning so that changes it to a hand and now you can do the the sort of stuff that Fernando was doing so I do that all the time so switching between O and P sort of zoom in or zoom out um O and P any questions on that it's not terribly enlightening but it's very useful G oh oh yes yes yes so thank you thank you uh... so G is the grid when it works again uh... sorry I guess I'm losing mouse focus that's what's going on here um... because the mouse is sliding off so G is we'll set the grid and unset it and what what else did I have I said O P 8 L G H, C, D, yeah cover them all um alright uh... so let me oh so the other thing that the uh... one of the ones that Fernando glossed over is that you have these uh... subplot parameters that you can configure and this is actually um... adjusting the height of so this is when you're like getting ready to do a publication and you want to fit that title just nicely or you want to scoot it over so you can adjust it you know from the top or from from the left and the bottom and whatever and then there's this thing called H space and W space you might say well that's not doing anything uh... that's because it toggles the horizontal and vertical space between multiple subplots so if we were to do uh... let's try it to do it two by two so how do we do that subplot a little control window that you get may look a little different it depends on which GUI toolkit you're using that little window that Paul is using may look different but it contains a simple challenge so this is subplots go back to adjust this thing and now you see what I'm doing the W space on that and then the H space does it by the way the mnemonics are H is for height and W for width yeah I you know the yeah I have a thing against mnemonics or for them or something like in the ipython notebook it's uh... you know control m a to put something above and control m b to put something below and I always think oh do I want to before or after and it's exactly opposite and and there's another thing we debated that I know yeah yeah and the same thing with uh... this vertical and horizontal splitting what it means to split vertically and horizontally uh... so this is what we do on our Friday nights is we just debate on the list but the merits of one versus the other so I'm gonna do another example so this is the slider demo so some of you may not have seen this plot pop up the way that for now I hate your window manager this is being recorded so this is permanent record of my hate of your window manager uh... uh... you might have not seen some uh... uh... these nice little widgets that you get in qt you might have seen something like this uh... which which uh... uh... might have seen these little blue things that allow you to click in a different spot and adjust the h space w space and the uh... all those parameters as well so so this is this is a little example that you can see and so we're doing event handling as i'm clicking something in my figure the the actual uh... one of the drawing axes is changing i actually have these little radio buttons so if i want to change the color of this thing i can do that and so and this will work on any sort of uh... interactive back end the map plot lib uses so you didn't have to specify like a qt or wx specific application allows you to just click on things on the uh... directly on a little axis so here's the key press demo so uh... go through the code a little bit here so the the main point is of which which key is it that we press and so i think if i press the x key here what happened what do you see i'm looking at the sorry i i just i just remember that it's the xk i haven't shown the code that does it but uh... but basically you get so there's the browser on press it's yeah sorry it's called off that's why okay thank you because i'm like it shouldn't be that long uh... okay so here's here's a little function that we define that when we have an event uh... that event is going to get passed to this function so long as we hook it up in this map plot lib connect sort of way where we say oh give me a all the key press events that you're going to see and uh... so that's going to pass it an event an event has an attribute called the key so it checks that if the key is x then we're going to set the xl to be visible uh... we're gonna get its current visibility and then we're gonna flip it and make it whatever it's currently is to the opposite of whatever currently is right so this little thing so we can you can imagine just very quickly uh... making why do something interesting you know if event dot key equals equals why and what should we have it do uh... how about we do xl dot set color two uh... so this will work at once but uh... yeah all right sounds good uh... uh... you want me to do that god you guys are so yeah it's true i know i should have uh... it's all great to me i mean this is on this much medicine i'm totally fine you guys are the ones that have to sit through this this is like not even a second in my space time so uh... well let's flip between two colors maybe c equals equals red uh... yeah well if i do it in else uh... well yeah you're right dammit fernando why do you have to be right uh... well we'll just do it once screw it we're doing it live doing it once we're just gonna make it blue so shift on her and so there you go so x makes it disappear and appear and why uh... what what happened so it did see it did see that i pressed something oh i pressed the wrong key no it got that i pressed y so what happened i didn't draw right i didn't draw and how to one way to make it redraw is just to resize the figure by full screen this right i should have i should have had like we had a call to fig canvas draw here i should have had another one or you could really you could move this call out to here somewhere so it does it sort of at the end right that makes sense does the coordinate system set up? is top left zero zero or bottom left zero zero? uh... bottom left is usually zero zero and axis coordinates but in there's different there's different ways of uh... sort of traversing that and i think in figure coordinates there's such a thing as figure coordinates and at the top left is figure coordinates and that's i think inherited from like x eleven something like that but usually typically at the bottom this is in axis zero zero and this is in axis one zero so x is one and uh... y is zero and then uh... you know this is one one and zero one remember correctly the event gives you both coordinates so you don't have to have different dimensions if i remember correctly you get an event to get both the data coordinates and the axis coordinates from where the event occurred so that you can write your logic to determine the sense the documentation actually explains all the fields that the event object has so it has key data coordinates uh... yes so so so yeah we're gonna get into that so so this we just dealt with the key press event and so one one little assignment that uh... that's useful at least if you're like me i i i i page through data a lot so when i collect my eye tracking data or something like that just go left and right so i'll zoom into a certain level where i can see the features that i want to see and then just go like left and right and you can hook that up easily and i have it like with uh... the comma in the period which is like the less than a greater than signs you can easily get the current limit right the x-lim of the axis that you're you're you're looking at and now updated so that it's one if it's if it's the current with plus whatever it was currently at so that's a very simple thing with this demo you just add a few lines of code and you've got something that because you're gonna to do this right you're gonna use the object oriented interface you're gonna be able to just pass a current axes or you can do something like if you didn't pass an axis it's gonna get the current active axis right so uh... did that make sense or did i just get too excited about this i got like one nod alright so uh... so here's here's another little so these are these are just map out lib gallery example so uh... you can also hook up not just uh... not just keep press events but these pick events so that's uh... a mouse being uh... let me just run this a mouse being clicked on a data point so if i click here nothing interesting happens but if i click on oh sorry yeah yeah thank you so when i was clicking clicking around nothing interesting was happening but as i as soon as i click on a data point changes another plot down below right so that's that's very cool so that's you know you have some visualization it's a way of like being able to paint your data and and look at it in different ways and so this is another little useful demo so these are like pick events that it's picking up it's finding the nearest point to in your data to where wherever it is that you clicked so long as it's within some radius because if i click out here it's not going to change it right but if i once i start getting close then it's and it's shown that radius in this demo by itself so that's cool so these are this is like taking from from a lot of what fernando showed you is is these uh... sort of inline plots things that you would do for your publications this is very useful for just exploring your data and trying to get a handle on things and maybe maybe uh... you know you can you can do things where you have you have some some cluster that you're hooked up to and you have a parameter space that you want to explore and you just want to literally like sit there and click randomly wherever it is that you want what whatever it is that you want the initial conditions to be for some problem you're trying to explore and then get the results back update the plot and then be able to look at it all sort of in one go stephan yet sure yeah totally yeah anything we're going to do an example alright so that's all i had for for this mpl events uh... any questions on that i think i covered it all uh... which one is it this one alright so here's the elaborate example it's gonna uh... sort of touch on a lot of the things that fernando already uh... touched upon but but uh... do it a little bit better we're gonna recreate this full graph in matplotlib and it's gonna be awesome i have a feeling so i'm gonna let you take a look at that anyone familiar anyone not familiar with xkcd no okay we're all friends here uh... so okay so gonna start off with inline mode right and first i'm just gonna get the data and i'm just gonna plot it so let's see that what that looks like uh... let me do this alright so we're inline mode so i'm gonna have all of these and you have the notebook for this but i'm just gonna sort of broad brushstrokes here so i'm gonna get x and y and uh... something called intel and inan for inanity and so i'm gonna plot them and and you'll notice one thing that fernando didn't uh... stressed but is very useful is that as you plot multiple things they the there's this uh... sort of ring buffer of colors that get keeps getting updated right so that's nice that you don't you i know because by default blue is always the first one and greens the second one and the next one will be read and so on uh... you can also change this and and set your own sort of color ring buffer that you like to go uh... go over so so this is uh... pretty uh... good plot let's let's do some label so for now already have an example of this let's put a legend on it so i'm just gonna uh... uh... hit that so i've inserted this label and by default that label isn't going to show up unless i create a legend that label doesn't has sort of no meaning it'll it may have meaning so that when i can look at an axis is uh... lines object i'll be able to pick it out out of like you know all the labels that are non sort of blank uh... blank line but so here we've got a legend that got placed right here and that's not really ideal so what can we do there we can look at plt dot legend and c this work plus check it out alright so uh... plt legend takes a argument so you can specify lines and labels but you can also do this thing called location lock and yet you can do it as a string or is this location code i never really use the location code less i really need to save some keystrokes but so we're gonna there is a location called best probably the best rate just i'm just guessing so he needed to do this can you believe it uh... for now how to make it go away so we're gonna do that we're gonna look at best and see if it does something sensible and it does cool best will always do the best thing that you want because it's not a mind reader but it is pretty good job of like getting out of the way for for the data that you have so if we had some scatter plots over here might move it actually to the center or something like that okay so but i'm not happy with this because uh... we don't have labels so we have to we have to label our axis and so here i'm using the pipe lot interface just do x label i could also do that as ax dot set underscore x label and so human proximity to cat we've got that uh... everything's relative right if we look back on this uh... plot there there aren't really like unit zero there is this foreign near thing but intelligence and inanity don't have units so let me get rid of those so everything's relative so i'm gonna say it set the x ticks to be uh... at position zero and position point eight which is sort of the end here i'm gonna call that instead of having a display zero point zero and whatever i'm gonna have a display far here and near here but it's kind of down on the play button at the top okay cool and the play plays the next one great awesome um... okay so so what we just did is we got rid of that we said the y ticks is an empty list right so that that displays no ticks so now you know what the different things in map lot liberal right this is a legend these things are ticks and uh... the the little little guys taking out a ticks and then they have labels right so so this is a sort of keep that in mind because then you can if you want to change the color of these things or access them later this is this is how you would do it and uh... as was mentioned in an answer to a question earlier this is the sort of thing where you can so here we did it just as a list we signed where the ticks will go you can actually change uh... something i have it later uh... down on the notes and under miscellaneous at the end of the uh... notebook something called a locator which decides where the ticks get placed whether or not they get placed you know base ten or base e or whatever it is that you want some some weirdo thing that where you want it one at one one at three one at seven and one at twenty two or something like that uh... so that's a locator and then there's also something called a uh... formatter which will decide that at a given location how will represent that that string and that's actually what plot date does so plt dot plot underscore date will convert a date into an integer but then when it represents that integer it converts it back to something that looks like a date ill you know twenty eleven you know slash or or whatever whatever format you choose for it that makes sense so so now you know about four matters and locators so the next thing that i'm going to do is uh... if you follow edward tough you know he hates boxes and i i i'm not a fan of them either so we're gonna liberate our axes actually have a method that in my little my own version of plot live auxiliary functions that i call on all my projects which is called liberate access that does something like this which is to get rid of these now these things are called spines that are around the axis and you can actually display uh... what we're gonna do is we're gonna hide them so that they look like they did up here right you notice that there is no but there's a box around the whole cartoon but there isn't a box around the exact right so because because there's no it's sort of arbitrary to limit your axis that way to actually extend that way and so press play and so on is not so much nicer you have these boxes there there's confining you telling what to do uh... and another thing that i did here was i uh... if you were uh... typing along is i kept around the returnable legend call so that i could do this thing i could set uh... draw frame equals false another way of uh... so i all let me go through so i there are axes like i said have the spines and i decided to take the uh... the top one and make it invisible the right one make it invisible and then i said don't draw the frame which is the box and the background of a legend so now actually if if we were to if we were an active back and it were to can this thing this line would go right underneath there wouldn't be it wouldn't be this white background it's still a box that's around it blocks you from seeing it or you could also you could get the frame and then maybe set the alpha so it has has a nice little uh... maybe set an alpha like half like point five and then it'll be will be nice and sort of you could still see it but then it's just nice right i mean that's that's what maplot lives about to me i'm totally high right now this is great uh... i don't know how great it is for you guys this great for me alright so the next thing that all you'll notice this all all this is nice right but but what is that what is that that's not nice what is that that's ugly i don't like it does anyone know what it is it's a take uh... i don't like tix let's get rid of this nervous take and how do we do that well we can just set say a x axis tick just the bottom don't take the top tick just the bottom for me please play that uh... just got rid of that take uh... okay well legends are nice but now i have to do this color thing and if i represented this black and white and if i want to stay true to what the cartoon look like originally uh... i really want to label this line right you don't want to have to like look back and forth and if i had more lines in this i'd have to keep going sort of like all to the left and to the right and sort of go to legend and legends not the boss of me i'm going to do my own thing and what i'm going to do is uh... i'm going to annotate it and i'm going to do this this fancy thing uh... called keyword arguments uh... who knows what keyword arguments are all right what are they that's pretty good uh... okay so keyword arguments allow you to uh... uh... you notice that there's a lot of parameters you can pass any given function and so here at this function we're gonna pass it uh... text coordinates arrow properties and they have this with the head thing and we're gonna do that multiple times right we have multiple annotations and and then later on i'm going to keep calling annotate down below some you're gonna save it into a dictionary and then the way that you unpack a dictionary to be passed as keyword arguments things that were uh... remember we're saying like alpha franando saying like alpha equals point five i'm getting some confusion so let me let me say what i mean so the way to do normally would do something like plot one two three whatever and then so here's a keyword argument alpha equals point five right and i might have multiple keyword arguments listed alpha point five and color this in that and lined with so-and-so and what we're gonna do is we're gonna make a dictionary out of those so where the where the keys are what the keyword arguments were and the values are what we want them to be and uh... it's actually if you use the the sort of the dict way of building uh... a building a dictionary then it just translate exactly you can take it sort of the same code and uh... so you make a dictionary that and then the way that you pass it to a function as you do this unpacking using a double star case that's very useful i do this all the time because you end up sort of specifying all these things about will how it is that i want an annotation to look like i don't have to pass that and copy paste that over and over and over again and it clutters up the code you forget what it is that it's actually doing where so now we say oh it's that no arrow keyword arguments i call it that because here the uh... the head with is uh... is a specification of how much of an annotation uh... that an annotation normally uh... you create an arrow which has an arrow ahead but here i'm gonna get rid of that so that it uh... resembles the plot like we had up here without errors it's just a line so normal uh... if if there was an arrow ahead it would be you know like a little triangle sticking out over here we're gonna go without that and so you have this sort of label that's a python built in sort of things so it's a standard standard procedure way of writing functions of something like this args here is an unpacking of the argument so when you were when you would invoke so this is sort of an aside as general python aside if i were to foo one you know four and then t or something like that that would unpack to args that that's a these things get passed as a list and here i'm gonna on unpack this list so args is going to be a list in this function and then if i were to specify things like foo equals thirty then then that would be that would end up in the dictionary of keyword arguments so now that we've got that well we had to uh... have this fancy scroll back to the top this you're a kitty thing and we can actually do that a map lot live i don't know why you want to somebody took the time to do it that we were gonna say josh that it's important so you're a kitty that's cool okay so i just did that sort of for practice so we can have that to look at and and again i'm i'm gonna use this thing where i i say kitty keyword argument so i don't have to keep re typing all of these parameters next time i want to create this kitty annotation the next time i call it i'm just gonna say well here let me just read the read in the image uh... show it so this is i'm reading part of the comic i'm gonna cheat a little bit and i'm gonna just copy paste the the stick figure guy sort of out of the image and place it in in another axis to complete our our thing so this right now is using this jet color map set it to gray the default color map then that looks more sensible and now i can zoom in to a specific little guy in that image so what i just did there that used i am read appear to read in this comic which which shipped with the uh... get repository and uh... so now i have access to that comic happens to be just black and white and now i can start adding axes and uh... putting in different parts of the comic different versions of this guy into it and then annotating the last one with your kitty with just passing it just keyword arguments like that and have to re-type that whole big long thing so that looks pretty close but it's got all this gunk around it right all these labels and so uh... well let's let's put it all together first so so that this axis gets our previous result right so i'm just copy pasting so we've got our intelligence and inanity of statements and your kitty so this all looking better uh... let's see did i want to make any point in here alright so then i'm gonna i'm gonna make a little function this is back to the sort of object-oriented way of things i'm gonna make a little function that says oh this is a cleanup axes function it's gonna take an axes object and it's gonna set its frame to false so the frame is uh... a shorthand way of specifying all the spines whether or not they're visible uh... so it's and and also that that background white image so it's gonna set that to false and it's gonna get rid of the x and y tick so that there's no more ticks and then you guys are leaving but Fernando's gonna talk some more about Mayavi and uh... i feel like i'm buzz killing your thing Fernando sorry okay and so finally now uh... so at the end of what i do is i call this cleanup axes and i change the figure size a little bit so that the aspect ratio looks nicer and you have your result some things that i didn't mention is uh... briefly just two minutes or something for another is that cool uh... forty perfect okay is inanity and intelligence are actually different things right they shouldn't be plotted on the same axis here we just you know just for for fun we plotted them on the same axis but you could actually uh... call something called twin x to uh... when you have it let me just show you what that looks like so so here i i create a new intelligence which is normalized in iq right it's it's out of a hundred and i create a new in anity which is actually an exponential of what it used to be i create an axis i plot the intelligence so that'll be from a hundred to zero basically and then i twin that axis twin it's the sort of along the x so the x will be shared and overlaid and then it's gonna create a new y-axis for the data that i'm gonna plot there and so and that data i'm gonna plot as a semi-log thing because now i said that that in anity is actually an exponential thing things get more and more name as we've evidenced by this presentation the longer you go on press play here and so here's what it did right so this is iq on the left and this is in anity on the right and it just created a new axis and there if i were to go pan this left and right it would pan both both of them right at the same time so it's a shared axis shared x-axis but the y-scales are actually different and so here's to do the same thing with labels inanities measured in twits i decided and so here we have twits iq points and as a fun sort of challenge homework assignment the thing that matlab i think does by default when you do the sort of twin x trick is it'll make this whole thing red including sort of or at least i think it's a visually clearer if you were to make whatever color this line is make this whole uh... the all the ticks here and the label and maybe even that thing that color as well sort of to differentiate so that you know that that this scale applies to this line and the other the linear scale applies to to that uh... some other things that that uh... in the notebook is that you can actually do things like broken axis and so i i put a link to the broken axis example and uh... so that's useful when you have like outliers and things like that and uh... yeah for now to told you about how to poke into the all the a x dot methods that allow you to set the different properties and i talked about rc params and uh... talked about ticks and that's it thanks any questions okay so couple things to know one is that we posted the solutions to the homework from homework zero and were zero indexed chris is looking at me funny i did and then word i've now just posted the homework one which is obviously related to to these uh... settle lectures i just wanted to go over that with you just very briefly and then we'll give it back to franando alright so homework one uh... the first thing is to reproduce a paper uh... uh... uh... a uh... figure from a paper that you've actually published if you publish something if not try to find a figure like in a recent journal article that you may have read and your job is to reproduce it uh... using that pilot as best as possible so many of you may have produced it with matlab and then you can just do this fairly trivially or you may have produced it with some old code and hopefully still have the data around if not you're not really a scientist uh... so uh... try to reproduce it essentially purely in python and what you'll do is you'll hand back to us the python code the new figure and just perhaps a link on the web uh... or a screenshot of the old figure so we can compare how well you did and obviously won't be able to do everything exactly the same way but try to reproduce it as faithfully as possible this will give you sort of uh... that that belief that you can actually start using matplotlib to actually make your own figures for publication quality uh... i think that's an important thing to do again if you don't have your own figure that you're really proud of uh... where you don't have that date anymore you should be able to find something on the web that you can reproduce uh... okay so the next thing is to uh... basically reproduce this plot and we've given you the data for that in a bunch of csv files and you know it's sort of again a sort of publication nice looking thing uh... and you'll be able to make use of what paul just showed you where you have two uh... different axes on the left and the right hand side uh... so this is really just kind of a grind it through and figure out how to how to make it look like this and then the last thing is something called brushing so i'll show you an example of what that might look like the idea here is this we want you to build a data exploration uh... a framework where you maybe uh... have sort of a high dimensionality of of your data you perhaps you know have five or six different attributes and you want a plot you know attribute one versus attribute two attribute two versus attribute three etc so you make these plots uh... and then perhaps you're interested in stuff that's like over here and over here and what you want to do if this is this is being dynamic is i might want to highlight this and here and in doing so i wind up sort of graying out uh... different points over here you can imagine taking what uh... what you already saw from paul we're now perhaps you click on the points that are not grayed out and you might get some pop-up information so i'll let you be fairly uh... uh... expansive with with your approach here but the point here is that you're going to wind up having to capture mouse clicks on one of these different uh... axes and you're gonna then have to take action all on these other different axes this whole idea of brushing is something that you can look up on the web and we give you an example of a code called viewpoints we actually can just play around with brushing so you can grab some example data uh... and you can start to building this question to do that you're gonna have to do with event you're gonna have to do with event registration it's a good idea that in the map of the gallery, there's a section of examples on it I hand it, so grab a few of those, run them first they go beyond what paul showed and you'll get an idea of the kinds of tricks you need for this for this particular moment and hopefully this will approach something that you actually would actually want to use in your daily work flow okay so i'll turn it back to Fernando it's pretty pretty clear that we're not gonna we're not going to wind up getting to work on the homework with all of us around uh... so you can start sending us emails as soon as you have questions and obviously we'll wind up having our help session on friday, we said from three to five all that stuff is posted already on the web so i know it's late and we've covered a lot of ground the notebooks that we gave you have have a bunch of this stuff already in them and we only have a few minutes so instead what i'm going to show you is demonstrate uh... one homework problem that i had for you and i'm just going to show it to you was this understanding images notebook where your task and you could try to do it um... yourself, your task was to create now they've all come up, so your task was to given an image file to create to create a window where you could see your original image and pan and zoom around it but also where you could decompose your image into color channels so that an image a real image, an RGB image, has a red channel, a green channel, a blue channel so decompose them be able to explore the image so that when you zoom into one into one of them you're zooming so this idea of linking all the axes for all of them with real images and then doing channel mixing so that you would construct the grayscale the grayscale image both with a naive with a naive image blend and with a more realistic model of the human visual system where the balance of red, green, and blue is different so that for example if you this is what simply averaging the red, green, and blue channels will give you whereas if you use a better model of the how we perceive color you'll see that you get better contrast so these are very basic image processing tasks and I give you a couple of images there to play with as well as getting an image histogram, I think I failed to recreate an image so I ended up with the histograms that were laid here but these were the histograms for luminance and for each of the channels so you have a notebook in there that has already all of this done and I provided a couple of image files for you to play with so that you could see what this would behave like the statement is in the main problem so have a look at this and if you finish the very easy homework that Josh just gave you in an afternoon then you can try to write, basically it's a little kind of image explorer that doesn't do any processing but it shows you how to use matplotlib for understanding the structure of images but finally what we're going to wrap up with just a quick demo is with Mayavi so up until now we've been looking at matplotlib which is mostly a 2D plotting library it has some basic 3D plots and you can look in the examples at what it does but for really heavy-duty 3D visualization what you want to use is a package called matplotlib I'm sorry called Mayavi and what Mayavi is it's a beautiful project that uses the VTKC++ library for 3D rendering and it is really a package for heavy-duty scientific visualization that uses OpenGL for rendering the images that allows you to do interactive zooming and panning and whatnot so let me close this so here we don't have time for a lot but I'm simply going to show you very quickly some of the examples of the kinds of things that mayavi can do because the beauty of Mayavi is that it makes it very easy to do fairly sophisticated 3D visualizations using this library that I mentioned to you, VTK which is a fairly complex library to use and if you read the VTK if you try to use VTK by yourself and you read you go and look at the VTK tutorials, the book is about that thick you have to understand this nasty deep class hierarchy you have to create actors and a pipeline and inject filters and render the scene whereas Mayavi lets you do something like this you have a bunch of lines, here you have created the X, Y and Z coordinates of a set of lines in 3D, in 3D, in this case there are lines that wrap around a torus and simply you say plot 3D, X, Y and Z and you can control the radius of the tube which color map the API is modeled after the Matplotlib one so if you know Matplotlib you already kind of know what to do, they're different projects but they're very similar in many ways and boom, there you go, there you have a nice 3D interactive fully interactive window which you can pan, you can zoom you can move around you can control the views and so excuse me and this is actually an anaglyphic render of it so if you have if you have 3D stereo glasses you can actually see this thing in 3D so Mayavi is it's a little bit more heavy duty than Matplotlib, you saw that it went between when I clicked run and the image appeared took a while because VTK is a large and complex library but it's a spectacularly powerful library, I'm going to just very quickly show you some of the examples for example that was the line plot if you need to if you want to call points, plot points in 3D same thing, X, Y and Z very similar to scatter but now you have points scattered in 3D instead of in 2D you can have simple surfaces, you have a surface defined mathematically, sine of X plus Y plus sine of 2X minus Y plus cosine of that and you want a surface, you just say surf and that's it and you get a nice surface plot that you can rotate that you can so many of the 3D visualization things that you see online are actually done using VTK Mayavi makes VTK very accessible and very easy to use with this MLab interface but you still have the full VTK API so Mayavi doesn't prevent you from using the raw power of VTK it just means that kind of the 80% of the jobs are quick one-liners like these where you can get what you need done rapidly and then for really more complex things you have to open VTK itself for example it has a flow renderer, what is flow? Flow is a streamline renderer so this is a case where you define a vector field and you can sort of see some weird lines there, if I grab it I'm now moving a sphere and what this sphere is it's acting as a source of point particles and then VTK will integrate the trajectories from those point sources and generate generate the streamlines that come from that point source what can you use this for well to understand the structure for example of the system of differential equations the classic Lorentz attractor from non-linear dynamics 1963 MIT when sort of the the first big chaotic system of chaos theory these are the non-linear equations that make up the the Lorentz system so here you go you simply create these you compute that vector field and then this is a slightly more complicated example where now you are adding a stream tracer by hand so flow is a simple wrapper but with a few more lines of code you can build something like this which is this nice this nice thing which is showing an isocline one of the surfaces and here you have the source for the streamlines and so you can see how the attractor gets built up as the these are the initial conditions for integration of the ODE and they all converge into the attractor because it's a dissipated system so they converge into the attractor so building something like that which is a fairly elaborate and realistic being examples from the Mayavi gallery so these are all I grabbed them with load pi the same thing that Paul was doing these were all fetched last night with load pi from the gallery and I showed you only the simple ones and one of the advanced ones but there are some really elaborate ones in the gallery for very like remeshing a regularly sample data in 3D spherical harmonics for those of you who do quantum mechanics or if you need to plot in fact there is one of plotting atomic orbitals of atomic orbitals of hydrogen by also representing the phase the phase of the orbitals on the surface so an actual water molecule being displayed here it's a very very powerful library and what Mayavi does is it lets you access it with kind of a limited amount of effort and it gives you sort of an on-ramp because these these examples that I showed you here as you saw the main the most frequent calls flow points 3D surf plot 3D those are just very simple calls and then if you need you access you access the objects themselves and one thing that I the that I want to close with is let me just run this one because it's as good as any is this is the window where you pan and zoom and and you and you move things around what are these things at the top just like the matplotlib window had a few buttons so some of these are very obvious these are views you want to be on the x-axis from one side or the other view from the y-axis view from the z-axis or the isometric view okay these are just camera positions this is an outline that lets you another this is the isometric view this changes the perspective correction this toggles an axis indicator down here this full-screen set this lets you save like the matplotlib one that lets you save but the most interesting one is this one that's the miavi logo if you click on this takes a minute to come up what you get is this is everything that miavi is doing for you vtk is structured the abstract idea vtk is you build a render pipeline where you put actors each actor is capable of doing something you hook them up into a pipeline and then you call render on that pipeline and each actor feeds its output to the next output and you can inject filters in that pipeline well if you click on that icon you get the actual pipeline and now you can manipulate everything you want about about the objects in there so you can choose you can add new elements to it you can add new filters and modules you can add new elements so this is for example this is the iso surface and if i click on the visibility for the iso surface you saw it went away so that was the iso surface that was displayed there the green iso surface i can turn it off so it lets you control everything about the underlying vtk c++ machinery interactively without having to know all of that so i've i've read through the vtk book it's kind of a nightmare with my avi you need to know very little of it because between the high-level interface and the fact that the gooey lets you control all these things most things you need to do with visualization you can do from here and their documentation does have examples if you go to the bottom then they do have examples complex examples that use the raw vtk api so you can dig pretty deep if you get that far it's because you really need it the point is you have an easy way of getting started with very sophisticated 3d visualization that complements matplotlib matplotlib does basic 3d but it doesn't do these kinds of fast hard work accelerated rendering so that kind of wraps it up on 2d and 3d data visualization basically with python and these are the main tools that most of us use for everything from interactive one-off things to complicated plots for papers alright thanks a lot