 So, we will be building on the python we have already learnt, probably it is a good time to take a few minutes, if anyone has any doubt on the python you have tried out any, anything this would be a great time to get a quick refresher going so that we can get on with the topic of the day, okay. We are going to go through a lot of stuff, but like I said we are going to go through it interactively. So, you should whatever is there after the dollar please remember that says type. At the dollar prompt type ipython-pilab you should see some announcements and followed by an in square bracket 1, so at the prompt do ipython-pilab it will make a lot of noise and then finally give you a prompt in followed by a number within square brackets very strongly advise you not to write down. As a trainer my opinion my experience you are welcome to ignore me of course it is a free country please it used to be so you are welcome to write down, but if you want to learn I would extremely strongly suggest that you forget the existence of things called pens and paper use your brain and the computer, the computer is optional, alright. This way you try you talk to me when you write I do not know of any paper that is as interactive that talks back to you since we do not have that yet maybe we will arrange for it in the 2025 workshop till then I suggest you close down whatever notebooks you have brought tries to lose the pens type and talk you are lot more likely if you have a problem talk about it you will retain lot more than by writing down because whatever is written down there is only one use for it namely forgetting where it is after half an hour you are not going to index it there is no way to refer back and find out something in the written notes standard operating procedure for any written notes is it is written never read so we will not do that alright so what is that we are supposed to do type these two commands at the prompt I n prompt p equal to lint space minus p i comma p i comma 100 and then plot p comma cos p if you are following me p equal to will be i n 1 and plot would be i n 2 now what is p how do you find out simply type p at the prompt and hit enter see what happens what do you get arrays sorry we are all divided by the same language the way I mispronounce English is different from the way you mispronounce English so p is an array of 100 points so you could check what is the first point by p of 0 p of minus 1 will give you the last point lint of p will tell you how many points are there unsurprisingly it will tell you 100 alright so it does not take a rocket scientist to figure out plot plots to arrays where are two arrays coming from p is an array and cos p is also an array p is an array of 100 elements cos p an array of the first element is the cos of the first element in p and so on so it is essentially plotting 2 arrays but it by default the default properties it plots a continuous line rather than a scatter diagram types of course all of this is changeable the color it plots with this thickness of the line everything is changeable we will see that now we are using ipython which is an IDE of sorts for python so ipython provides some additional help to learn python the most important is that question mark so you type linspace question mark at the prompt ipython will tell you what is linspace so presume we already know how to plot we understand that we are plotting 2 arrays and we are plotting using the default plotting characteristics which as of now is simply a continuous line and in blue of a certain thickness and we also see that ipython has one convenient characteristic namely it will tell you something about the function if you do linspace or whatever question mark alright can we go on now we would like to change the color change the thickness and so on do a clf and guess what clf does it clears the plot area do a plot p comma sign p with r in single quotes you should get the sign curve but the plot characteristics we are changing now instead of the default characteristics we are getting the color characteristic is changed to red by that r we have one more line width again no prices for guessing what characteristic that changes it will produce a thicker line and you can give a dot instead of a continuous line it will plot points if you make any mistake the best thing you can do is stand up and talk about it because some other people have not intelligent enough to have made the same mistake so they will lose the opportunity to learn from that so you made a mistake stand up say what happened that would be the most convenient thing you can do for all of us to learn the point is r under dot does not give red colored dots prefix the dot with an r put it in a single quote rather than give two strings that is one of the nicest things about working interactively you can make a few mistakes I python will grumble sometimes even call you names but you ignore it it is a bloody program after all and then you can go ahead and learn much better than writing believe me when you try it out you will get lot more lot faster and for the record I do not use that much of python for scientific computing so some of the like say Prabhu or Madhu so some of the minor some of these things I have not done so I will I will give you a very similar answer try it out and r dot also works that is what he said that is what he said which brings me to another point when somebody says something please drop what you are doing and listen if you are going to make every mistake yourself and learn we will be here till 2015 it is a wonderful idea to learn from others mistakes that is a more intelligent way in my opinion so when someone talks drop what you are doing and listen to what the person is saying this will reduce the amount of things you have to do on your own anyway dot r r dot the issue is that if you give two separate strings it would not work give a single string it works as expected additive that is all and as usual plot question mark will tell you lot more about plot than you ever want to know I am not joking just look at it it will give you so many things three four pages or if not more so that was one type of doing some extra fittings to a curve or a plot let us do some more like adding a title unsurprisingly the command is title when you say title it adds to if you know latech you know latech produces lovely output for mathematical expressions in particular you can add latech expressions to title the presence of a dollar tells you that tells title that you are using a mathematical expression that is all alright can I move on you can add labels to the axis x labels adds the axis for the label for the x axis and y label once again you can use latech expressions essentially embedded mathematical expressions using a dollar for where any title labels are expected you can also annotate meaning draw attention by marking a particular part of a curve or the plot by giving some annotation text for example in this parable of you have said this is the local maximum and the x y equal to command says where to position that text and that x y numbers are in data coordinates not in any pixel size or anything the actual values you are plotting 2 and minus 1 or in the context of those values alright let me repeat the last point these x y values are what you are this 2 and 1 is the x y values in the data set it is nothing to do 2 inches or anything in the graphic output space I would like to call it as the data space whatever data you are plotting the 2 value in the x and minus 1 value in the y wherever that is the corresponding position because you can scale you can do different things to the chart but the values will remain alright can you move on now if you look at it so if you look at this there are some we can improve this by adding a x label y label we saw that let us let us add that so you can see all of that appearing on the screen but there is one area in which the graph is still visually unappealing there is a small bit of real estate here there is a larger bit of real estate here and this is touching the axis so if you are going to do a present this graph you would rather like that part to be much better correct I am choosing the problem in the chart as I have presented in a different chart you may want to chart only a certain area you may want to highlight so you may want more control over which parts you are charting correct so for that purpose we have two commands they are the x limb and y limb now if I say x limb without any arguments it gives me the answer the value it is using is it is plotting x from 0 to 7 please remember we never said anything like that you simply said p equal to minus 0 to 2 pi it decided to go from 0 to 7 similarly y limb tells us minus 1 to 1 now these functions tell us the current limits are the current axis limits of the chart we can change by the same function we can say x limb if you give a two values it will use those as the new limits sorry this is what I did I set up a new x limb now let us look at the chart you see there is a uniformity the extra space here has been chopped off right then but this part is still touching so we will change that by doing y limb this is the first command we did for setting up x limb then we will do y limb also I know it is minus 1 to 1 so I am doing this so you can see there is a bit of a space all around understood so x limb and y limb are an interesting type of functions when invoked without arguments they tell you what the values are when invoked with arguments they set the values if you are used to programming in say something like java you have the idea of a getter and a setter in an object oriented context getters get you the value of variables setters sets the value of those variables x limb y limb are both getters and setters without arguments they will get you the current values if you supply arguments they will make them the new values understood this sort of concession of notation is something that permeates python because rather than you have to invent two new names and all that nonsense x limb is x limb without arguments obviously natural meaning is I want to know what the limits are when you give arguments the natural meaning is I want to set these limits for x so there are not two separate functions for that purpose can you move on now this is all nice and dandy to do all this stuff but some of you would have discovered you would have made a typing error so you have to start from the beginning so what happens if every time you have to type that is not going to work our aim in life is not to create such beautiful pictures and look at them we obviously want these are similar charts to be produced to be part of a document okay so the way we normally work is interactively try different things figure it out once you have done it we do not want to reinvent the wheel we want to be able to say next time I want to plot this I do not want to go through all this typing I do not want to go through this mistakes correction simply it should come in one shot that is what we call a script we can capture whatever we did into a script and run the script there are two ways of doing it one is you can open a text editor remember what all you type type it p equal to lin space plot v x label y label title x limb y limb. Save it run it everything will happen that is not interesting why should you retype why should you repeat work you have done interactively you have worked so far there must be a way to use the work you have already done and there is I python provides you the command called percentage hist so whatever you typed encoding all the mistakes you made the commas you forgot the times you decided lin space is better spelled without a a all of that is available remember not to exit next time I do not know does history persist across sessions write out I think it is settable I think it is settable in ipython whether history persists or not if you have done the shell scripting you know there also shell history is persists I do not know whether ipython history persists I remember it persists does not persists I think it is settable I will look it up and tell you but typically the way we want to work is we interactively because you do not want to remember these commands believe me that is not the way that is not required which is why I am a little harsh about please do not write down stuff if you have to remember these things the game is lost so interactively you sit down do type half the commands hit tab hit question mark figure out try it out then at hist gives you what is available so if you run at hist you will see that the commands you have produced are there your type so far are there now the next step is to save that command into a script rather than type again correct and that is possible that is percentage save percentage save plot underscore script say get the output for the hist percentage hist then percentage hist 5 also I get but the last option that percentage hist 5 the previous one I did not get it what error did you get invalid literal for int did you type 10 or 1 oh yes 5-10 is a spelling mistake she cannot please make a note to be fixed in the archives 5-10 is not required 5-10 space 10 alright thank you for the spotting that now you can save the commands you have typed into a file percentage save is the command for it then you give the name of the script in you want to save she cannot move the blocking somebody percentage save plot underscore script dot py says save into this file 1 space 3-6 space 8 says save line 1 lines 3 4 5 6 and 8 they are not particularly magic number you have chosen some example to tell you so in real life what you will do you will look at and then say I want this you will say I want to look at these commands produce whatever I wanted to produce right but what if I want to reproduce this chart right with the title with the x label everything what all do I need I lean to correct I need 3 4 5 6 then 16 and 17 correct so I will save that how would I save that let us remember the numbers because my screen is not big enough to keep both so it is 2 to 6 and 16 and 17 so it also tells you the following commands were written to this script so this represents a very canonical way of doing things you interactively try out different stuff once you have figured out once you have got it the way you want save it into a script then you run the script it will repeat whatever you wanted to do or you may want to modify it for different parameters different things it is lot simpler to edit a file than to retype everything understood so how do you run that how do you know it is there it was not there already we do not so let us do it again right save it run it see remember to do a CLF does it produce a chart does it no it won't because there is a significant one important difference between running a script and typing interactively whatever you type interactively will get shown in a plot area without you doing anything but when you run a script that is not going to happen why the script may not even run with a screen correct this may be part of a large program you write which is running off some server which doesn't even have a monitor it may be generating this data of a cluster of computers which is somewhere else so when you run the script the assumption is not that there is a monitor available the assumption is yes the curve is generated you want to see it you tell me how do I tell how do we tell it because chances are you will rarely want to look at the output of a script you are more likely to want to save it to a graphics file png file jpeg file or whatever rather than look at it looking at things is something you are more likely to do interactively for that and the other reason namely when you run a script the script is capable of being run off a machine without even a monitor there is no default display attempted by any python program you have to do this show to show it you need not do a show you may directly save it as a png jpeg file we will see it in a minute how to do that which is what the slide explains when you run a script you are not in interactive mode so the default activities of the interactive mode namely update the plotting area after every command those things do not work you have to explicitly say show and chances are like I said you may not even want to do that because you may want to simply save the output this is the way so you will take that add save fig to the last command then your script runs produces a chart or a plot saves it to a png file in this case and exits ask away ask for the mic and ask away if you write the python code you want to plot some figures we need to end the python code in show function it will plot yes you can add show as the last command in your script yes yes that is the idea since we had not discussed show till then you are doing it interactively but show is available for the script also but like I was saying chances are low that you will run a script to see a chart interactively but yes show can be made the last command in a script so at the end it will show but you will you are more likely to have a script without having show or save run it do a show interactively to check save interactively because you may want to save with a different name and so on that is more usual use case so save fig takes one argument based on the extension given it figures out how to save it the extension supported are many you supports eps those of you who are used to generating latex documents will know eps is embedded post script and generally it plays well with latex documents you can generate eps graphs also alright let me summarize this the way you work is interactively you try out different things once you have figured out what to do you use the his command to look at what commands work because you would have made some errors you would have made try two three values and so on you use the his command pick out cherry pick the correct lines put them into a file using the save file command and then you are ready to repeat that set of actions every time you want understood any questions it supports like I said a large number of formats png pdf ps eps svg eps is likely to be one of most interest to you particularly if you are writing documents in latex already we have created a file called python script dot py there we are having some four lines as I said if I want if I want to include show if I again create I have to create a new file or if I use the same file it will be overwritten how to open that line in that file open it in an editor how do you how do you add to any file it's a normal file in your operating system nothing special you created it through the save command other than that it is nothing special it is like any normal text file or any normal python file if you are trying to save from my python obviously it is going to overwrite it is it is now a regular text file you want to add something probably the preferred method is to open it in editor and edit rather than retype the whole thing in or run another save command sorry run another save file command you can do that you can type a show and then look up the previous command and do it but yes it may be simpler to edit but let me repeat the idea is to work interactively cherry pick the commands save it into a script and then name it nicely so that it is reusable for you understood now one thing we have been doing is we have been issuing different commands all these commands are additive if you notice when you type the x label whatever was on the screen remain the car was there while I will remain title everything was there in the same way if you to put two plots you will see that both appear on the plotting area try it out but once again this is not anything special you should expected by now because this is what you have been relying on every command simply adds to the existing content on the draw area x label y label title annotate so plot is yet another command so that also is additive alright sometimes that is what we want often in experiment we may want two different related experiment data to be on the same multiple colors and as you will note ipython rather matplotlib which is working behind the screens for us takes care of changing the color for the two plots right if you do two plots both appear on two different colors you did not say anything for it this is one of the things we often talk about in python the defaults work by and large type in english what you want 95 percent of the time that is the command you want a title type title you want x label type x label just think what you want to do type a command chances are very high you have got 90 percent of it right and it will work as expected there is you do not have to worry about colors if you are adding two plots ok this is often nice behavior because this is what we want we want two plots in the same area and so on but sometimes this is not what we want what we do clf clears I want two charts I want the same data I do not want to type everything right I may have this x equal to lin space so much of work done I may want to plot flight you two different things those sort of situations you have oh sorry before that when you plot multiple things the legend is a useful idea normally when you are plotting only one you do not need a legend but when you plot multiple curves in the same one chart you would only want to annotate saying the green one is a sine curve the blue one is a cos curve etc right or in this case one is with ten points the other with a 500 points so if you do this legend by default and see there are actually two here now a legend notice that I python python mat plot whatever you want to think of it as takes care of keeping track of which color goes with what legend nothing you do not have to remember any of that you plotted two so your job is to ensure that you give it to the legends in the right order the first string must be for the first plot the second string must be for the second plot that is your job connecting with the correct color it will take care all right also the positioning of where the legend is is what mat plot flip thinks is a reasonably good idea normally it tries to hit a unpopulated region but sometimes it may or may not coincide with your idea where the legend should be so you have the option of changing its location notice that unlike other plotting elements legend is replacing it is not additive you do not see two legends again common sense you will have two labels but you will rarely have two legends for a chart so if you give two legend commands chances are you are trying to correct rather than type two legends so python takes the common sense decision and says throw the old one put the new one if you give two legend commands there are other options location again right left try out different things best fit and so on it will decide where it goes you can also use the figure command to do the other thing we are talking about I want to see two three charts decide which one to pick if I keep overdoing overlaying I am not going to get the best idea so there is a command called the figure command which essentially says whatever follows applies to figure number one if you give a figure one command it essentially says from now on whatever I type update drawing area one figure two from now on whatever I type update drawing area two and so on so you can switch from one to the other give commands essentially it is setting the current workspace if you want to think of it that way so the figure command sets the current workspace so figure one says here figure two says there whatever command follows figure command by itself does not do any output it nearly decides where the output of the next command is going to go in the previous one the legend just fixes it into the center center right or left but I want to fix it on the right most corner how to fix that location is it possible I use the x y it is not working how do we answer such questions how do I mean I am not here when you go there how do you answer such a question how do you find out the answer to such a question alright can we get back to the figure command the figure command essentially decides where the output of the next plotting command is going to go that is all so you can have multiple figure commands given you can have four or five any number open at the same time typically you will find two three suffices for all your practical needs the most common use cases you are trying out two different plotting possibilities or colors you want to decide which one looks good so you do not want to type the commands for lint space and all the other functions again and again just the plot commands you want to look at in two different things if you overlay obviously it is the same plot it will merge one over the other you would not see anything so you say a figure plot figure two plot look at the two decide which one fits your purposes better agreed you can also run say fig commands again that will also be associated with the figure the close command closes the currently active figure and shifts to the next one you have three active and you are in two you close what happens you figure it out so you can do subplots typically you may see four five plots inside one log log one log semi log all that stuff subplots are the way to go so essentially you are specifying how many you want subplot two one says I want two two rows in one column so two one one says I am referring to the first one two one two says I am referring to the second one try it out and see how it plays out once again these are based on what you need to produce sometimes the subplot is a better way to produce two sometimes you want to independent ones you choose based on what is more appropriate for your purpose all right so between default overlaying the figure command and subplot you can get any permutation combination of whatever suits your purpose all right can you move on but we rarely want to print these nice looking analytical functions who wants to print Shinax and Crossx we all know how it looks like they are rather boring you are more interested in plotting data you have right and that data obviously you are not going to type okay so chances are that data comes from some other program or a system and is available in some format it is probably available as a particular data file if the data file obeys certain rules or can be that data file can be massaged to certain appropriate formats then you can directly ingest the data into a python script you can read the data in one shot through something called load text you should have a file called primes dot txt in your so let us first download the first download this file primes dot txt from Moodle go to Moodle there is a resource called file somewhere from there you should be able to download this file primes dot txt open it in a text editor don't edit don't change anything view it in a text editor and convince yourself what it contains is ordinary text data all right so they have even got the primes dot txt file some answer in a language which does not use silence would be acceptable have you all downloaded primes dot txt who hasn't and people say yes if you keep quiet you expect me to have ESP to figure it out if you print it you will see something like this right so it contains data one column at a time nicely have you all got it can we move if you have this file now we want to ingest this data inside so we simply issue a command load txt and that gets us sir excuse me sir there the text file is written in column wise primes dot txt they have inserted number column wise if we insert the number in row wise in space is given this print is giving a list in the primes dot txt the numbers are written in one column that is one after another one data per line if we write it in one line it is producing a list the resultant of out print primes is a list load text is not think about it this way if the only use for load text is for one column data will it be useful is how frequent is one column data in real life actually load text will load multi column data when you give it one line it is looking at that many column or data only one of them that is all load text always gives you a list a list of list to be precise it is always giving you a list we have just in order to explain load text we have chosen a slightly unrealistic example of one column data which does not exist in real life you need at least two x and y for any meaningful data but before that before complicating the whole thing we want to show you with one column data the basic syntax of load text that is all you have to actually give the complete path if it is in the current directory you do not have to since in my machine it is not in the current directory I am giving the complete path so now what type of data is primes it is a list in other words you had your 25 data items so it is ingested it into a 25 item list now in a file what is the first data one in the first line fourth data one in the fourth line in a list what is the first data what is the fourth data that correspondence still remains correct when you print primes the way to display an array is like this array inherently is not horizontal or vertical right so the data is still in the same format it one important distinction the file data was all integers this is all float by definition load text and entire python data processing assumes you are dealing with float data which is most likely the case and unless you are doing heavy computational number theory you are unlikely to be dealing with integers so you can understand how time saving this is simply ingest some data obviously very rarely you will have 25 programs in the typically talks in terms of gigabytes of data which he produces after running programs for couple of months and then runs another set of programs on those gigabytes for a couple of more months producing even more gigabytes of data boggles the mind anyway so such data is very easy to load into a variable into a program very simply we chose the simplest example supposing it was two column layout what would happen let us try it out let us try it out the simplest way to do anything is to try it out so let us create another file so I created a file with three data we do not we do not need 300 pieces of data to test something you get a two dimensional the first line is here the second line is here third line is here and so on which is this is more useful you are more likely to have multi column data than anything else and even thousands of lines of data particularly if it comes from another source all you need to do is save it into a file and pull it in one question so if we do type P and we are getting numpy.nd array what is the question question is like what is that type like so I know it is in coming in array but I do not understand the prefix that numpy is the numeric pipackage which is required in the word array has a very special meaning we will take a minute to understand that have you done lists already for them this is already done so why is it called an array and not a list it is still in the same square bracket right why is it not a list why is it an array so logic tells you there is a distinction between a list and an array arrays are not an or not a built in data type in python lists are but lists are heterogeneous data structures you know the papers we have distributed are magical whatever is there will be there even when you go to the room so you can read it then also obviously Mr. Ebenezer does not believe me it is magical he believes unless he reads it now it will disappear that paper is magical paper whatever is printed there will be there even when you take it to your room ok it will not disappear normally I refuse to distribute paper for this reason because this is one of the few times teachers behave like children they cannot stop reading something fantastic it will disappear I will not learn it if I do not read it now let that idiot blather something on the stage who cares remember how you would feel if your students are when your students do it you also know where they are learning their bad habits alright so a list is a heterogeneous data structure it is a built in but it is not the most convenient for the type of work we are doing now for scientific computing we do not want heterogeneous data structure we are wanting to do high performance numeric computation so python by definition by default sorry does not have anything to handle it the list are the only data structure so we use external libraries numpy which stands for numerical python is the first of such libraries which provides specialized data handling data structure the first of which is an array an array is not part of python an array is a homogenous data structure it is assumed to all contain floats plus an array is normally immutable so by making these additional assumptions about what is in an array what operations are expected of it python can generate much more efficient code so inverting a hundred by hundred matrix will actually happen in seconds instead of days if you use lists because the list supports a variety of operations lists are expandable lists can contain lists which can contain lists each list can contain different number of elements so all of this flexibility comes at a performance computation cost none of which is necessary when we are dealing with experimental numerical data so numpy gives you that data structure called an array so when you use load text you are creating a numpy array and not a list agreed so this is many things that happen under the table some of the things you have to make it happen yourself which we will talk about a little later in the program but this is probably a good time to worry about our ingesting for rather than python ingesting data alright any questions before we break for lunch suppose I want to get the same data type that is stored in the file usually file in file will store characters since it is a text file so I want to get back the characters instead of float I want to get back it as a character why would you do that in scientific computing don't use load text read file the normal way have you taught them read file yes read file the usual way this is saving you lot of typing please remember load text is a shorthand for read a file line by line parse it into how many or many pieces are convert the data type of all of them into float load them from a list into an array all of that line some 40 50 lines of code is compressed into one for load text so load text as a use case if that is not what you need don't use load text so then how to find out a mode of a curve who is he that's a joke what is the mode of a curve there should be a which part of which packages you expect to contain it make an guess statistics there are statistical packages yes so there is in sci-fi mean medium what is there I think we will look it up we will look it up there is the we will go into what are available in numpy sci-fi a little later here we are just dipping the toes in the water so we will go through a more thorough a little more detailed look into what are the various functionality available in sci-fi and numpy a little later sir if you are having multiple columns and if you want to have 3d plot is it possible 3 dimensional plot 3d plot not in matplotlib but yes 3d plots are in the Prabhu Ramachandranan specialty it is possible with pythony it is possible very much possible matplotlib the underlying library we are using now is a 2d plotting library but there are other 3d plotting libraries which you can use matplotlib also does some 3d some 3d but I mean it is not in the I think it is in the what you call the underdevelopment version not in the regular version not yet released if I remember right victory was talking about that to for our purposes the plotting all you saw whenever you type plot you would have seen some long string saying matplotlib xyz that matplotlib is the library which is being used by python to do the plotting so matplotlib is a 2d plotting library so you need to invoke a 3d plotting library to get that done and let us see whether we have some time to show it if we have we will maybe demo mayavi mayavi is lot more than a 3d plotting thing it is a full-fledged high muscle power visualization tool kit I know very little about it I know Prabhu wrote it and I am very carefully keeping my knowledge to that level