 Welcome back. So, there are several questions people have I will quickly answer them and then get on to the other topics and some more exercise problems for you. How to find other existing? So, the first question I will answer is how do you find other existing modules like math module? Well, you cannot quite find them that easily in the sense that you cannot do a find or something like that. The best way is if you have installed any new packages you know what you have installed, but by default if you want to find out the standard library, there is excellent documentation available at the python.org I will show you the page docs.python.org. Unfortunately, the network for me is down. I am not able to show you that. So, I will write out the link here. It is a standard place where you get plenty of documentation on python. There is a full fledged python tutorial and information on all the standard library packages. From there you can find you can even search there and you will be able to find what you can do. Is there any test today? I think we will have one more little test today and then we will give you some assignments for the week. Next question, how can we import package from other directory than from current directory? Unless you install the package there is no way you can import it in some arbitrary directory. You have explicitly to cd into that directory if you really need to do it. So, typically when you are creating many modules, the general approach that you follow is you create a package hierarchy or you create a set of modules in a current directory and then eventually you install it so that other people are able to do it. But unfortunately this is a little more complicated. It is a little too advanced for me to introduce at this stage. But there is a mechanism by which you can install python modules on the system and I am not discussing that right now. Third question, is there any command for exporting and importing the history of commands worked in IPython? Very good question. So, there is this command called percentage hist. We usually cover it in the advanced sections next week you will be looking at it. But percentage hist, so percentage hist or history basically it input it prints out the input and output history for all of your sorry the input history for all of your commands type so far. So, let us try it percentage hist and now it gives me the line numbers if you do not want them you say minus n, it will tell you all of the lines. These question mark commands are not legal python and therefore, they will not be executable lines of code. Everything else is actually legal lines of code. But all of these IP magic things are all IPython specific. So, do not rely on them. But supposing I am typing print hello, notice that I will get the print hello. Depending on your version of IPython behavior may be slightly different, but the hist command percentage hist will allow you to look at the lines of code. There is another command called save. This will allow you to set us save a set of lines. Remember that every line of yours is numbered in 1, in 2, in 3, in 4. So, you can always create a file by saying save line 1, line 3, line 5 so on and so forth. So, look at the options. This will be covered in greater detail next week, but there are commands to do this. Third question, please repeat creation of user defined modules. I spent so much time on this. Well, all you need to do is simple. Create a dot py file, make sure that the name of the file is a valid variable name. That is, it is an alphanumeric not starting with a number, starting with an alphabetical character. It is case sensitive. So, basically alphanumeric with underscores, no dots, no periods. So, basically it should be. So, if you want to say module foo bar, it will be foo underscore bar dot py. Create a file like that. Then you can import that as or hello world dot py can be the module name. Then I can say import hello world. That is all. So, basically create a file dot py file with a valid variable name as the name, the base name of the file. Say whatever python code you want inside that you should be able to import it. That is the python module. That is all there is to it. The only other thing you have to remember is any code inside that is going to be executed. So, most often your modules will contain various functions and things like that. Now, you may not want it to execute any code and print anything at the time in which case you have to do all this name gymnastics. You have to say check the value of name. If name is main, you can print something. If the name is not main, it will not print anything. So, if you need to print anything when it is running as a python module or rather when it is imported, you do not want to print it. Then you put anything inside the main. But if you want to print, then I run it like an executable. If I say python hello world, you put it inside if main. That is all. So, the name variable is local to every module and is equal to main only when the file is run as a script. That is the only thing you need to remember this last line. So, if you want to run a script and produce output, it is a good idea to put that inside an if underscore underscore name equals underscore underscore main dunder. And typically people will call double underscore as dunder. So, you say dunder init dunder because it is easier to translate double underscore double underscore. They call it dunder or dunder main dunder means two underscores main two underscores. Next question, how to run python on windows? Well, there are plenty of solutions. Please do that I am aware of. There are actually three. You can just download the python executable, install it, you are done. There is another package called python xy which is free. There is an yet another package called EPD, n thought python distribution which is an excellent python distribution. But it is free for use by educational institutes. It is not free for commercial entities. So, you can download it, install it and you have python. So, all of those options are available. Please list editors and links. Well, there are plenty of editors and plenty of links. I think you can find them out for yourselves by using Google. I will not directly recommend any of them in particular because there are so many editors. There is eclipse, there is Wim, there is Emacs, there is spider, there are several. And if you search on Google, you will get many, many hits. And I would suggest you learn something like Wim or Emacs because they are generally useful to know. Maybe Wim or something like that you can use and configure it to run, use it with python. So, let us get back to basic python. So, here is a little module fib that now returns fib of 100. So, what I want to look at next is take a few more little problems and look at something called test driven development in little more detail. So, we will get back to the slides in a little while. But what we will do is we will take a simple problem and we will try to motivate for you the use, how do you use tests, why do you use tests and how to do all of this test driven development. How do you become a better programmer? I will take it later. So, what we will do now is we will take a few simple problems and I remember that I said I will solve one particular problem again by looking at it a little closer. So, the problem we will consider is again let us look at this Fibonacci sequence itself. We wrote this function and I want to know see how I can use a process or a workflow by which I am able to write this code and make sure I am not making too many mistakes. So, first thing is we already wrote this Fibonacci. So, let me just make some unnecessary mistake. Imagine this is a typographical error, somehow I accidentally made this mistake, I do not know or I said is. So, I have made this mistake. How do I know? So, imagine you are starting to write this application, you are starting to write this Fibonacci function and you do not have it correct, it is wrong. Remember we were trying to count the number of words in a file and we wanted to deal with punctuation and all of those cases. How do we now solve that problem? So, the first thing if you remember was I said you first think of you know the function you have to write and now you need to think of the corner cases to know you have to check right. The function does something I want to make sure yes this function is correct. So, firstly what is the answer? If I give you Fib of n what should I get? So, Fib of say 5 or Fib of 2 should give me what? What should I expect? It should return a list 0, 1, 1 correct. Why? Because it should print the Fibonacci sequence until it hits 2 and before that. So, if 2 is hit it should not print that 2 right while current is less than n is the condition. This is what I expect right. So, let us see what I get if I do Fib of 2 what happens? First I have to import Fib or better still let me do percentage run Fib what is happened? It is gone into an infinite loop. It did not even get out what happened? So, let me do it again. It is not even returning. So, I cannot even make a function call. I cannot even import that module because it is gone. Let me import it and see what happens import Fib. This worked because I have not called the function. When I run it it will execute the Fib inside main right, but when I import it it is not because I have carefully put the Fib inside the main. Notice only when name is main is it going to do Fib. So, if I run it it will run that code. If I do not it would not which is nice. Now, let me do Fib dot Fib 2 gone infinite loop. So, to get out I press control C it will get out. What do I do? So, I go back debug debug and said I made a mistake current should be next. Now, import would not work. I have to reload. So, let us just run Fib again. Now, it works. Reload Fib. Now, when I do a run the Fib module is available the Fib function is available locally because it actually runs that and gives me that the contents of the name space that is there. I think some of you may have been confused. There are two things going on here. When I do percentage run I Python will execute Fib find out the name space and put it back into the current name space. So, when I do run Fib it ran the file, but it also makes the Fib function available. Now, if I do for example, import Fib how do you view and modify pi code on prompt use VIM use an editor. One of the questions is how do you view and modify a dot pi file on the prompt use an editor any editor G edit tile VIM E max eclipse whatever you want any good text editor. Don't use Microsoft word or something like that that is not a programming editor. So, now Fib is available. So, I say Fib of 2 gave me 1 1 it should give me 0 1 1 correct. Now, I make a mistake again. So, now problem is if I check for it now I only check for it once two days later if I come back make some mistakes I do not know which test to run I do not know which cases I gave as input what is correct what is wrong. So, with test driven development what you do is you write what are called test functions. So, now I write one more function called test underscore Fib. Now, inside this I can call Fib. So, I will say result equals Fib of 2 correct. Now, if result equals 0 comma 1 comma 1 print else very simple. Now, instead of doing this print business here I will simply run. So, what am I doing? I just wrote another function which will call Fib with an argument that I know 2 for which I expect the answer to be 0 1 1 correct is this clear to everyone. Now, all I do is I will say run says error unfortunately it does not tell me more than that it simply says there is an error. So, what can I do? I can say I am saying I expect the first I got the second now let us run this. So, I expected 0 1 1 the result I actually got was 1 1. Now, I also know what the error is. So, now I can go back. So, this is a very rudimentary simple test, but notice that I am doing this. So, now if I want to say no I want to now test Fib of 300 I will have to write all this again if result is this do this. So, one way of doing it is to make that into another function then call this function and then print out the necessary message. It turns out Python has a very simple convenient way of doing this it is called asserts you use what are called assert statements. So, I can say assert the keyword result equals 0 1. Now, I will delete all of this notice that the code is much simpler. Now, it tells me assertion error warning failure executing 5. Now, one convenience of I Python is that it has something called PDB mode the Python debugger mode. So, what you can do is at this stage you can do you can use the debugger if you want, but better still you need to figure out what this result is. So, the nice thing about PDB is well the difficult thing is I will have to introduce you to PDB show you what you can do, but instead one simple thing you can try is you know that you got an error at this line how because it says line number 14 look at this look at this saying home IITB 5.5 notice this line it is pointing to you specifically saying line number 14 assert result is wrong. So, now I know which line is failing I can go back here what is the result I got 1 comma 1. So, the nice thing about this test is now this test is permanently there it is in the same file which means in the future if someone was to run this they know oh this code should satisfy all of these tests. So, now let us flesh this out let us correct it correcting the code notice that it did not give me any errors because the code is correct. So, I just run it no errors means everything is filed everything passed, but now I say I want to test something else I can say assert fib of 0 should be what fib of 0 should give me m t then assert fib of 1 should be 0 correct 2 new test cases I have created passes. So, it looks like my code is correct let us try something slightly more complicated. So, if it passes all of this I am pretty certain my fibonacci function is correct now I can call fib with some crazy argument and I know it will produce the write out. So, this is what you typically call as test driven development well there is a little bit more to test driven development when test driven development you will actually write the test first and then you will write the actual code, but I want stress on that too much the general idea is you write reusable small pieces of code like the one I have talked about here and you write a little test function to make sure that this code is executing correctly and what we use is asserts in order to do this. Now, this is the beginning of test driven development the instant you start doing this was 80 to 90 percent of your test driven development is actually over the rest is a set of tools that you use to make this more easy and automatic. So, there are some slides here about some other tools that are used to write tests I am not going to get into those tools I will use only one particular tool to illustrate how you can use test driven development tools to write nice code using tests and as an exercise we will use all that we have learnt to solve one more problem now in about five minutes. So, how do you do test driven development the basic idea is decide upon the feature to implement and the methodology of testing it write the test for the feature that you have decided upon write enough code so that the test can run, but it fails improve the code to just pass the test at the same time passing all previous tests run the test to see that all of them run successfully refactor the code so on and so forth go back to one. So, the idea is you write your tests write the code the feature first the test is going to fail because feature does not do anything yet implement the feature make the test pass make sure no other tests fail and now once you are all the tests have passed your new tests have passed your code should be functional. So, that is test driven development perhaps it is little too formal right now and at this stage may you may not understand or appreciate the value of this which is an extremely important process that is useful to follow. So, this is a simple example again I went through the same example in the context of our Fibonacci sequence this is just the same thing with a different approach of using the GCD function instead the same idea. The idea is you have the test code you want to check whether GCD of 48 and 64 is 16 if it is not 16 you print out a big message I have automated all of that by using these 4 or 5 lines of code. So, now they are slowly systematically going through that process to create the file create the feature create the function knows tests. So, there is this package called knows which makes it extremely easy to run tests like the kind that I have defined these tests as you can see are very easy to write it simply requires if you switch to the terminal. So, we look at this mean these are fairly simple to write is just a few lines of code and almost the same code you would have written when you are testing your code you had done Fibonacci 5 you would have seen the values the only thing you have done here is you have put this assert and you have compared it with the actual result. So, you have essentially. So, what this knows module does is there is this function there is this command called knows tests you just run knows tests on your module it will run the module run all of the test star functions check that the output is correct. So, what we will do is we will use knows in order to do this testing. So, let me just edit this file and make a mistake let me go back and make the mistake of 1 comma 1. Now, if I run this file it will say it failed there was an assertion error I expected result is this, but I did not get it. Now, knows test has a bunch of additional options I can say knows test minus help it gives me plenty of options perhaps the most useful supposing I run knows test minus D Fib dot pi it will give me detailed errors. So, it is basically saying Fib of 2 actually produced 1 comma 1 and 1 comma 1 is not equal to 0 1 1. So, as you can see I can I do not have to do any additional work I just do knows test minus D Fib dot pi it actually tells me oh you wanted this you got this they are not the same and I do not have to do inspection debugging print statement nothing I just do knows test minus D it says oh Fib of 2 is returning 1 comma 1 does this make sense this line is basically the result this was the actual error that is the line of code that was failing it is expanding on that and telling me that this is the detailed error saying that Fib of 2 is actually 1 comma 1 and it asserts that 1 comma 1 is equal to 0 1 1 which is not true therefore, it is failing. So, now I can go inside edit it now it says everything ran no problem notice that it says ran one test and saying okay. So, now that you understand the basics of testing let us put it into practice shall I repeat I will repeat myself one more time on what how you do this first what you do is we will take a simple problem okay I will put I will write one more function now for you. Let us solve this the solution to one of the problems that you had solved during your test which was factorial. So, I have created a file called fact dot pi so I will first define the function fact okay this is what I want to create I want a function called fact given an argument n and I will give it the documentation string now I will not finish this I will just put pass here and I will write the test case. So, what should I find now first assert fact of so give me something give me a test case. So, this is where you have to think of all the corner cases fact of what fact of 0 right what should factorial factorial of 0 be should be 1 assert factorial of 1 should be also 1 assert factorial of what should 5 be this should be enough right. So, if my factorial passes all this I am super happy so how do I test this well let me create another terminal. So, now you run no stress okay obviously it is going to fail because I did not write anything I just said pass. So, let us write our first attempt return okay very happy. So, this fellow gets a failure it says maximum recursion depth exceeded okay that is it kept recursing and it died essentially it went into an infinite loop why I forgot the escape clause everything works how about will this work may not work I do not know the answer. So, at least this function call should return correct there should be no error when running this. So, it is simple for me edges add one more assert and say okay I do not know what the values it should be some positive value okay error. So, basically python has a recursion limit you cannot recurs infinitely because it has a finite size stack okay. So, if you want to really write a good factorial you should not write a recursive. So, now comes the power of tests if I want to change the implementation in the past I will have to again go and say fact of 0 is 1 fact of 1 is 1 fact of 2 is 2 fact of 6 because accidentally it may work for that one case that you are testing whereas now I have all these 5 cases I do not have to type 5 lines I just do no stress and I know it is run correct. So, the point of good testing is once you write it you have finished next time you can refactor it change the implementation optimize it implement it in C in for whatever you want you can do. So, long as you can call that one function called fact does this make sense. So, now you can imagine if you have a large application and you want to make sure everything works you change one small bit here everything breaks you do not want to have these things happen. So, now let us rewrite this in a non recursive solution how do we do it how do you get the factorial of a number n into n minus 1 into so on and so forth right. So, can I use a while loop right. So, first I say I must first initialize some quantities I need a counter I say count counter equals one thing you might want to do here is you may want to say if n is less than 0 what should you do or should I raise an error I should actually say there is some mistake something is wrong. So, let us use the same strategy we will say assert it will basically say assertion error. So, if this quantity is true here n is greater than 0 it will be fine, but if quantity is not greater than 0 it will say I am sorry or greater than equal to 0, but that is fine. Now, we define a counter and we want a result I will call it f as the factorial f is equal to 1 while counter greater than 1 f will this work well I do not care I can always try now that is the beauty of having tests unfortunately the factorial number that I have given here is too large this factorial will be enormous which is why it is taking so long to run, but the previous code did not even get so far correct. So, let me put it out of its misery look where is it failed it is failing here it is not failing it is basically taking forever. So, let us let us make it let us remove this and make sure the rest of it works it works. So, our implementation is correct, but our test is actually a nasty test. So, let me make it smaller even this is a large number by the way this will also be extremely large hopefully it should finish we can wait for it in the meantime I could even consider writing a separate solution. So, I can say I do not want to do it with a while loop I want to do it with a for loop that also can be done correct. So, I could say f is equal to 1 the mistake here is settled range of n will not end at n. So, I have to do n plus 1, but well we will be lazy we will expect no stress to find out well even this is taking very long to compute because let us try if this works at all assert fact of 1 is not 1 there is one more problem in my implementation. I starts at 0. So, 0 multiplied by anything the answer will always be 0. So, I need to say 1 to n now let us see failures fact of 2 should be 2. So, why is this giving an error saying fact of 2 is 2 it has not given me the value it is not 2. So, basically I need to change n to n plus 1 everything runs and in fact this test also works increasing it by 1 order of magnitude is too much. So, as you can see I have written 3 different solutions to the same problem I only wrote my tests once. So, very often you will feel it is a pain for me to write these tests it is not you would have written these 5, 6 lines anyway when you are trying out the function first time all I am asking you to do is write those functions and put them in a separate test case and you should be good to go. So, this is the heart of test driven development the idea is you write a little test which collects all of the corner cases you are talking about put them in a test underscore something function like what is seen here test fact give that function that you are going to write or whatever you are going to write arguments for which you know the correct answer test if that works that is the only way you are able to assert that the function actually works or does its job. So, supposing you are building a car how do you know this car is going to perform because you actually perform tests many engineers do this unfortunately a software engineers we do not do any of this we just write programs cross our fingers and hope that it will work that is the worst way to write programs if I even build this chair we would not be sitting in this room if we did not do some engineering testing we will always do engineering we will say what is the size of the thing. So, that the load will be calculated you will calculate the load you will build a solution then you will test that solution by trying to break it you will over design it by making sure that the weight the size of the truss the size of all of the members that you use is large enough and then only you will sell it if not you may kill somebody. So, generally in engineering we tend to do this however in software engineering many people seem to be very lax when they write their code even research codes there will be no tests the code will barely run it will produce some numbers no one knows whether numbers are actually meaningful or they are garbage and the worst part is very often you will get garbage and you will not know that it is garbage you will get nice looking pictures oh this plot looks beautiful, but ultimately there will be some sign mistake there will be some stupid mistake you will still get the right answers, but it is actually wrong. So, if you write test systematically and you make sure every function is doing the right thing you will ensure that your code quality is very good. So, it is a very good practice and as I have shown you it is not very difficult. So, what I am going to do now is going to give you a little exercise. So, let us solve this one problem create a file called word freak the problem statement is this is your problem statement I will give you 15 minutes write tests and write the function and show that you are able to write this test driven kind of code approach. If you do not have knows installed simply create an if main if name equals main call your test function the assert errors will still be produced. So, first call the function let us call it now if you do not have knows installed you can just do if now please write the test some of you are saying that we have done it very fast well I have repeated myself and actually the concepts are very simple write a function write two functions I have introduced one new concept called assert and one new tool called knows tests. So, simply say knows tests whatever module you want to test just give it that name it will do the job for you. So, you have all of the information try to solve this problem using tests. So, remember this is the same approach I have been talking about all day a morning I said think about what are the corner cases think about the function and now once you are forced to write these kind of functions you will write nice functions because I can use this function know somebody else can call this function because it is well written I can give it a string it will give me the answer I will give you an additional hint what you present as the output of this frequency can be a dictionary use a dictionary it will help you solve the problem much easier what are you trying to do you have to find the frequency that means for every word how many times is that word occurring in the string you want everything to be lower case. So, convert the string to lower case one more hint use your good old split and all of the other functions you know in strings and just check for every word it is very simple use a dictionary to keep track return the dictionary writing the test for this is a little more involved, but it is a good exercise because you will force to think how do I write this function how do I make sure it is correct. So, I will give you 15 minutes please try to do this. So, I have given you plenty of time to write this now let me quickly run through the solution and I will make the solution available to you also. So, first thing we need to check is I need some test data. So, let me give some simple sentence some text is a b c a d e you do not even need so much. So, I will see a a b or a b c a b a. So, now we know the answers I know exactly what frequency I should get. So, I will say result is word freak t x t assert result. So, I will use a dictionary this is something I have left as a choice to you I will make sure that result of a should be 3 and finally, I do not want it to be get finding some random words I know there are only 3 words in this. So, the length of this dictionary should only be 3. So, assert length result should be 3. Now, obviously this is going to fail I run it because I have not written an implementation let me start writing out the implementation. So, first thing I will do is lower the case. Now, notice that none of my test cases here worried about the case. So, I will change the test case also. So, as to make sure I am not missing out on upper case and lower case that mistake is not being made. Now, I need to count. So, I will say the result is a dictionary words is lower text but split correct. Now, what should I do? I have to look at every word check if it is occurred or not keep track and do counting. So, I do a for loop for word in words result of word now the problem comes if there is a result initially dictionary is empty I do not know what word is there. So, what I will do is if word in result result of word should be incremented by 1 else what does that mean? It means the word is not yet there in the dictionary correct. So, what should I do now result of word equals 1 that is it. So, let me go over the code before that I will run it once first. So, my first implementation is correct and I know it is correct up to this point. Now, obviously I can make this more and more complicated add more test cases give it more meaningful text put new lines add more space put comma all of this nonsense I can do, but I know that this basic test will always pass should always pass because I have written a test case for it. I will go over the code once more so that you are able to follow I simply take word frequency I have written the function that is already done I converted to lower case I create an empty dictionary. Then I iterate over every word in my input all of them are lower case. So, I do not have to worry if the word is already there or let us consider the case if the word is not there in the dictionary word is not there in dictionary I will set I have found one word now. So, I will create an index in the dictionary saying result of that word equals 1 the counter set to 1 next time if it is found word is there in result I will simply increment the count finished and now a result return result. So, as you can see writing a solution in python is extremely simple in 5 lines or 6 lines of code I have been able to write out a clean solution and better yet in 5 lines of code I have written a test case which at one level even documents my code because now I know how to call this I know what arguments to expect and I know that my function is performing as per specification and it is very easy to add additional test I could create either I could create one more function if I want to or I can just add more cases here itself add some new line cut paste copy this and say this is another test with more complicated data. So, it is up to you on how you want to test it, but the idea is you have a test with which you can precisely identify where the problems one good strategy to use is to make sure that the tests are precise that they address a specific case. So, that you are able to debug the problem easily and as you saw the test passes very nicely means our implementation is correct. So, I will make this code available to you and I urge you to experiment and try out and learn how to follow this workflow that is of writing good tests and writing functions that are reusable. One thing you will notice is all the code I have written here is reusable because I have designed it such if it was not reusable I cannot even write a test. So, in this case I have written a function for which I can give some input get back some output and I can test that output which means if somebody else wants to use this function the same holds for him he can import my code call the function give it input it will give him back output. If I had written the same thing with print statements instead no one can test it because nothing is written it is only printed to screen and I have to manually inspect to check correctness. So, does this make sense? So, it is a strategy you should follow and it is a good practice for you to make sure that you are able to test as much as possible and it is not always easy sometimes you have to break your head, but it is worth the effort because your code becomes immediately the quality level of your code has improved because you have put in that much more effort and thought into your programming and ultimately that is what matters. So, what I will do now is we will stop with this today.