 Hello everyone and welcome after a quick break. We have a talk about static typing in Python by Sanjit. So I would like to invite Sanjit to the stage now. Can you hear me? Yes, as long as I can hear you. Hello everyone. Great. I can hear you as well. And yeah, feel free to start and let me know when you want me to share the video. Yeah, you can go ahead. Sure. So, okay, I'm just trying to set up the video and because of network issues, we are trying to do this pre-coded talk. So Sanjit will be present with us for all the questions. Feel free to ask Sanjit any questions that you have any time. We'll make sure that Sanjit gets your question after the end of this talk. Okay, so you have the hop in chat. Feel free to put the questions there and stay there. I'm just just a few more seconds. And yes, we have the talk right here on the stream. Nice. I guess you can see the talk and here we go. Hello everyone. This is Sanjit and I hope you all are having very good time. I'm also very excited to be here. Before we get started, I would like to thank Ficon India 2020 team for giving me the opportunity to present this talk. So as you can see my title that I'm going to talk to you about a started typing in Python. So without further ado, let's get started and see what's the agenda. So primarily I'm going to actually try and set some context about typing. And we will understand the world of different type of typings available in Python. And why do we actually need static typing at all? And because Python is a dynamically typed language and people love Python for this feature. And once we understand that how static typing works, we'll also understand that how to be static type Python code. And then we will jump on to see that what are some of the complex types available in Python's typing module, which we can use to actually static type our legacy or large code basis. Then we'll also see one important tool called type checkers. So I'm going to primarily talk to you about MyPy, which is one of the most popular type checkers available at this moment. And then we'll end the talk with some pros and cons of static typing. But one disclaimer I want to give that I'm going to talk to you about some of the basics stuff as well. So in case you're aware about some of the basics, please bear with me. I will be covering because there are some people who may not know some of the initial concepts as well. Okay. Okay, without before moving forward, let's also let me just quickly talk about myself. So as I said, my name is Sanchit and I'm primarily working as a lead software engineer at E-PAM India. And I'm very passionate about open source communities and Python and especially about Docker. And I'm a very community-centric person. I have been working with communities from last almost three to four years. I helped run a couple of communities that are called Hyderabad Python user group and also DevOps and SRE user group Hyderabad. And I'm happy to talk to you about anything with regards to open source communities, Python, Docker, or about, especially about Hyderabad. And you can find me on my social handle, which is in a list. I'm available on Twitter, LinkedIn and GitHub. So feel free to reach out to me and I'm happy to get connected. So let's get started with the talk. So let's revise some concepts like static typing, dynamic typing, and etc. So here what we see that this is a basic example of static typing. So I chose to use C code because that is a very familiar language for all of us. So essentially there is a function called main and there's a variable called age here, which is typed as int. So age can only accept integer values. So I'm trying to print that particular variable here. So essentially that's how you started typing code that your compiler would know that before runtime itself that there is something called age, which is supposed to be an int value. So in case you try and assign some string value or take a value, you will end up having an error here. So that's how basically I started typing scenario would work. Let's also understand what is dynamic typing. Let's try and revise that. So this piece of code is primarily written in Python 2.7. You can ignore that because typing is primarily available in 3.5 plus. But let's understand this piece of code. There's a function called foo here, which has two conditions. If a is greater than zero, then do something and if in the elsewhere do something. So essentially when I'm trying to call, let me also actually include the pointer so that I can actually navigate and help you understand. So now when I call the function called foo here, right, by passing the argument called to this guy goes here and the function and the condition gets fulfilled and it prints high. Okay. Give me high as an output. So this line doesn't get executed, which means that this type checking of this line is not happening when I'm running this function. Okay. So that's how the dynamic typing things work here. But when I call foo of zero, that is where the condition gets fail and then tries to actually evaluate the type of it. Which means if it is not being called, I mean, if it is not being evaluated, the type is also not going to be checked. Okay. So that that's the beauty of dynamic typing that you don't have to define the types. You may have some bugs in your code with regards to type of the objects or the variables, but you don't have to write that specific code by defining the specific type of variable, etc. So just a quick reminder about how does the dynamic typing thing works. There is one more concept called duck typing. Okay. There's a saying called if it cracks like a duck and if it walks like a duck, it is probably a duck. Right. So I'm trying to actually replicate the same scenario here. There is something called class as duck, which has a method called fly. Then we have something called bird, which is again a class and it also has a method called fly. Right. And in the end, I'm trying to actually iterate through both of these classes and trying to call the fly method. Right. But what you see here is that Python is not worried about that what is my class type. Right. It is also it is only looking about whether the fly method, which is available here or not. If it is not available, then it may give me an error, but it is never going to be worried about that. What is the type of that particular animal item here? Okay. So that's how duck typing also works. So till that, till this time, I want to actually just do a quick revision of how static typing, dynamic typing and duck typing works. Let's quickly understand the why should we actually care about types? Right. Why types are important? I mean, as we see, you saw in the initial example, right, that you define your tribes and then you will end up having not those confusion later on. So you can see what I'm trying to say that Python is a dynamically type language that you don't have to spend too much of time writing those types and define things. Right. But what will happen that let's say if you go towards a very large code base or your code is getting multilayered sort of thing, then you will end up having tough time figuring out those box or issues where in there is some data flow happening and you are having tough time to figure out that what this particular argument is about and from where this particular person is argument is being called a function is being called. So those type of scenarios which can occur, definitely Python is a beautiful language and it also has this feature called dynamic typing, but it also has some cons. So we'll also talk about those cons as well, but I'll talk about the cons of static typing essentially, but dynamic typing also has some cons. If you probably are from the world of Java or from other languages, you will understand if you have your code statically typed, it helps you actually find out the bugs much, much before if it goes to production. So that's I think that's why we should care about ties. Let's move forward and understand that with an example. So there is an example I've written a very basic example is a function called do something, which normally has four arguments here, something called user class client, retry count and data. And I'm trying to import my user class from the my module. So what you see here is that there are a bunch of arguments, but let's say if this whole stuff is available probably in a legacy code base and you are supposed to fix a bug here. You'll end up having a very tough time finding out that where these arguments are defined and what are the type of and what do I expect from these. Okay, maybe there is some code, but you'll end up having maybe have a tough time to figure out those issues. So that is why static typing becomes very important. And so what you will do maybe even before you don't think about typing, you will actually find and find out that if there are any doc strings available, maybe you will try and search for doc string and try and see that if you can make out something with that. Or in the other case you may also try and find out if there are test cases available. Okay, with the help of test cases, you can figure out that how the code is working or you may also have some bunch of other statements. So these all scenarios are there and possible wherein you can use those things to understand the code. But still, there is a possibility that you will end up having debugging the code and reading the whole piece of code base just for a single line of fix. So these are all possible scenarios and it can give a tough time to a developer who is new to the code or probably working on a very large code base. So that's why I think static typing becomes very important. And so once you actually write the same code with type annotations, which was introduced in PEP 484, 483, I think, if you static type your code using this way, right? So what will happen that you have your same function do something? It has an argument called user object. We have client, we have retry count and data, but it clearly specifies using the typing modules types available. So I'm trying to import list, double dictionary and set from what you'll call typing, which is natively available in Python 3.5 plus. If you use that, you can actually specify that what a specific argument type is about. So now if you want to static type your code, you can just apply a colon on top of your, beside your argument and you can define your type here. The same thing I'm doing here. I'm saying that user object is primarily user class and client is primarily a dictionary of string as a key and list as a value. The retry count is basically integer and the data is basically a tuple. And ultimately using an arrows and I can specify also that it is going to return a set of integers. So essentially it is helping me to understand the code right away and help understand that what do I need to do about these arguments? If there is something which is happening in my code and some bug is there, that can primarily help me understand those things. And in my personal opinion also, it's subjective, but it's my personal opinion. So rather than writing doc strings, if you write static type, your code right, it becomes much more cleaner and easy to understand. With those long, long, bright doc string, where you define your specific argument and whatever example, etc. But if you write these doc strings, it becomes very much easier to understand. And in any way it's going to help you figure out those bugs much before. I'll talk about how this thing can help. But before we move forward, I also need to set one example here that static typing in Python is not actually natively implemented. Python is still a dynamically typed language. And static typing is available as an optional thing. So we can use static typing, but it's often available. I'll talk more about this probably later in this session. Let's also try and understand more on the static typing front, and then we'll come back to that part. So let's see that there is another example called, there's a function called get employee first name. Primarily it is taking an argument wherein I'm passing a name and then returning the name after doing the slit of it. Then there's a dictionary called dummy name wherein I have first name and the last name. Then we have a user input, which basically takes input from the user and then try to return the first name by calling that specific function here. And in case the first name is not passed here, it basically called the same function again passing the dummy name. And then if I try and in there, I'm trying to print hiSunchit. So basically if I pass my name here as Sunchit Balsandani, so it is going to print the first name of my name. That's going to be Sunchit. So it can say hiSunchit. So the code looks fine, but there's a bug. And what is the bug here? So essentially if you see, if I don't pass anything here, and please provide your name, if I don't pass anything here in this input, what will happen that this condition would be true and now the function would be called as get employee first name with the dummy name. And dummy name is essentially a dictionary. And the dictionary doesn't have the split method. So if it failed, that's what is happening here. So it is possible that if I don't do that test case of passing an empty value, I won't cache this bug at all. And this code may go to production. And maybe after that sometime it may break because of this input. So how do we cache these bugs? So you can see that there is an error status coming and it's saying that attribute error dict object has no attribute called split. That's why it is failing at line number two. So how do we fix this bug? I mean we can fix that bug just by using typing as I said. So you can use the typing module. You can say from typing import dict, you can specify your dictionary here. You can also specify arguments in your function. You can say the name is always going to be a string value and return type is also going to be a string value. But the important thing is that I'm going to also say that my user input is also string and first can be a string way. But eventually this would make our life easier. So when I'm saying the name has to be an str value, that means a string value. So whenever this condition fails, if not first name, and I try and pass the dummy name which is a dictionary, I'll end up having that error in my type checker. So that can save my life. And I would know about that particular bug beforehand itself. So how can I do that? You just have to actually use that static type checker. Let's say if you use mypy, and you run that file called mypy 1.py, you'll end up having that error saying that getEmployFirstName function has an incompatible type called a dictionary of str and str, but expected type is str. So that's how you can easily find out and validate whether there is a bug or not. So these are some of the basic examples. Let's also talk about how do you annotate the variables. I think we also had an example in the previous one, but let's also understand how does the variable and function annotation works. So again, a very basic example. Again, we have something called the function called odd numbers, and odd numbers primarily have numbers as an argument, which is basically a list, and it also returns a list as an argument. But essentially, if you see that, I am trying to actually say that my odd variable is basically a list of integer, and then I'm trying to append something in odd. So if I try to append something which is not an integer, and if I run through this whole code using mypy type checker, I will definitely have an error saying that you are trying to append, which is not an integer value. So these are the things, I think, your variable annotation, also the function annotation that can help you figure out those bugs much ahead of your time. So I just wanted to help you understand that how these things work. Let's also understand that what are the different complex types. And why I'm talking about these complex types, because once you try and start implementing static typing, right, you'll also have opportunities to work on large code bases or scenarios within the type return types of functions or uses of those things are kind of little complex. So typing module provides a very comprehensive of different types. I'm going to cover only a few of them today, but you can feel free to reach out to probably look at a typing module and understand that how many different types are available. So let's talk about a few of them today. So there is one thing called union type. And it's very important and also very common if you work on large code bases. What will happen? Let's say there are two functions here. One is do something one and one is do something two. Both are accepting one argument called argument one and argument two. argument one is basically an integer value and argument two is basically an string value. But why I'm trying to actually specify that you may have a use case but you're not sure that what type of argument or what type of type you're going to return. It can be class A, it can be class B and this class and class B is just an example here. It can be any other thing. You are not sure about a return type of your function. It can either return a dictionary of some values or it can either return a set or a couple. So no matter everything is going to be an object. So it's up to you and according to use case but if there is a use case then you are not sure about what's going to be return type. So you can keep that thing in the union type mentioning that it can either be class A or class B. This is probably a bad example. I'm using classes as a return type. It should either be objects but let's assume that these are all objects. So either class A or class B would be returned from this function. So type checker would understand if your function is probably well written it has a return type as class A or class B so it will understand that it is a correct implementation. It's also possible that my class or my function would return none in some cases. So you can use none as one of the arguments here in the union. Let's also look at optional type. What's optional type? Since we talked about union and also we had none in one of the argument in the union type. So typing module actually supports that natively and in cases wherein you have one of the argument as none so you can use optional here. So what you see in this example that there is something called handle employee and the argument is primarily E here which is saying that it is optional of employee. So it can be optional or it can be none as well. So you don't need to define those default values. It can come as none. So depending upon that you can write your code and you can actually write the code and try and follow the logic. But optional would help you have that thing natively available. So you don't need to actually have a union of employee and none. So Python natively provides I mean the typing module natively provides an optional type which you can use and that will automatically understand that there can be a none coming up here. They can be a none. They can also be an employee but they can also have a none as well. So that's an optional type also very handy and useful. There is one more thing called any type which is very, very important because a lot of time you will not be sure about the type of any particular object. So in that case you can use any type. It can be of any type and it can have any type of function but it's not going to be a built-in. It's not going to be a completely built-in but any type of object you create you can actually have that as any type of type from typing module. So again similar example that we have something called do something and arg is basically a dictionary key is a dictionary and value can be anything. It can be any type of object be it a list or a tuple or a set or any custom object. So it doesn't matter. You can just mention any here. So for a lot of cases if you have to fix right once you actually start implementing your satir typing legacy code basis I ended up using any type at a lot of places because if you're not sure about something you can use that but it is always good to specify that what that specific type would be but any can be your I mean it can be your go-to thing if you're not sure about any specific type. Let's also understand how do we implement that type instead of typing so there's something called sequence available in typing module. So similarly I mean like I spoke about that typing that if there is a method available which means that if there isn't either method available you can also available in list type and also available in sequence type so that's why I'm able to actually iterate through the sequence of load and it works well. So that's basically example of duck typing. So basically if as I said that if fly method was available in both duck and word classes it was working fine. Similarly in the case of sequence also if it has a data method you should be able to iterate through it. So you can use sequence if it is required in any of the cases. Let's also talk about generics. So if you are probably from Java background you may also be familiar with the generics. So you can write the generic types. So I'm trying to create a generic type here and passing it as a another thing in the calculator class. So what will happen that I have something called item one which is an one type and then have wide input on the type. So ultimately the written type is also the generic. But it depends on me that if I have when passing one and two here in the calculator class object so I can do an add function and add method by passing one and two but it will work for me because I'm trying to create a generic here. That's how the generic concept works. I'm not going to explain how the generic works generics work but essentially you can try and create generics using static typing. I mean natively it is not available in Python but in the help of typing module you can create those generics and it will give you the same scenario which basically is available in other languages. So I'm creating a generic and passing the argument as one and two and it will end up having the three as an output because the type is is kept as a generic and similarly I can also use str here and pass as a generic type and it will end up having my string concatenation. So a very good example to understand that using this way you can implement generics and especially when you work on those multilayered architecture you have your interfaces etc it is very handy to have these things available. So this is about generics now since we understood about how static typing works and how do you implement static typing along with some of the complex types these are some of the static type checkers available primarily mypy, pyree and pytype mypy is primarily in your pythonslap and also pyree is from I think from facebook and pytype is primarily from google but pymypy is the most popular one which was developed I think at Instagram so let's understand that how do you use mypy I think I already gave an example of how does mypy work but let's also understand quickly again you can just quickly install mypy using pip install mypy and any script you want to actually run through which is already statically typed you can just say mypy script.py once you say that so you will end up having those those errors so I am just trying to show you another example let's say this is a function called myfun and it has a statically typed thing and you can try and pass 10 value which is likely accepting an integer value sorry string value but I am passing an integer value I will end up having those errors here so it says that argument 1 to my function has an incompatible type call in so that's how the mypy type checker is going to give you the errors beforehand that's how you can use my type checking in mypy and there is also something called monkey typing so once you actually work on a legacy code which is not statically typed and you would not want to actually spend a lot of time to type check those code or add annotations you can use this project called monkey typing which can generate stubs for you so I would recommend you to use to read more about monkey typing on this link which I am going to share in my slides so essentially it is going to generate stubs and using those stubs you can apply annotations on your legacy code base that will help you start to type your code and essentially your legacy code which is already written but not statically typed and I would recommend if you are writing a new code you should start writing type code from this scratch so how do you approach started typing so essentially if your project is very new you can 100% you can use started type code because it is going to help you a lot with regards to finding out bugs later on and you should definitely include that in cf so any type checker you can also talking about mypy or pyree or others you can use those in your CI so that before things are going up I mean before your test cases are being run your type checkers can actually run and understand and help you figure out those bugs beforehand else if your project is not new it is a legacy code base or a very large code base on which you are supposed to apply started typing you can either use monkey typing as I spoke about because that is going to help you to generate those annotations automatically and also you can actually start working in the large code base when you are working in the new code right you should actually try and start to type your code because gradually if you can build up your starting typing it will help you in the longer train and again you should definitely implement a CI also for the large projects or the older projects and I would recommend that you should start using started typing only for those piece of your code which are kind of most commonly used so that the bugs are kind of can be can be little less so I would recommend these approach if you are planning to use started typing in any of your project yeah so let's also talk about pros and cons before ending the talk so as I said in the start that started typing is definitely going to help you to document your code it makes your code much much cleaner and it also helps you write list is because basic data flows wherein you are trying to test whether a function exception in pj value or a string value those type of tests can automatically be eliminated because you are going to implement started typing and type checkers are going to take care of that and as we have been talking that you can cache those bugs beforehand for sure and refactoring also when you are probably coming back to the same code after let's say 2 years the refactoring of your code would also be much much easier because you are going to use started typing and it is going to make your life easier and certainly there are some cons as well so let's also talk about those cons definitely once you import typing module it is going to come up with some import time it is going to add some slight delay there but yeah that is understandable because typing is also heavy module and there are a bunch of things being imported so every time when you do from typing import this a lot of things get imported so that is a downside and also definitely since Python is dynamically typed that you don't have to actually add a lot of types at this moment but once you start working on your started typing you have to spend time you have to understand that what is the type of going to be for a specific function or return type or an argument so those things kind of take a lot of time and if you are working in your legacy code and trying to generate desktops those things I think kind of add a lot of duplicate files you may not want to be there because it makes your code base a little mess but yeah I mean if you definitely want to implement typing I think Stubs are also on monkey typing is also one of the good way to do that so they are taking two sides of the coin always that there are some pros and cons available as well so I think with that I would like to conclude that if you definitely try started typing it is definitely very helpful and especially for new projects I would recommend using typing and you can explore these things I have referred some of the blogs and also some of the videos you can actually go here and understand some of the good videos and blogs I found I'm open for any Q&A thank you hello everyone thanks Sanjith for the great talk and I can understand that it was recorded because of all these network issues and for those who are asking about the link to the video just to make sure that we will get the entire video that you are seeing right now not that video but this entire stream with the Q&A and everything and we will post it on YouTube so you follow PiconIndia on twitter and all the other social platforms that we have and when it comes to the presentation that Sanjith was using Sanjith will share it on Zulip later on and over to you Sanjith we have some questions let's quickly take some questions do we have to use typing for example can we use list instead of dict capital D and capital L yeah so those classes are available in typing module which starts from capital L and capital D these are specific to typing module you should not compare it with available default native classes like list and dictionary you cannot use them in compared to native ones the second question is does static typing checking slow down program execution yes it does up to some extent so I would recommend that you can use some performance tool or you can also use the off time tool available to check that how much time it takes but definitely it comes with some downtime I mean it comes with some time so whenever you do import typing it's certainly going to take some time but yeah there are some cons available as I talked about in my cons like but yeah there are some benefits as well so in case you can actually manage those times so you should use static typing we'll take one last question because of this time constraint and the last question is would not using any that is capital L they really defeat the purpose of type checking sorry capital A yeah so I think you are partially right that any if you're using any so as I said that you should be cognizant about using any type you should first of all try and find out that what's the type is about and then you should use that specific type but yeah in case you're not able to find out any specific type of an object which you are not sure about then in that case typing modules provide you this way of using any as well so yeah it's up to also very dependent on the use case you are trying to implement such it and rest assured for rest of the questions such it will be present on hopping chat as well as on julep chat that is 2020 power slash stage power slash deli so make sure you subscribe to that stream such it will be present there for all your questions and he will reply to the questions on hopping chat which are still present there so thanks a lot Sanjith thank you