 for making this possible. So next up, we have Dustin Ingram. He's a developer advocate at Google, focused on supporting the Python community on Google Cloud. He's also a director of the Python Software Foundation, a maintainer of PyPy and an organizer for the PyTexas conference. So welcome, Dustin Ingram, very glad to have you here. Thanks, Jason. So I could probably guess for that intro, but where are you streaming from? I'm based in Austin, Texas. Austin. Is it hot and sunny there? It is right now, yeah. Awesome. Well, I will turn it over to you and awesome. All right, let me share my screen. Does that look good? Okay. So hey, like he said, I'm Dustin. I'm a developer advocate at Google. I am a director for the Python Software Foundation and I work on PyPy. I'm not here to talk about any of that today. I'm here to talk about static typing in Python. So quick pop quiz, is Python dynamically or statically typed? And so it's sort of, think in your head, maybe what is the answer to this question? So the answer is that Python is dynamically typed, but it can optionally be as statically typed as you want it to be. And so that answer might not make sense to you if you've, even if you've used Python for a while or if you are familiar with other static languages. And that's okay. That's kind of the point of this talk. We're gonna talk about what that means. And so the steps to understand the answer to that question are sort of this. We need to talk about types in Python, just regular types, type systems in general across other languages. We'll talk about dynamic typing in Python and then finally we'll be able to talk about static typing in Python. And once we understand those things, we'll talk about how to use static typing when you should use it and maybe sometimes when you shouldn't use static typing. So let's talk about types and specifically let's talk about type, the built-in keyword in Python. And you might be familiar with this. So type tells you what something is, what an object is or what a variable is. So if I call type 42, Python will tell me that it is class int. So I know that that's an integer. And I can do the same thing with floats and strings and lists. So you might say, okay, I recognize these as built-ins. They tell me what types things are, but I can also use them to change types or do sometimes what we call type casting. And you'd be right, right? I could take a variable a and when I sign it to the value, we say that a is a type of an integer and I could cast it. I could use that float built-in to turn it into a float. And I can just a string built-in to turn that into a string and I can use list built-in to turn that into a really ugly list. So I imagine some folks watching might have had a weird bug where they accidentally got a string turned into a list. And guess what? That is a type error. If you've experienced that before, you're definitely in the right place. So why does Python let that happen? So it would seem like these are the only types that are available to us, right? The things that we can cast from one type to another. But these are actually just classes that happen to have corresponding built-ins in Python. So basically it's kind of just doing class matching here. We can say whether the number 42 is an instance of the class int and when we call type it gives us that same class back. So it's sort of just a class and it's sort of a shortcut or an alias for that class name. But there are other class types as well that don't correspond to built-ins. So for example, none, we've all sort of used the none built-in and we've seen none type before, but they're not the same. Similarly for a function, we don't say when we initialize a function, we don't instantiate a class, we say def-funk, but that has a class underneath the two. There's a function class and same for ellipses and actually lots of other things as well. And the place where you can sort of find the classes that represent all of these things in Python that we might want to interact with is from the types module. So you can import types, this comes with Python, and you get a whole list of different types of types. And the kind of fun thing about these is if you wanted to, you could instantiate a class with a function type and put things inside it, but actually that gets pretty long and verbose and it's a lot nicer to just write normal Python. So when we say the Python is a dynamically typed language, what we mean is that a variable, once it's defined can be any type. We can initialize the variable, we can set it to be one type and we can change it. So for example, what type is A here, right? I'm importing random and I'm taking a random choice from this list and that list includes an integer, a float and a string. So when I call type of A, this could be anything. It could be any of those three types that I put in the random options. And so it could be a string, it could be an int or it could be a float. And Python doesn't care. Python lets us set A to be any of these value types and that is what we mean by dynamic typing. This means that arguments and return types of a function can also be any type. So the same is true for a function as it is for variables. The things that we return from functions or that we pass to functions, they can also be any type. So in Python, when we write a function like this, how do we know what types this function is expecting, right? So if you saw this function somewhere and you didn't know what it does, what would you expect the argument types and the return type to be? So you might say, okay, well, I'm seeing that it's adding things together. So maybe these are integers, right? And you'd be right. You could call this from the gate function with integers and it would give you the sum of all those integers. But you could also call from the gate with strings and it would concatenate the strings into a single string. And actually this works for lists and really anything else that would support this addition operator in Python. So we can say that the arguments to fromicate and the return type of fromicate are dynamic. And then this can cause problems though, right? Because if we don't have any way to sort of guarantee what the arguments are, we can have type errors. So for example, if I call it with two integers and a string, we can't do this because the addition operator doesn't let us mix types like this. And so we get a type error because we're trying to combine integers and strings together in the body of the function. All right, so that's pretty confusing. How could we fix it? So one thing we could do is write really long and detailed doc strings about what the function expects and what the function should return. This is great for developers and sometimes we do this. I don't do this. This is a lot of work and it's a lot of work to maintain this documentation. And on top of that, writing these comments while they're helpful for developers doesn't actually guarantee that someone using your function is calling it correctly, right? These are just sort of like when you're reading a source code, it tells you how to use the function, but it doesn't give you any guarantees at all. One thing that could give you some guarantees is if you assert it on the type of every single argument and of the return type every time this function is called. So I can assert on A, B, and C that they're integers. I can do my business logic inside the function and then I can assert on what the return type is and then finally return it. We don't usually do this either. We usually try to avoid this for us because every assertion here is just a little bit of overhead, right? That's another line of code we have to run. It's an extra check and actually this can also be wrong, right? It's sort of dependent on whether the developer has written the assertion correctly or remembered to add the assertion to their function. So there's things that could go wrong here as well and we don't do this very often. So what do we do instead? So instead in Python, we do something that we call duck typing which is that if it walks like a duck and it clacks like a duck, it is probably a duck, right? We rely on how the variable is used to sort of determine what type it should be. And so for some examples in each of these you can kind of guess what the variable bar is based on what is being done with it. So in the first one, you can tell we're iterating over it. So it's probably like a list. Could be a string though, it's hard to tell. In the second one, we're comparing it to zero with a greater than operator. So bar is probably some kind of number like an integer or a float. And in the last one, we're calling it, right? But it's kind of ambiguous. In the last one, it could be a function. It could be a class. I don't know what I'm gonna get back for foo in each of these. So it could be anything. And yeah, the ambiguity here if we wanted to reduce the ambiguity is what static typing would give us. So static typing means that the variables and the return types and the argument types of function are defined and they will not change. So there are actually lots of statically typed languages. I wanna show you some examples real quick just so you can see what they look like. So this is C. This is the same from the key function but written in C. So you can see that there's it's declaring int for the return type and int for all the variable types. This is Java. It's a pretty obvious if you use Java before because Java has this public static int in cantation at the beginning but also declares argument types to be integers. This is Rust. So Rust has really fine grained control of the integer type. So that's an unsigned 8-bit integer that it's accepting and returning. And this last one is TypeScript. So in JavaScript, JavaScript doesn't have static types but there's a variant of it called TypeScript. And in JavaScript, all numbers are the same thing. They are in number. So in TypeScript, there is only the number type. So everything passes argument and this function is the number type. So we can sort of put languages into two categories, dynamic and statically typed. You can see that Python right in this slide is in the dynamic category and things like C and Rust and Java and TypeScript are in the static category. But we have to put a little Astrid here because Python is kind of an odd duck and kind of we can do static typing with it as well. And technically Ruby is gonna get something very similar to Python at the end of this year, I think, last time I checked, but for now I'll leave it in the dynamic column. So like I said before, Python is dynamically typed but can optionally be as statically typed as you want it to be. So this wasn't always true. And the story of how Python became a language that can optionally be statically typed is also kind of the story of static typing at Dropbox. So Dropbox is a large Python shop that use a lot of Python, a lot of Python needs to work there. And there's some things sort of in their journey from going from untyped to typed Python code that led up to static typing as we know it in Python today. So I'm gonna give you just a quick overview of everything sort of the progression of static typing and how Dropbox is involved and that kind of thing. So in 2006, we have a PEP, PEP 3107. This came out when Python three and this allowed us to do something called function annotations. So it allows us to take a function like this and add basically any metadata that we want to argument and annotate the arguments and return values. So thing to know here is that this has zero effect on execution of the function, but what it gives us is a nice little attribute on the function of the annotation. So it will actually evaluate the annotations, put them in this nice little dict based on the variable and the return type. I'm not sure what would happen if you were passing an argument named return. I guess maybe that's probably not found or is found upon, but yeah, you would get this annotations dict and anything that was sort of putting together this function would have access to these annotations. So this is interesting and the PEP included a bunch of ideas about what this could be used for and they all kind of boiled down to typing and then maybe some other stuff like documentation. So this allowed us to write a function like this if we wanted to do, you know, evaluate typing with it. I could as the annotation pass this class or the key word that represents that class and then when I access the annotations it would tell me what I had written in my source code as the annotation for those variables and the return type. But that's all it gives us. It doesn't give us a way to evaluate whether this function was being used correctly or whether these return types and we're actually being returned. And it only gives us a way to annotate functions. It doesn't give us a way to annotate any variables or anything that might be used within the function or outside the function. So around this time, Yucatec Toslo was working on his PhD research at the University of Cambridge and his research was about the unification of a statically typed and dynamically typed language. And he wanted to use the same language for everything from a tiny script that doesn't need static typing to a sprawling multi-million line code base. And he was also focused on the gradual growth. He wanted to be able to slowly migrate from an untyped prototype to a statically typed product. Everything from, you know, take that little script and slowly migrate to something that was large and statically typed. And this is the idea that you basically don't have to do it all at once, right? He wanted to be able to make static typing completely optional. So he published his thesis in 2011 and it said something basically along these lines. He found that adding a static type system to a dynamically typed language can be a really invasive change would require coordinated modification of the programs, virtual machines, development tools, basically everything that the language touches would have to be changed. But he found that adding an optional, pluggable type system would not affect the runtime semantics of the program and could be added to a language without affecting the existing code or tools. So it sounds really great. It sounds great for Python because we're already set as a dynamic language, being able to statically type things optionally makes a lot of sense. So at 2013 at PyCon US, he introduced something called MyPy. So if you've heard of MyPy before, what he introduced was not exactly what you might think of MyPy. At the time in 2013, when he gave this talk, MyPy was described as a experimental variant of Python that supports running programs that mix dynamic and static typing. So in his research, he wasn't able to use Python as the language that he was using to do his research. And so instead he created his own sort of variant, which he's doing research, so it sort of makes sense that he could create a variant just to sort of prove his theories. His variant looked like this and he actually called the variant MyPy. And the variant looked like this. You can tell it kind of looks like Python if you squint at it, but there's some other stuff happening here, like you can see where the function fib is being declared as an integer and the argument type kind of looks like a function annotation that we were talking about before. So it kind of looked like Python, what wasn't Python, but you could compile it to Python. And so the issue was that even with the function annotations, Python couldn't support everything that was necessary to be statically typed. And so what happened? He presented this at PyCon and he talked to Guido afterwards. And Guido said, basically, let's drop the custom syntax, let's drop the variant on the language and let's just do this in regular Python 3. So MyPy also included a type checker for the variant and that is actually what we think of as MyPy today and since it was now modified to check Python itself. So in 2014, we got pet 483. This is a theory of type hints and this is basically Guido putting down his ideas about how static typing should work in Python. And his ideas sort of aligned pretty well with what Yuko was working on. So he said that typing should be optional. Adding annotations shouldn't affect the runtime of the program, shouldn't get in your way. An annotated function should behave exactly the same as an unannotated function. And I think this was probably had a lot to do with the lessons we learned from the Python 2 to 3 migration. We wanted a adoption of static typing to not be a breaking change, not require a huge migration and that kind of thing. He wanted it to be gradual and this allows us to annotate only one part of a program or even just one function of a program and sort of leverage the aspects of both dynamic and static typing together. He wanted to add variable annotations so we could annotate more than just a function. And so that means that we could do something like this instead of having our variable biz-baz be unannotated, we could add a comment like this called a type comment and this sort of gives us a type annotation for that line without changing the Python syntax. And he wanted to be able to support type hinting for Python 2 because function annotations only existed in Python three at a time. So he wanted it to be possible for even the people stuck with Python 2 to continue to be able to adopt static typing. So that means that in Python three, we could write these nice function annotations but in Python two, we could fall back onto these type comments even for the function definition and that kind of stuff. He introduced some special type constructs which were sort of building blocks that we could use to define types. So that adopted all the existing types like int and float and then created some new types that come from the typing built-in module like any, any would be a wild card for matching any type union would combine two types together into a single type. Optional is the union of none and basically any type that you want, tuples, subtypes for collections, that kind of thing. So that would let us write a function like this. Once we have special type constructs, we could say that the last argument to this front make a function could be an integer or it could be a float but the first two have to be integers and then we could similarly define the return type to also be the union of int and float dependent on what was passed into it. It also defines some container classes. So these are things like lists and dictionaries so we can type the keys or values of a dictionary, for example. So like I could define a list that contains all integers and trying to append a string would fail. I could define a dick that has a key of a string value of an integer and similarly trying to go against the typing would fail the type checker. It also introduced some generic types for when classes and functions behave in a sort of generic way. So we could do things like this, just say that work accepts an iterable, give us some aliases as well. So if we wanted to we could union all the numbers together into a single number class and be more like JavaScript. And next in 2014, we got type hints and this was given to us in Python 3.5 which was the least with that support. We got syntax for variable annotations so that let us go from the comments style syntax to pure inline Python. We could annotate variables, we could initialize variables with type so that actually happened to give them a value. Similarly for classes in Python 3.6, we got about 5.26 support which almost had everything we needed. Last thing that we needed to finish the puzzle was a type checker. So the difference between type checkers here is static and dynamic. So my pie at this point had fully transitioned to a static type checker. That means it's gonna check all of your code at rest and it works kind of like this. You would install it from PIP, you would create a Python file, okay, and you would define a Python file and then run my pie on it and it would tell you when you're using types correctly or incorrectly. So at this point there were actually lots of static type checkers and dynamic type checkers as well which would check your types at runtime. So basically all of the big Python shops had their own type checker and still do actually. So Dropbox basically owns the development of my pie. Google has a tool called pie type. Facebook has pie or Microsoft has pie right and actually some editors have type checkers built into them like pie charm. And then yeah, there's a bunch of dynamic type checkers as well. I think those are not quite as popular because they do add a limitation at runtime. So one of the questions I get a lot about this are the differences between the static type checkers because if they're just implementing the standards like what could possibly be the difference maybe one's a little bit faster than the other. So I'll talk about the big sort of philosophical differences between my pie and pie type which I'll call cross function inference and runtime lenience. And these are sort of comes down to a difference in philosophy about the behavior. So for cross function inference, here's an unannotated file. And if I were to run this in Python I would get a type error, right? Because what I'm trying to do here is concatenate a string with an integer and that would fail. So just executing Python, I get a type error. If I run this through my pie, it actually succeeds. It doesn't tell me there's anything wrong here. If I run it with pie type power, it will give me an error. And the problem is that my pie doesn't have the ability to infer types across multiple function calls and pie type does. So pie type says, well, that's gonna be a runtime error. We need to raise an exception or fail the type checker here. Similarly, another difference in philosophy is sort of whether functions that succeed at runtime should be type errors or not. So in Python, I can annotate a list to contain only strings, but then it's totally valid to append an integer to it. That won't actually create a type error in Python. And so if we run this, it works just totally fine. If we run it with pie type, it says there are no problems. This won't create a runtime type error. My pie is a little more strict though. It will say, well, you define the type for that list to be a string and you try to put an integer in it so that will fail. All right, some more questions. You might just be saying, all right, why? Why and when should we use static typing? So first I'll talk about when you shouldn't use static typing, which I think is basically never. I think static typing you should just use at your leisure or as liberally as you want. You should be able to experiment with it. One thing I'll notice is that static typing is not a replacement for unit tests. And there's a bit of similarity here. Sometimes unit tests end up looking a lot like type checking. You know, you sort of pass certain things in and assert on the output of the function, whether it errors or succeeds. But really unit tests are just a bad type system. In reality, you probably should do both. So when should you use static typing? Basically use as much as possible. You know, use it whenever you can. You should definitely use static typing when you're a millions lines of scale. So if you're Dropbox or Google, you've probably already invested a lot in static typing and Python. And actually Dropbox found that at their scale, dynamic actually was a liability for Python. It made it really hard to understand how functions and how a code base would work and began to impact productivity. So all those companies had invested a lot of money in the work that was already done here. And you know, the lack of typing, you know, you can think of it as a liability once you get to be that big. Made a little handy graph. So the time to start thinking about static typing is not when you're the size of Dropbox or Google, but sometime before that. So you're probably here. This is when you should start thinking about adding type annotations. Here's when you'll probably actually do it, but that's okay. You can gradually add it as you go. You should also use static typing when your code is confusing. So let's be honest, we've all written confusing code before. Some people like to say that annotations are a machine verified documentation. That's true. If you feel like you need to document a confusing function, specifically the input output types of it, for example, you should probably just statically type it because that would be just as clear. And then you can do static analysis on it as well. You should also use static typing when your code is for public consumption. For example, if you're going to put it on IPI as a module. So adding type annotations helps third-party developers that want to use your module know how to use it. It also lets IDEs do auto completion and tell you how to use consume the API. And then if your users are already using static typing in their code base and they want to pull in something from your library, they will absolutely love that you can support static typing as well. Another place that you can use static typing is before migrating or refactoring. So before you make a big change, go and add some static types, move things around and see if any of the typing breaks. That's probably a bug. And finally, you can use static typing just to experiment with static typing. It doesn't hurt. It's totally painless. And it's just be a fun way to learn about a new part of Python. So to wrap up how to use static typing in just five easy steps. First, you have to migrate to Python greater than or equal to six. Now this is actually optional. You can use type comments in any version, but let's be honest, you should migrate to Python three anyway. Install type checker locally. I don't care which one it is, but install it and integrate it into your editor so that it runs automatically when you write or save a file. And then you can start optionally typing your code base. Start with the most complex function or the easiest to understand function. Don't try to do it all at once. Remember, it's intentionally a gradual process and pick sort of critical areas and start there. Last, or second to last, run the type checker with your linting, put it in CI and have it just run there anyways, assuming that you have CI. And finally, convince all your coworkers to join you. So if you need help convincing them, feel free to send them my way. Thanks everyone to the talks. And I wanna give a quick shout out and thank you to the organizers as well. This has been really amazing, seamless online conference and they absolutely deserve your thanks as well. So thank you everyone. Thanks for your attention. Absolutely, thank you so much. I am supposed to be able to play this at the open door of the audience. So yeah, so I have one question here. Oh wow, actually this is going up to five in the last few seconds, but we don't have time for five. We'll do as many as we can. So anybody, if we don't get to your question, look and discord for talk static typing, you've got to dust in there some more. But let me see here. So someone said, is there any tool which can build API docs of the return types of each function by extracting static type annotations from your Python code? So you're saying automatically building documentation from static type annotations? Yeah, I think there's some stuff that exists and definitely there's a lot of the documentation tools now will take into account whether you have statically typed functions. You don't need those long doc strings that I was showing for in order for your API documentation or your module documentation to pick up and show what the expected values are for some. So I haven't actually used any of them personally, but yeah, I imagine that some of the more prominent documentation tools definitely should be supporting that. Yeah. Okay, Francisco asks, how can one combine type annotations to perform static type checks with myPy with detailed doc strings? So it's kind of along those same lines. Oh, so you already have detailed doc strings and you want to add type annotations? Yeah, that's a good question. I'm not actually familiar with a way to do that. So it's sort of, I guess there are type checkers that check doc strings and there are type checkers that check static typing annotations. I'm not sure if there's any that do both. I'll try and find out and I'll let you know in the chat. Someone says, what about myPyC? Is there a future for compiled Python using type hints? I think there might be. I don't know a whole lot about myPyC, but yeah, definitely I can connect you with folks that are working on that. And yeah, there's some interesting developments there, I think. All right, well, once again, thank you very much. Thanks, great to be here.