 So, welcome everybody, I'm Andrey Blasovsky, and I'm going to talk about how you can benefit from type hints. So I come from the city called St. Petersburg, it's a big city in Russia, and I work for a company called JetBrains, and I'm one of the developers of the ID called PyCharm. Also, I've been contributing to this new PEP for it for called type hints, and I'm very glad that it will be the part of Python 3.5. I would really like to say thank you to the authors of this PEP, Guido, Yuka and Lukash, and also to the BDFL for this PEP, Mark, who actually accepted this PEP. So, what are type hints? Type hints are like any information provided by the user about the types in their programs, like types of function parameters, types of class attributes, and so on. And the PEP for it for defines the formal notation for type hints. So, what I'm going to be talking about here is basically the motivation behind type hints in Python, and how you can get started with them. And also I will talk about the best practices for annotating your code with type hints. What is out of the scope here is the overview of the type system, what kind of types can you write in your code, and also I want to walk through the whole typing module, and this is out of the scope here. Probably some of you have visited the talk by Guido and Rosson earlier, and I have a couple of questions for you. Who of you has actually listened to Guido about type hints? Please raise your hands. Great, almost everyone. And how many of you have actually looked into this document, PEP for it for? All right, about two thirds, I guess. And have you ever had a chance to actually play with type annotations in your code? Just a couple of you. Great. You came to the right talk for you, and yeah, I'm very glad. All right, so type hints in Python, they are used to be informal. So basically Python programmers would define type hints inside their doc strings, and those definitions were usually given in an informal language or natural language such as English, for example. So there's a piece of documentation from the standard library about the filter function. As you can see, there are lots of words highlighted in blue that actually refer to type hints. And the PEP for it for basically defines the formal notation. And as you can see, this is the type hints for the same filter function, and they are a lot dense. And if you have a bit of understanding of what type hints are, this notation, I think, is much more clear. And just one thing I want to remind you from Guido's talk is that Python will always be a dynamic language, and that type hints are completely optional. All right, so from my perspective, the main benefits of type annotations is readability. Type hints are all about readability for both humans and tools. What do I mean by readability by humans? Actually I mean documentation. Documentation with type hints is a lot more easier to grasp. If you already have some familiarity with type hints, then lengthy natural language descriptions. And by readability by tools, I mean the possibility of creating type checkers and flinters for Python. And also the tools that help you write code, for example, for code completion. Type information is crucial for those kinds of tools. And that's what we are going to talk about in this talk. So let's start with type hints for tools. What tools do support type hints right now? Actually, I know only two of them, is the MyPy, the original tool that was the main inspiration for this PEP for it, for written by Eucl-ElectorSalo. And PyCharm. And I should know that we have a free community edition of PyCharm that is completely open sourced under the PyCharm 2 license. And all our type inference and type analysis code and stuff like that is also open sourced. PyCharm 2.5 that was released earlier this year has already some support, some basic support for type hints. But the PEP wasn't ready yet at the time we were releasing PyCharm. So only some parts of the specifications are supported. And PyCharm 5 is coming this fall, and it will include full support for type hints, PEP for it for. Several other developers of code analysis tools like PyLint have already announced that they are going to support type hints in their tools. And for PyLint, I guess it will support it in 2016. All right. So let's pick an example for understanding type hints. And as an example, I've chosen the library called elementary. It's a part of the standard Python library. And I would like to ask you how many of you are familiar with elementary? Please raise your hands. Almost everyone. Right. And those of you who are not familiar with it, it's a library for handling XML data, for passing XML and for dealing with it with an API that is nicer than the standard DOM API. So let's start with this library and let's write some type hints in order to see the benefits of them. And just for the sake of brevity, let's call it E3. It's our own modified version of elementary. Later on, I will show you how to deal with the library without modifying it. So here's a little example of using the elementary library. And if you are quite familiar with the API of this library, probably you would notice some errors here. Does anybody see the problem here? What's wrong here? Sorry? Please speak louder. Sorry, can you hear? Yeah. Yeah, bite literal. Right. That's it. So basically, when you run this code, the result will be a type error because elementary cannot serialize binary data. By the way, there is another error here. You cannot specify integers at values of XML attributes. So yes. You have to basically run this program in order to find this error if you are not that familiar with the elementary API. And as we will see right now, the type checker tools will find this kind of errors for you automatically with a little bit of help from type hints. So let's start annotating just the constructor of the element class. Here we add the stir annotation for the tag parameter. We expect it to be a string, not a by-string. And we expect the attribute parameter to be a dictionary of string keys and string values. So it's a generic type. All right. And with the help of this type hints, what we can get from tools. Here is the output of those two tools. PyCharm will underline the problematic elements with this yellow squiggly line. And it basically says that it expects the type string, but it goes by instead. And the mypy being a command-line tool outputs basically the same information, but only in the textual form. All right. So type hints are already somewhat useful. Let's continue annotating the API of the elementary library. Most type hints you will write in practice are quite simple. For example, the registerNameSpace function accepts just two strings at return nothing. Basically there is no return statement in this function. So we annotate it with the return type none. And the element, and the isElement function over there, it accepts basically any value and checks if it is actually an XML element. So we specify the return, the parameter type of the element parameter as any. Some type hints are a little bit more complex. For example, below we are using some genetic types. The iter function returns an iterator over elements. And you will notice this string quotation Widow has mentioned in his talk because we are referring to the class itself while we are still defining it. And the iter text function returns just an iterator over strings. All right. So with the help, suppose we have annotated the whole API of the elementary. What could we get from that using the tools? A little piece of code in PyCharm. Let's find out what this program does. It's a program that creates an XML element inside the h1 tag. So it's under HTML. That contains the string uppercase. So heading, the heading variable is an element with the upper text hello in h1. And then we print basically the text of the first node, textual node of this element by iterating over text nodes, sub elements inside this XML element h1, and print the string world here. So the result would be uppercase hello, comma, world, exclamation mark. And actually there is a problem in this code that PyCharm would notice with the help of those type hints. And in order to find out what this problem is and why PyCharm will find it, we should think of an algorithm called type inference here. So let's think as the type inference would think about this program. We have to find out if the indexing operator, the square bracket operator is available on the result of the iter text function. So let's find what iter text returns. But we still don't know which class contains this iter text method, because we don't know the class of the heading variable. So we need to go and look through and look up the definition of the heading variable. It's right there on the previous line. And it is assigned a return value of the uppercase heading function. And knows that uppercase heading has no type annotations whatsoever, but a type checker would try to infer the return type of the uppercase heading function. In this case, it returns a local element variable, and it is assigned a value of basically calling a constructor on this element tree.element class. So we know that the local element variable is actually a variable of type element. It's an information inference from this very basic example. And we know that the return value of the uppercase heading function would be element. So heading variable, the global variable heading, is also of type element. And let's go to the previous slide to find out what iter text on element is returning according to our written type annotations. Here it is. It's the last line on the slide. We have annotated the iter text to return the iterator over strings. As many of you would probably know, the iterator is an abstract base class, and it defines only two methods. It's the dunder iter method and dunder next. So there is no dunder get item method that is available on the iterator. So we cannot access, have access to the elements of the iterator by using the indexing operator. So it is clearly an error. And by chance, with the help of some type inference, quite basic type inference in this example, but we could come up with quite more advanced ones, can infer, can find this kind of error. And know that we don't have to annotate any function. Some annotations would be enough in most cases. Another good thing you can get from type hints is code completion. So in PaaCharm, we offer so-called context-sensitive code completion, meaning that it is based on the actual type of the variable. It shows only methods and attributes of the type of the variable. So this is the same iter text function. We are iterating over an iterator of strings. And we have invoked code completion for the variable we are iterating over. And we know that it's type of strings. So we show the methods of the string class and the methods of its superclasses. So those are the benefits you can get from tools. Now let's switch to benefits from the documentation. Often documentation could become quite worthy if you are carefully writing down all the types of your function definition in English. Here is a real-world example from the standard library documentation for the function called command of the elementary module. And not only is it worthy, but it also contains an actual error here marked in red. It says that we can pass a byte string in here. But as you can see from the example below, if we actually pass the byte string literal and then try to serialize it, it will print the byte string literal as its wrapper, be quotation marks and so on. This is crazy. This is absolutely not what you are trying to do here. So either the documentation is written badly and not tested, or the implementation has a bug in it. Something is definitely wrong here. Or maybe I just don't get the English text here. I'm a foreign speaker, so maybe. All right. So what would it be better if we annotated this function with type hints and used just the doc string from the library? I did exactly this here. I used the Sphinx tool, the famous documentation generator for Python, with the auto doc and extension. Auto doc extension can extract the doc strings from your code as well as type annotations. And from my point of view, this piece of documentation is much more clear, and it explains what is going on here using better, more formally defined syntax. And I'm happier with this kind of documentation than this lengthy cumbersome documentation that has errors. Know that you can actually check your type hints in some kind of test. You can write a test suite for your code and run a static type checker in order to check functional annotations, but you cannot check the English natural language documentation. This is the idea. All right. So now I'm going to give some tips about how to actually use type hints in order to be productive with them. And the most important thing, actually, I want to tell you in this presentation, is that you should annotate your public APIs. If you are an author of some kind of library that is used inside your company, your colleagues are using it, or you are an author of some open source piece, and you want people on the internet to start using your library and you want to get more users, please, please do use type hints for your public APIs. This is the most important slide I would like you to remember from this whole talk. And it is so important that I put a reference here to this very slide so you would find it on the internet later. So please, if you would like to take a photo on something like that, yes, this is the main idea. Documentation for your public APIs will make your users happier, because it would act like, first of all, the better documentation, and it would act like a safety net around your library. Basically, you are defining a clear interface between your code and the user code. And if some kind of error is happening in your library while your users are using it, they could run a type checker, or they can just look the errors in their ID, and they will see that they are using it wrongly. They will not report wrong bugs to your tracker just because they won't get your documentation right. So this is the idea. Types are like better documentation, better for IDs, and so on. And as I've already told you, you don't have to annotate everything. Just annotate your public API and see if it's good for you. I think that in most cases that would be enough. Another tip is start with simple types. In most cases, it is enough to use types like stir, built-in types, int, and so on. You can use built-in collection types like list or dict, and if you want to specify the types of elements inside your collection, please use collection types from the typing module, the uppercase versions, capital case, dict, and so on. The path for it for, as you know, defines some advanced type constructs as well, but use them only when you really need them. And remember that you can always put any if some particular type is tricky and you don't know how to express it, or it's too long, and so on. So try to stick with simple things. They would be simpler for your users, and you won't spend your time trying to invent some crazy static typing construct in your code. All right, so another tip is be liberal in what you accept in your functions and conservative in what you return. Interest, a little piece of code, is basically a function that iterates over iterable integers and filters out the even elements and returns a list of them. Here, you are using only the under iter operator on numbers because you are using the generated comprehension in the code. So you don't have to restrict your users to pass a list here, for example, or an iterator. It is enough to say that the numbers should be an iterable. The more generic type you could probably mention that would work for this function. And speaking about the return values, in this case, I think you could be quite specific about what you return because you might want to allow your users to use some methods on the concrete type you are returning. For example, here we are returning the list explicitly. And you might want to allow your users to avoid, for example, the append method or something like that. Note that the typing module defines all kinds of these abstract collection types like mapping or sequence when you can access stuff by index operator, set and so on. So they are good for your parameters, but they are somewhat less good for your return values. It depends, actually, on you. All right. So far, we've seen some type hints written formally using Python 3 functionalizations, right? And in order to use those annotations, it is enough for the user to just keep on stall your library. The type annotations are in the code, they are available, everything's good. And this is compatible with Python 2 and some earlier Python versions if you install the typing module from PyPR. Right now you have to specify typing equals equals 3.5.0 beta 1, otherwise you will get just an empty package. But this is because the library is very new and the developers haven't had the chance to fix it. All right. And what about Python 2? I guess many of you actually are using Python 2. So here's another quick poll about Python 2 versus Python 3. Who of you are using Python 2? Please raise your hands. Ninety percent, I guess. And who of you are using Python 3? Wow. This is really good. Python 3 is finally coming. Great. All right, so for those of you who still need to support Python 2, there is this thing called Python Stops, just a quick reminder of what they are. They are Python files with the different PyI extension and they contain no bodies for methods, just that leapses a little. And here's our little stop for the elementary library, the actual elementary library of the standard library of Python 3.5. And this is how we should define type hints for the standard library. Not by modifying it, obviously, but by writing this kind of stops. And you can use these Python 3 Stops with your Python 2 code. And the type checkers will understand that. So stops for Python 3, code for Python 2. All right, so stops are very good for annotating not only the standard library but for third-party libraries as well. Type checkers would expect them to be somewhere on your system path. So if you are creating some kind of project that uses lots of third-party libraries and you want to annotate them in order to get better code completion, you can just create a folder inside your project. For example, a folder called type hints and put your annotations for all the libraries in there, just added to your system path. And then you can, for example, share this folder with the rest of your project using Git or Mercurial or whatnot. And your colleagues will have your type annotations. All right, so if you do that kind of stuff, if you are interesting in creating Python sub files, please share your stuff with others if you are annotating third-party libraries. There are several ways of doing that. The main is to send a pull request to this new type shed wrapper. It's the wrapper mentioned in 4.4. And it is supposed to be the universal collection of all the stops available. Basically, it's modeled after the definitely typed wrapper for the type script, the language that has type hints from its early days. It's still quite new. And this model of creating stops for libraries is very successful. It's very, very well established in the type script community and in the JavaScript community as a whole. For example, people who are still using JavaScript, they clone this definitely typed wrapper in type script for their project and check their code against these type script definitions of libraries. So in Python, we will have the same situation where you can actually check your code against this large collection of stops for all kinds of libraries. Another way to specify it in the app is to create a package with stops on PyPy and use it. All right. What if you don't want to invest in writing type hints, either in functional annotations or in stops? Then you still can benefit from them. For example, if any of you are PyCharm users, you might not know it, but you are already using function annotations. Because in PyCharm, we have this kind of behind the scenes repository of type hints for the standard library. Not all of it, but for some parts. So basically, you are already using type hints, but they are written using our own legacy syntax. And we are going to switch to this new syntax. That's why we are interested in this proposal, and we are contributing to it. We want all the tools to use the same shared collection of stops in order to be more productive as a community. Another way of using stops by other people would be cloning this repo and start using it. All right. So in conclusion, I urge you to try type hints. They are really good. I'm a fan of them, as you probably already know, but they're really good. And there are a few very simple steps how to get started with it. Just grab the latest version of Python 3.5 beta, or use some older version of Python 3, and get some of the tools, mypy or pycharm, and just start using type hints. Try it. I do want to know your thoughts about it. So please contact me. Here's me on Twitter and my website, pyrx.ru. I really want to hear what is your experience with type hints. Do you like them? Do you find them good for your users, or maybe bad for your users? Are you going to switch to Python 3 with type hints? I'm very interested to hearing all those kinds of stuff from you. So please, please contact me. And the slides are already available on the internet via this URL. So thank you for your attention. Very interesting talk from Andei. Any questions? Thanks for the talk. I'm interested if you can do an easy introspection into the type hints to do validation for RPC, for example. For RPC, well, you have to write some kind of tool that we would understand the introspection API of your RPC library and generate those kind of stops. So it's a piece of software that is perfectly valid, and it's a good idea to create these kind of things for your RPC mechanisms. But can you use type hints to do that, to respect the type hints and do validation based on that? Yeah, yeah. That would be also a good idea, yes. Thank you for the talk. So you said we should all annotate our public APIs. And you also said that PyCharm, for example, has already the tooling to infer type hints from the code itself, in some simple cases. Maybe there is also a way to create stops for us, because we are lazy to annotate everything? We are thinking about this kind of feature in PyCharm 5.0, but it's still work in progress. So we will see what is possible here. We are quite good at inferring the return type for functions, but it is very tricky to infer the type of parameters to the function, because the only source of information for parameters, actually, there are two sources, the attributes to accessing of the parameter inside the body of the function, and the usages of the function. But they are all kind of not so, they contain much less information than you can get for the return type, based on coding to constructors or literals and so on. So yeah, we'll see what's possible here. Is it possible to use if punches in stop files in case, like when type's different on Python 2 and Python 3? Sorry, could you repeat the question? Is it possible to have an if Python 2 in a stop file? Yeah, definitely, yes. The PEP44 defines several ways of using conditionals like that, but we don't support them in PyCharm 4.5, but we are going to support them in PyCharm 5. Do you think that it would be a good way to provide source compatibility with Python 2 to make tools like 3 to 2 to generate the stop files from the source code instead of repeating yourself and defining the API basically two times? You mean like generating... No, I mean targeting Python 3 in development and then generating the stop files from the Python 3 source code. Ah, yeah, yeah, it is actually a very simple task. I guess it's a good, it would be a good addition. For example, for the MyPy tool, yes, yes, it's definitely possible. Hi, is there any effort or project that is putting the type hinting on all the standard library? Because it could be a nice party to put in all them and when Python 3.5 is released, having all the standard library annotated? The project is called TypeShed, so TypeShed will eventually have the stops for the whole Python 3.5 library and Python 2.7 library. There will be stops for the standard library, yes. Is there any to generate the stop file from doc string type information? Not, I'm not aware of any kind of tool like that. It would be not so hard to create one, but I'm not aware of any of the tools. Thank you. So I played around a bit with MyPy and the first thing I looked into was trying to detect if a function returns an optional value so it can be none or it could be something else. And that's something that MyPy does not currently pick up. Yeah, yes. Does PyCharm do that or is that hard? It's a question with some bit of history. We used to support those kind of checks when we still didn't have any type annotations whatsoever and we used only the inferred type information and we used to have these checks when you have to actually write something like if value and then you would be able to use the value and you will be sure that it is not none. But then we received a lot of reports from our users that this is not what they were expecting from PyCharm because the inference type could be not so accurate. And if you are explicitly specifying that you are expecting an optional type or you are returning an optional type and then using this type, then I guess we can trust your annotations because this is what you are expected from the defined behavior of your functions and we will support checking for optional types in PyCharm 5. Any more questions? So thanks him again, thanks for the talk.