 How many of you already use my pie to to check your production code? Wow, that's quite a lot For those who aren't this might be a little bit steep in the beginning But I hope I won't scare you away from my pie My name is Vita. I'm a software engineer and now a co-founder of of a company called one plane We started five years ago and what we do is we trade stocks mostly in Europe and We do that automatically and semi automatically we are based in Prague and Everything we do on the back end is in Python 3.7 at this point and We also happen to use a lot of async I owe and I like to think we were very early to start using type annotations and and my pie so Static typing is still Quite a new thing to the Python ecosystem and the community We're still learning how to use it and the tooling is Still being actively developed and for those reasons it is sometimes a bit difficult to maybe not get started with static typing but to actually cover complex code bases with static types But this by these challenges I Believe it is really worth it because When it is done properly it can help you avoid a lot of mistakes and in the box before you even run your program Before you even run your unit tests I'm going to talk and two Chapters the first one is the the high level approach you might want to take when you when you have a big code base and you want to cover it with my pie and Then we'll talk about a few examples of code that is not the usual hello world function and how you might go on about typing that and In the end I'll remind you that it really is worth it even though it will look a bit complicated sometimes so I Mentioned before that we started using static typing quite early at a point where we already had a couple hundred thousands lines of code and My pipe was very early back then and it it was crashing on the code I don't mean spitting out Typing errors, but actually crashing so we had to start gradually and only cover our code step-by-step and a big lesson we learned Unfortunately not in the beginning but over time was that the default my pie configuration is quite lenient and If you don't make it slightly stricter than the default is you might Learn a few bad habits that will come and bite you later And you will still have to fix your code and their annotations. So I would recommend you Whatever the code is that you're going to run my pie on I Would recommend you to have full coverage meaning There are no functions which have no type annotations or partial annotations So these are the config options you you might use for that Second these are optional, but you might want to consider them These restrict some forms of dynamic typing in your code Some of these options are difficult to enable but if you can do it or if you're starting with a new code base I would definitely use them and and this You really want to do with my pie and and static typing It's sometimes easy to fall into a trap where you think you know what you're doing and you or you think you know what my pie is doing and It might not immediately tell you that your understanding is not quite correct So enabling some warnings will help you with that Since covering an existing large code base is a huge amount of work. You want to go step by step so you begin by opting in meaning you run my pie only on the modules you've already covered and You might even start with a single module with a very small step and then you then you just keep adding on and And you want to defend your progress by adding this check into your CI Pipeline so it runs before your test to do and then of course you never want to make that list smaller You only ever want to expand it What worked well for us was doing an internal hackathon where a couple of developers stayed at the office overnight and work hard to to increase our coverage and We're still not completely there, so we might have to do a few more sleepovers when you go the opt-in route you need to deal with imports because You are covering maybe just a few modules, but those modules might be importing other code which You maybe are not ready to check in the beginning So this is how you tell my pie to not complain too much about other modules A word of warning that follow imports Directive has another option called skip the documentation warns you not to use that we did and It was a terrible idea. Don't do it At some point when you're opt-in list is sufficiently long You might you definitely want to switch to opt out meaning you run my pie on everything by default except for some modules that you exclude in your config and of course, there might be dozens of these Ignored modules in the beginning when your exclude list is huge But then of course over time you work to make that list smaller and smaller until it disappears The benefit of getting too opt out is that any new code you add to your any new modules You add to your project will be checked by default covering unit tests is Tricky matter Despite me recommending strict strict configuration for my pie I will backtrack on that for tests and Just make my pie a little bit more lenient for reasons explained in this code sample By the way, I'm going to put the slides up online So you don't have to date photos of everything There will be a lot of configuration and code when you use mocks and monkey patching in your tests, which you often do There is no way to explain that to my pie as of yet. It is a very complicated problem So you just need to ignore those places where where you monkey patch But despite these challenges, I would urge you not to ignore all your test files completely Because even when you partially cover them with my pie You will get some benefits because my pie will be able to check that your tests are using your your tested code as Intended meaning the annotations are being respected if you build your own Python packages You should know that even when they Do you have type annotations in their code and my pie passes on them? If you use that package somewhere else my pie will not follow those annotations by default So you need to tell it. It's very simple. You just need a marker file added to the package and that's it But unless you do that You don't benefit from from annotations in packages When you use third-party packages, which might not have type hints There's a few options. You have you might write stubs. This is something we don't really do so I won't go into detail You might want to ignore all third-party packages or Better you ignore again very explicitly ignore just those that Don't have annotations So now that you know Generally how to approach a code base We can talk about a few examples of what you might find in your code base the first example a Very useful frequently used tool generics and type variables Who here has heard of these or maybe even use them? Wow, that's that's really good. I think this is one of the most useful and needed features Let's take the example of a weighted average, which is a very simple formula simple computation where you add up values and you average them using weights and Critically we will want to implement this average as Incrementally updatable meaning you can keep on adding values to the average and getting back the result so you might start by writing a very simple class for example that starts with some internal pre-computed values and You will be able to add a weighted value to the class up to the to the average and Then you just add a simple method that that can calculate the average at any time You will notice that we are using floats as the data type in there So we are explicitly saying that we can only calculate averages for floats but Imagine you not only want to do that, but you might also want to use decimals Which is a an arbitrary precision data type in Python so of course as written as annotated that class will work for floats as expected and Of course, it will not work with decimals because you said your values were going to be floats By the way, the reveal type function is extremely useful That is provided by my pie. So it's undefined at runtime But for debugging what my pie thinks your variables are it is a very useful function. So if you want to allow floats or decimals a Good good way to do that is to parametrize your your weighted average so you make it a so-called generic class and you say it is parametrized by this Type variable which we called algebra type in this example and we restricted that type variable to either be a float or a decimal then your original class will be very similar to the previous version, but You will suddenly have a small trouble with the number zero By the way, this code block contains a very small lie Maybe some of you can see it, but it's not very important at this point. It works and Then in the rest of your class instead of saying float you will be saying algebra type So you've parametrized the type Now when you want to use that class When you instantiate it you need to add the value for that type parameter When you create the instance like this, so the first few lines are a weighted average of floats and You can see that it also returns a float and of course the second part is a weighted average of decimals What is nice is once you create a weighted average of a certain type you cannot change your mind and start mixing the types that is desired There is an even possibly nicer and cleaner solution Which is to say Actually what I need to do with my numbers is to add them multiply and divide them So I don't care if those are floats or decimals or something else. Ideally, I would just say there are real numbers So in theory that sounds great, but in practice the abstract number types in Python don't really Work that well or aren't that useful yet? And this is a good example of Of typing in Python being quite pragmatic it isn't and Ideal world it is a pragmatic world and you need to be pragmatic to you So your type annotations often won't be perfect. They won't perfectly describe what you had in mind, but they will approximate it and another very important examples example is Understanding the difference between nominal and structural typing So these are fancy sounding words, but it's nothing complicated Nominal typing you already know that's that deals with class inheritance. So Animal examples seem to be popular in computer science for some reason. So I went with one and There's a base class of an animal and then we have a duck that apart from whatever behavior animal has can also quack and then Suppose we want to make a function that accepts something that can quack and make it quack So as annotated here, this works because it's it's very trivial You're just telling my pie that your function needs a duck because only ducks can quack and and that passes However, imagine you wanted to have another animal that can quack and you want that function to work for that animal too So we could create a Penguin which possibly makes sounds close to quacking But it would be wrong to inherit that from a duck that would be very wrong So you just inherit from animal, but then of course your make it quack function doesn't work because it was told to expect ducks so Nominal typing means you use You use classes and class hierarchy when specifying the types you need in contrast to that We can be talking about structural typing where you where you describe your types in terms of the capabilities they have so here you're creating a thing called protocol in Other languages you might have heard a term an interface or maybe a trait and This this is actual code those three dots are valid python syntax in In case you didn't know and this really just tells my pie that there is an interface or protocol called can quack which exposes a public method called quack and When we declare this and change our functions slightly so it now accepts something anything that can quack then this will work for both animals and The interesting thing is that we didn't have to inherit from that protocol that protocol is a class, but that is more of a syntactic convenience and Now any object that will have a quack method will meet the requirements of this function so this is very useful for For duck typing pun intended Another example we encountered is when you want to Somewhat define your own type without creating an entirely new type a Very simple example of that is when you have a function called place order maybe that accepts a price and quantity of some goods that you're buying maybe and Doesn't matter what it does with that maybe it will save it in a database or whatever and Wouldn't be nice if we could somehow differentiate between a price decimal and a quantity decimal They really are just decimals, but there is a very clear semantic difference between prices and quantities If we could do that It would make our code more readable because when you read those annotations You will clearly see this is of type price and this is of type desk of type quantity and It would also make it hard to mix them up so you wouldn't be able to accidentally pass a quantity in place of price I Come from a company that trades under financial markets and confusing Prices and quantities or buy and sell is not a mistake you want to make so the first option that you might think of is to alias a Type so you say there is There is something called price and it really is equal to decimal and then you can use that price as a constructor That that works But unfortunately that doesn't create a new type This is just a convenience for you. So you don't have to type so much It does make it a code easier to read when you suddenly start writing price instead of decimal But as for type safety you get absolutely no benefit another option that exists in the typing module is a function called new type which kind of aliases an existing type, but It is a true alias in the sense that my pine now and Understands that it is a price and not a decimal so What what this does is you can still create decimals, but then you need to wrap them in your price type and from that point onwards my pine knows it's a price not a decimal and If we defined a function that takes a price you will see you can't pass a Bear decimal to it and you can't even pass a quantity to it even though it really is another decimal so this is what we wanted and We're now able to differentiate between the types This all works to a point once you start modifying the values You're back to the original type because really it is just a decimal under the hood and once you start making Operations meaning calling methods on that type It will return back the original type. So there is a limitation you should be aware of the only perfectly correct solution of defining a new type is To actually define a new class and implement your type and the behavior you want so That is nice and clean But of course you will pay a runtime price because your price Implementation will probably not be faster than the than the decimal type. That's already in Python implemented in C So that way lie a lot of interesting dilemmas about preparing static typing purity or preferring Pragmatic runtime performance and simplicity Now in Python you can do a lot of metaprogramming a lot of Lot of magical tricks and my pipe cannot always understand them a Good example of that is the data classes module. That's new in Python 3.7 Or if you're familiar with Django, then Django models Are an example of metaprogramming that my pie wouldn't be able to figure out by itself So you actually are able to write plugins for my pie that help it understand Magical code This is even newer than my pie itself There isn't much in the way of documentation yet, and there's just a few working plugins out there We also had to write a plugin and If you need to go that way too, then you might find our plugin useful because it's got twice as many comments as it has code and So this is a bit hard at this moment you probably won't need it But if you do I'm sure this will all get easier over time the final example I'd like to share is overloading function signatures that means properly typing the case where Your function might take different sets of types of parameters and return different types of results based on the parameters a Simple example is is Having something that's indexable Like a list is but let's let's say it's it's your own type so here we could have a series of Numbers and we want to be able to index individual numbers, but also slices and We want my pie to understand that when we use a single index a single value is returned And when we use a slice then a sequence of values is returned So we begin by creating a generic class. This should be familiar to you by now like I said, it's a very common tool and What we need to do next is to explain the the two versions of the square brackets operator Which is called done to get item in Python. So these two They look like method definitions. They actually are But they don't have anybody again those three dots are exactly what you want to put in there and you annotate them with an with an overload decorator coming from the typing module and All they do is they explain to my pie What are the possible ways of of calling those methods and Then you just add the actual implementation which in this case is very trivial and That real implementation has to be typed so that it includes all the versions of the signatures that you mentioned previously So this is a bit wordy But it actually makes a lot of sense once you get used to it. The syntax is easy to understand. I had a lot more In store for this talk, but I had to cut a lot so to fit into the time limit So I only have seven takeaways for you One is to try and make your life harder by using stricter configuration than the default is second is to go bit by bit don't take Too much on in the beginning and go module by module and get to opt out when you can Definitely learn to work with generics and type variables because those are your friends You will be meeting them quite a lot learn to use protocols because they are very much in line with the Dynamic spirit of Python so you don't have to create Classes for everything just for static typing Be aware of new type because it can add more semantic to your types Not everything has to be a decimal or an integer. You can call it user ID or price or quantity Writing plugins is hard But it is so important for my pipe to spread and become more popular that I'm sure it will get easier eventually and the last example overloading Looks like boilerplate, but it's not very that complicated and it is useful. So Who here thinks that typing is complicated after this talk? I certainly do but There are good reasons for that one is that we have to learn and understand new concepts as developers and that is great because they They force us to think about our code more and in ways we perhaps didn't before and another reason is that the tooling is still quite young and It's developing Very fast very actively, but there still is a lot of issues to cover and if I made just one final sentence that is once this becomes more popular and More prevalent once we learn how to use this then all our code will be Much less error prone and development will be more fun. I can already see that in smaller projects Thank you One short short question if somebody is close to microphone Thank you for your talk. I have a question. So beyond primitive type annotations do you think that Optimizing for a human which Python says readability counts and that kind of PEP 20 things And optimizing for machine that my pie in this case is a zero-sum game or it's kind of something else That is a very good question zero-sum game At this point we are making concessions to my pie. Definitely. We are adding code and Structures to the code that we maybe would not do otherwise But I don't think the gap is too big and if it is done correctly then You might be making the code easier to read for humans as well when you alias your your complicated type annotations and Use them cautiously Then I don't think we lose all that much very little and we gain a lot All right, and it's very hot topic, but we are running out of time. So thanks again Vita for this nice talk