 Welcome to the next session. Our next speaker is Grace Kumar. Hello. Hi, Grace Kumar. So where are you from? I'm from Bangalore, India. Very nice. So it's very far from me. I'm from Switzerland here. So I see you are talking about type checks in Django. So type hints. So I'm looking forward to that. And some important things. If you have questions, please use the chat again in the conference 3 need channel. And at the end we can come to the Q&A session. Okay, let's start. Welcome to my user python talk. I'm Jack Django app. Who am I? I'm a software engineer based out of Bangalore and Grace Kumar. For the course of the talk, I'll talk about recently introduced gradual typing in python. Very specifically for Django. Through all this talk, all the examples are recorded using the latest version of Django 3.2 and the static type checker used is my pipe. Types at runtime. Python is a dynamically typed language. So while you're creating a variable or creating a function, there is no need to specify the types of the input and output. So to find the types, but still behind the scenes, Python data holder expression has its own time. So by default, Python interpreter has this method called time where when you pass a data holder returns the time. Here is a small example. Type function takes a tuple of two elements. First element is library of string time and second one is integer and the type function returns the output as tuple. Here is a user object, Django inbuilt user object and the result of the filter is path on board and it returns query set. Static checker type. Similar to type, my pipe Python static type checker has got a inbuilt method called reveal type. So here we have a tuple address and port as its elements. When you pass it to reveal type the type is revealed as tuple, but it goes one level deep and also specifies the types of the elements that are string and integer. Same thing. Now we are passing output of the user manager's filter method. The reveal type says it's a user manager of user. As noted, like the reveal type is much more much more in depth. It goes one level deep and tries to find the elements as well. This is the difference between the dynamic behavior at runtime and the static typing behavior. So MyPy has a configuration file. Since Django app does not contain default type, since we need to add certain information to MyPy config so that MyPy can understand the custom types of Django. The first thing is for the course of the talk and to work with Django we need to use a third party library called Django Stubs, which is an internet organization called type Django. So first we need to mention that MyPy can find the custom entry point for Django in this plugin called MyPy Django plugin. Once we mention that, then the whatever settings that plugin needs, we mention it in a different section that is MyPy.plugins.jango-stubs and here to work with Django app we need to mention the Django settings. Once you mention these two pieces of information then MyPy static checker can internally figure out what are the types being used for example, what is the request type, what is the response type, what is the user manager type all those details can be inferred using this settings file. Rotations in touch. So how do you rotate a piece of code in Python? So you have a variable name separated by call it and you specify the type of that variable then the value here lang call you specify the language variable of string year code and date says year variable of type date. Same syntax follows for a function or for a method. So here is a function called sum which takes in two arguments A and B and both are of type integer A called a name, B called a name specifies it's an integer and the return type is integer. We are using arrow to specify the return type similar example we have for initializer for the class person self name is of type string age is of type integer is alive is of type boolean since initializer does not return any value then no need to return no need to annotate the return in a type in Django code let's start with simple view function you have view function index it returns a response so always a view function takes a HTTP request and returns HTTP response HTTP response can be of multiple subtexts let's say an example of that here is a simple view function which returns HTTP response not found there are three different ways to annotate the same function the input argument is same HTTP request but the output can be annotated three different ways in the first example we say the HTTP response second example we say HTTP response not found third we say it as object so now mypy according to mypy all three are valid return types now we will say how mypy is able to understand these three different types these are the different classes and not complaining they are not exactly the same so every class has got this method resolution order method resolution order returns the order in which from child class to the parent class hierarchy for example HTTP response inherits HTTP response base HTTP response base internally inherits object now let's another example HTTP response not found HTTP response not found inherits HTTP response HTTP response inherits HTTP response base and object since HTTP response not found is a special case of all these three cases mypy did not complain because they are they form this they are part of the same inheritance check so this principle is called LISCOP substitution principle according to LISCOP substitution principle that an object oriented program substituting a superclass object reference with an object of any of its subclasses the program should not break next is Django models create method here is a simple model question which has got two explicit fields the first one is a question text of character field here is a helper method create question which takes a question text and returns a question instance this is like a simple way to create a question object next is a read to get an object from the database we normally use a filter method or a gate method here we have used a filter method here is a method and here is a function called gate question which takes a question text and returns a question it uses filter internally it does question what object is dot filter and dot first now when you run this code I guess my pay complains that incompatible return value it is saying like the question what is annotated as a return type is incompatible and let's see what it says optional of any expected question so what it says is like the function that is annotated to return question but it is possible that it can return optional of any so what is optional here means none of the internals are first whenever the value is formed then only the first to return an instance will return none so my pay is able to catch that error that this particular implementation can return none even though you can be confident that this question takes existent database but it is technically possible that it can return a none value so my pay is warning that that it is possible that you are missing the none condition here how do we fix this there are two ways the first way is to go say that to the type checker that hey now I am changing it to optional of question that means this function can return none or instance of question second way is to control the config of my pay tell my pay not to complain when optional is returned so here is a simple example strict underscore optional so my pay works in two different modes that is strict mode or linear mode whenever you want none to be skipped you can say strict underscore optional is equal to false when you annotate it to return question my pay internally understands that if it is possible to return none but I will not complain when none is returned next is filter method so questions.objects.all returns a query set of questions same thing when you do a questions.objects.filter returns a query set of question that is query set always carries a empty is either an empty query set with models so query set cannot work alone so query set is a box type whenever it is carrying an items inside it all of the items will be of the same type that is of same model here is a simple example query sets.objects.filter and objects.exclode you say query set of question so how do you say it is off it is like user squared brackets to mention that outer type contains the inner type here the outer type is query set and inner type is question some other method will return queries that all it reverse etc now let's see aggregate function aggregate functions before getting into the code here we have two models one is a publisher model which has got an explicit field name there is a book model which has got six explicit fields name is of type character field page of integer field rating is of float field publishers of orn key 2 publisher model update is a dead field now here we have a function called get average price so what it does is it does average SQL aggregate function all the books present in the book table so average average underscore price is equal to abg of price where you mentioned the price is a column name the value will be contained in the average price variable so if you run the query you can see the result to be in a dictionary average price as a key the value in decimal now how do we annotate this we know the return type is going to be dictionary so we can say the dictionary we need to specify the type of the key and values type we know the type of the key is going to be string and we know the values type is going to be decimal so we say bit of str comma decimal always keys type will come first values type will come second next is annotate method now we have a function called count by publisher here now we wanted to do is we wanted to count the books published by publisher for example if it's a penwheel we want to specify how much penwheel has published if it's vintage how many books has vintage published so what we are going to do we are going to use annotate method so our annotate dot num books is equal to count of books so which will say like it will group the publisher by name and return the number of books returned by the publisher and it is going to return dot values okay now if you see the output it looks something like this a list of dictionary because there are more than one publisher percent and each key has got so the dictionary like structure has got two keys one is name other is num underscore books now if you remove dot values then what happens is annotate returns are very now count by publisher is not only returning name and num books value returns are what to say the query set now there is one more function called print underscore which consumes the count by publisher there are two word blocks here first is if it's not second part is for loop the if block the num books is value is passed the return result from the count by publishers further filter and any book any publisher which matches the number of books or greater than the number of books only those results are returned the for loop we print the name and the num books now let's try to annotate these two functions very specifically count by publisher and see what might I say to us if you see the output it works as an inspector name of the publisher number of books so this is a legit code excuse me since we saw this output looks like a dictionary so what we can do is we can actually annotate the output to look like a dictionary since we know the keys keys are static that we already know it's going to contain two keys and what are those values so we can use something called type tick so type tick helps us to annotate when there are more than one keys in the dictionary and when the key and values types are known in advance in our case we know it is going to be name which is of type string, num books of type integer and we give a name called published book out now since there is going to be more than one result we are going to say it's iterable since we are using a for loop so it is understood that iterable is a valid type so iterable are published to book out now let's see what does mypay say about this rotation now mypay complies with two errors, the first it says at line number 46 that is wherever we are annotating say count by publisher it says that you are saying it is iterable or published book out but it is supposed to return it is returning very set of any that means like annotate is returning very set of any but you are saying it returns iterable or published book out and further inside that for loop it is saying iterable or published book out has no attribute filter it is like inside the if block you see there is a filter further so now they are too valid yes definitely these are valid how do we fix this since mypay is confident that it is saying it is very set of any let's try to annotate this as very set of publisher now once you annotate that as very set of publisher and run this against mypay now mypay says first error gone so it is correct now publisher has no attribute if you look at the if you remember from the previous slides publisher had only one explicit attribute called name num books attribute was not declared in the model since it is an annotate method jango dynamically adds this attribute num underscore books in the return result and because of that because of the dynamic jango behavior mypay is unable to understand how this num underscore books was added to the result this is a valid use case because there is nowhere in the jango declaration that num underscore books was present how do we fix this this is actually a bug in jango because annotate method since it can add values to the result at run time it is not possible to annotate properly so what there is a workaround for this so from typing module import type checking and you can use this type checking block to declare certain type of types which is available only for mypay and not available during python run time so here what we are doing we are creating a new type called type publisher which inherits from publisher first thing so we have the name model now we are creating a new field called num underscore books which is of type integer field and third we are saying it abstract is equal to 2 now this type checker now while type checking block is executor now the type publisher is available to mypay so what we are going to do is like convay publisher now returns query set of type publisher query set as I said earlier can only contain instance of the model so type publisher inherits publisher so it will be instance of the model so it will not complain now let's go back to the if block since if block contains like a filter method filter method works on the query set yes that condition is satisfied now let's go to the for block in for block we have dot name this type publisher has got dot name now num books has got num book is declared in type publisher yes we will not complain that these fields are missing now that we have seen some of the ways for at a Django code now it is not always possible to annotate source code from the beginning so there are some tools which can help you to get from 0 to some position and then you can manually add types I'm going to speak about one such tool this is called py annotate what it does is like we are running the python program it can infer the types of run time and it can convert the run time types into static time types or there is a plugin called py test annotate which can infer the types during this execution and can write it to a file and later you can use those annotations to annotate your code also I have shown example so here is a sample Django code which has got two views there is no annotation for request and response now you have some test case like this where you set up a test factory and then you make a HTTP code and then there is some kind of a response here in this code there is no assertion but in general there will be some sort of an assertion now that you have a source code now that you have a test code now you can actually run the py test plugin with extra option called annotate output this implies that you already installed py test annotate and then you can pass this extra argument called annotate output and the file location so once you do this at the end of the test you can see a file like this where each and it contains a list of dictionaries where each dictionary represent type annotation for a single data holder which could be which could be a python function or a method or it can be land expression whatever it is so the structure is similar which has got a path what is the line number, what is the function name what is the time declaration, how many samples did this test case use now that you have annotation now that now you can apply this annotations to this source file to cotton pirated CLI and say the type information is available in annotations.py file and you add type annotations to this particular file called pole slash views dot p by views python 3 syntax now what would happen if the type hints are missing then pirated will automatically add types to each of the available code parts for example in our case there are only two view functions it is added request as visgi request response or HTTP response you can see that view function 4040 it adds a HTTP response and the imports are automatically added though there it is not smart of importing HTTP response and response not one for the same module it has got two lines for it as you can also see that there is a drawback that visgi request is annotated as visgi request since this is a drawback how Django tests are run it is not actually HTTP request it is actually a visgi request because of that it is adding it as a visgi request but in general you can replace it to be HTTP request now some learning resources since gradual typing is new or it is evolving there are not much resources to learn gradual typing so I have come up with a project called python typing kons so what does this python typing kons do it is a github repository it provides various examples for learning concepts related to gradual typing the code is organized into three structures one is python, other is Django, other is DRF once you've grown the repository and run the summary, use list of all the files which don't have annotations where annotations are missing as a learner you open one up the file and start editing the annotations and then run this file here is the output from one such file so here you have a file called 103 model managers.py and when you run against static type checker it says there are three errors and you can see the error first one it says missing type parameter for generic type query so now the function is missing return type annotation now as a learner what you would do is go through the source code and you start adding type things for all the data orders like variables, methods, add reviews, functions all the expressions and then what you would do you would again run this command and you would see the errors now when the error moves from 3 to 0 that means you have solved all the errors now you can be confident that you have some understanding of how gradual typing works in python in very specific concept as well so there are around like 25 examples and you can go through one by one and the difficulty level is there in the file name medium here says that difficulty level has media so you go over the course of time you can see from easy you go to media and you will learn the concepts in gradual typing that's it thank you for taking my talk in case if you would like to contact me to it I am Chris the king in twitter have a good day bye thank you very much for your talk I have one first question is it possible to use type hints for lists with different data types for example hello 3.145 world oh type one here right, right definitely it is possible to use there are two ways to annotate one is a union type union of string, union of load and integers that is one way, second way is to say any but by default since it's a gradual typing any all variables will have a default type any so it depends on the use case I would prefer saying using a union type which will be explicitly saying the only ABC types are allowed to in this particular list okay at the moment there are not more questions so I don't see in chat someone sometimes you have to give people a little bit time for questions sure sorry for the technical internet issues no problem that's what's really funny about live conferences these things have to happen so it's more realistic and more fun than I mean we could just watch youtube videos that would be really boring okay I see someone is typing so let's just wait a bit sure I'm okay we actually do have question one second okay the question is which is the best way to start a Django project with type annotation okay this looks harder which is the best way to start Django project with type annotation what does this mean so if you are starting a new project I would say start with type annotation so it will give you a meaningful work but you have to also take in the consideration like is it a big project or is it something a big project or is it only a prototype that is one second thing is start using Django stubs that can help you first try to cover the the breadth as much as possible later you can move on for the very precise definitions like strict mode versus linear mode so that could be the approach I would suggest so at the moment there are not more questions so let me thank you again for this talk and maybe one more question will pop up in the chat so I hope you will be around there and we can make a small break and then we will come to the next talk thank you thank you