 So now we have a talk on should we return to Python 2? In this interesting talk, we're going to be discussing the migration to Python 3 and Python 2 relics that are still there in the code. Our speaker is Miroslav Shedibi. He is a software engineer at the Trayport company based in Karlsruhe in Germany. I think it's going to be a very interesting talk. So hello there. Hi. Hello. Happy to be here at home with you, all participants of your Python, looking forward to meet you in person next time. But now let's have a look at some Python 2, Python 3 code together. So the question is, should we return to Python 2? The answer is no. Well, maybe why? Why shouldn't we return that and why I'm asking this question? Last year, at the beginning of last year, I just put some sort of joke on Twitter that people should abandon Python 2.7 and move to Python 3 finally because Python 2 doesn't work in 2020 anymore. And then later last year, I had some time to have a look at some projects and contribute to some online projects, GitHub projects of Python, and see whether they have already moved to Python 3. And most of them did. But I had an impression that they kept some sort of backdoor in their code and kept a lot of Python 2 code within their Python 3 code, but a lot of code that works for Python 2 and doesn't really make sense in Python 3. It works in Python 3, but you probably don't need it in Python 3. And then I wanted to look at it. So my name is Miroslav Szyini. I'm now based actually in Vienna together with Trayport Austria, company that uses Python for online trading of energy and commodities. And I'm every day using Python to make the sunshine, the wind blow and the gas flow. You can join our booth at the conference in the conference chat and I will be happy and my colleagues as well to welcome you and speak to you about what we do exactly. But what is now my talk about? I suppose I assume that you have already migrated your project to Python 3 to some version of Python 3. So it works in Python 3, but maybe there is still something interesting that we can have a look at that may be a possibility to improve your code and make it a future proof. The question is which Python version should your code support? This is the calendar of all the Python versions with the both parts in which months they are fully supported and the new features come in and when they are only security fixes. So we are now in summer 2021 and you see Python 3.9 is now the main version. 3.10 is somewhere in the near future. Everybody knows about it, but it has not been still fixed. And 3.8, 3.7, 3.6, they are still with security fixes provided with security fixes, but they are not developed anymore. And 3.5, 3.4 and 2.7, they are dead. Simply dead. So forget about them. So it means now which version should you support? Now there are two possibilities. One is you have your app, your program that runs on your computer and that should be like really supporting the latest features then go with the Python 3.9 because this is something that is currently developed and later in a few months you can switch to 3.10 and so on because nobody depends on you. But if you have a library and you expect other people to work with it then don't stick to a single version because if you want to go to the future your code has to be maintainable and your code has to be upgradable also from the point of view of the other application authors or programmers. This means that your code has to support at least two or preferably more versions so that if I am as an application developer I want to switch from one version to another then both of them are supported at the same time which means that if you have a library right now I would recommend you to support 3.6 to 3.9 later at 3.10 then drop 3.6 and so on and so on and then just move this cascade further and further. In general, if your code runs Python 3 of course your prints are functions you are iterating over the dicts not with iter keys, iter values, iter items but directly with keys, values, items you don't use ease range, you use range and probably you still have some pieces of code that uses 6 which was a library to support both Python 2 and Python 3 now you can probably get rid of them then check if you somewhere in your code have an if part where you do something different for Python 3 and Python 2 these are all the places that you can actually get rid of or if you try to import some library that is in Python 3 but not in Python 2 then you have probably some try except with import error this is everything that you can already get rid of so just check your code and before we touch the code actually we must make sure that it runs so if it is a GitHub project you may probably have already some nice running structure with GitHub workflows and probably you are using TOX, I hope so your TOX file very simple lists all the Python versions that your code should support and then you just run PyTest, do you run PyTest? You should try the PyTest and all this is run within GitHub workflows so you have some simple ci.yaml file that supports exactly these versions and now it is 36 to 39 but later you will update it to 3.10 and remove 3.6 and so on and this is something that in winter when I contributed to some projects I just checked out they are still supporting 3.4, 3.5 so just drop it and move to the later versions and this is something that every year should be touched every time a new version of Python comes out now the content of code also evolves because you also have to change your code because code in Python 3.5 is not like 3.9 and then there is this nice overview of all the main features of Python versions for my friend Jürgen Gnach and here you see for example if your code supports 3.6 you can already use f strings you can use underscores in numerical laterals but if your code is only using supporting several versions then you cannot use for example Volver's operator some generic types or the latest pattern matching because it is only in 3.10 so you have actually to take care of what can you support but from 3.6 you have already plenty of beautiful features then there are some places in your code for example at the beginning coding UTF-8 remove that line you don't need it because in Python 3 that's default if you construct a class that inherits from object don't mention this object you can just remove it because per default it inherits from object your super function call also the default is there that you are probably using and remove it make your code simpler, more livable there are for example Unicode literals you can remove them because everything is Unicode per default in Python 3 and there are plenty of such small things that you can do automatically more or less automatically but fortunately there are tools for that and for example Autonistotile wrote a great tool called PyUpgrade that you just apply on your whole code and you say I want at least Python 3 or Python 3.6, 3.7 and it tries to update your code as much as possible without doing anything too much and then your code will be automatically more modern you just do PyUpgrade and then list of files and then it is quickly and you can div and put it in Git as a new commit and then you see actually what changes have been done also make sure that the code looks good that your divs are minimal and every div is like needed because if your code if you want to add some new parameter in a function and you have to add commas before, after that then the div is not clear but if you use for example something like black, blue or yellow then the code will be more legible the divs will be more legible and more logical also make sure that Flake8 or PyLint are happy that you just run take Flake8 and you see if there are any issues with your code and fix them and then your divs and modernization will be much easier and much more transparent PyUpgrade will also touch your streak formatting so of course you should use f-strings because they are cool in most cases but it will not do everything because sometimes the f-string is longer than the older version so it won't touch but check it, see the difference and with your human intelligence, human eyes just say I can update it, it looks better after that there is one place where you should keep the old formatting and that's exactly in the log messages but this is something that a great one to touch and there are also some issues, for example here this was in PyTables, there was one line of coding let in one and I wanted to remove it like to make everything UTF-8 and the logic behind that was much more complicated and actually I had to do something that was not so visible at the first sight but that worked at the end so from the three uninventive characters I created a longer string than it works but this means that I was able to touch the code and to change it so it works again or that it still works because if you have code that you are the sole maintainer of the code and you are afraid of touching it because it works, then it's a bad code you should be able actually to touch any line of code that you are a maintainer of and then the code is probably much better much easier to maintain the Unicode characters of course if you want to type my last name you can type it directly as Unicode you can also use the code points but if you want to make it readable and you want to stick to ASCII only then you can use the last version which enters directly the Unicode names of those characters Numbers, there is something that has changed in Python 3 comparing to Python 2 and that is dangerous if you don't take care of it so for example, math floor, massale and round before these numbers returned flows now they return integers so in the old code that works both in Python 2 you have something like that but imagine there are plenty of young developers who have never seen Python 2 code and when they look at this why, mass floor, massale, round they return integer, why do they convert it to another integer? So please remove all the integers the reason why this has changed was that earlier there were not enough integers because floats was longer than the range of integers so they kept floats as a result of these three functions now integers I have undefined or like infinite range and that means that now it's easier to work with the integers and actually you should always work with integers they are much more sane to work with than floats so we remove the integers this is exactly the reason because in Python 2 the maximal integer was some 9 billion, trillion, quadrillion and the float was 10 to the power of 300 so now integers are not limited another reason or another piece of code that work both in Python 2 and Python 3 but they don't really make sense in Python 3 is for example something like this you want to calculate the average or the arithmetical mean of these four numbers if you do something like this in Python 2 then you will get the integer that's just wrong so usually what you would do you would convert the first part, the second part to float or you would multiply everything by one but in Python 3 integer divided by integer returns a float already so why would you convert one of those into floats or multiply it with one doesn't make sense so just remove it and do it the standard way if you are still in the migration just from future import division and your Python 2 code will behave the same way as Python 3 anyway so now this is actually correct and do it simply integer divided by integer and keep the integers as long as possible because as soon as you convert to floats you lose some precision and every time you calculate anything with floats you lose the precision because God make the integers all else is the work of man use integers for example there are some floats that even don't exist if I take this number in the first line it doesn't exist as a float because at this range, this high range of numbers the difference between two following two consecutive floats is already two it means that there is 992 and then there is 994 there is no number with 993 at the end as a float so and you think it is a huge number oh the number is not so huge the number is the number of nanoseconds in 100 days so this is probably a number that maybe you are going to work with and still if you then you have an offset of one then it's probably wrong so don't use floats for things that you can use with or work with integers another thing with floats is that float is some approximate number if I take a look and cut one meter from it then it is never one meter it's just like around one meter and if I take number 0.1 and then and I add it three times then I will get a number that is not 0.3 because there is no number 0.3 in the floating point range of the standard that Python uses and if you do even more calculations so for example, here I add the number 0.3 one million times so plus plus plus plus plus and then you see that I get a number that is not 300,000 I get some number that is next a little bit next to it but it's even wrong it's not the nearest float it's a float somewhere nearby if I want to work exactly I can use decimal so decimal 0.3 that is exactly 0.3 in our logic of decimal number system and then I will get the right number or I can use fractions so for example, 0.3 is a number that is also if I add it one million times I will also get a number that is correct but you see on the right side there are the seconds or millisecond seconds how long time this calculation needs so if you need something really exact like you are working with finances then use of course decimal even if it takes a little bit longer but if you work with some physical something like temperature or the length of something then use float as it is because it's not exact anyway so that's okay another place where you shouldn't use floats are for example numbers that are exact so for example one hour is exactly 60 minutes not 60.0 minutes so use 60 minutes and only if you divide if your number of minutes is for example 90 then 90 divided by 60 is already 1.5 approximately although it is exact but in the point of view of a Python it is approximately but it gives you the right result even if you add or subtract it the same for averages so you don't have to divide by 2.0 because you don't have 2.0 items you have two items so divide by integer so in your code check this is something that you cannot do automatically you have to check manually or search for all occurrences of .0 in your code and you will see that there are plenty of spots like this that use two exact numbers but they are too exact only because Python 2 did these things differently wrong as you see it but they did it differently and now if you write Python 3.0 write it correctly divide by 2 also if you want to have like the rounded you can use math floor or integer or then slash slash that also gives you an integer division round that's interesting why should you actually use round I never use round round gives you now an integer that's fine okay that's maybe use it where I could imagine using it but in Python 2 round returned a float but there is no float that is exact some number but there is a float that is approximately some number and even there is something like this round to two decimal places that's plain wrong why would you want to round to two decimal places there is no float this exactly two decimal places exact there are some flows that are nearby and if you want to you have to get to decimal places that's probably for printing because of course if it's a human I don't want to see a floating point number like this I want to see something that I can read so with two decimal places so of course this is something that I could use but then I will do it when converting to string my string can have look like two decimal places although the number behind that is random or random it's a floating point number that has any precision or what I can do is like this with equal sign it's the debugging function of the recent Python allows you actually to print it even nicer another thing are percentages why would you work with percentages it means that multiply by 100 if you want to print something because usually you don't calculate work with percentages you work with one divided by seven so one seventh but you don't work with 14.3 and in a Python you have the possibility actually to format your string with the person and then it will multiply by 100 and add the person sign and do the math everything you need numeric literals if you have like a big number and you do something like this 10 E9 is a float so if you have one second as an integer multiplied by 10 E9 then you will get a float which is not exact you probably wanted to have integers so what you can do of course you can write it like this but that's not eligible now you can use underscores it looks a little bit better or 10 to the power of nine that's also an integer that's maybe better than 10 E9 these were numbers then now let's have a look at the path that's something that you absolutely should use because a piece of code that's used for some CSV files reads them and processes them within a directory you can write it like this and it looks much, much, much better you don't work with paths as strings you work with them as objects and you have object oriented so with attributes you can access everything parents, names, stem, suffix everything you can create directories you can check everything so do it like this and also what is better is that if you work with strings then you work with paths as strings then you have the strings and you don't, you see a string but you don't know is it a path or not but if you have a path object and you pass it around your functions then it is clear you are working with path objects and it's great so have a look at this check it and really you can rewrite your code and make it much, much, much more readable and of course if your library supports 3.6 to 3.9 then for example unlink of an existing or missing file is not possible before but from 3.8 you can already use it so if you have library that supports several versions of Python then you have to write it the first way but then later rewrite it the second way and it will be even nicer so you have to write nice code with paths you can also use PyTest as you told me you are using PyTest already so that's great so for example here you want to test something and you want to have a temporary file and the disadvantage of this is it creates somewhere in your temp directory some file that doesn't have a really nice name deletes it afterward so you don't really get it when your test is done and you cannot really see it but for example in Python also in PyTest with Python 3 you can use temp path and that's path somewhere in the slash temp I will show you where and then you can work with file names that have normal names that mean something to you and for example in this case it will create temp PyTest of the user mirror PyTest1 then this something is the name of the function and then the name you gave it and PyTest1 is the first instance of my PyTest and then if you run it again it will create directory PyTest2 with all the test functions below it and then Python 3 and so on and default is that it will delete it will keep only the last three PyTest runs and it will delete everything before so it means that every time on your system in slash temp you have the last three runs and you can check you can see directly the files and they have meaningful names so that's beautiful use it the time is the last thing we have for example in Python 2 you could write something like this if you wanted to have the unix time stamp so the number of seconds since 1970 of a daytime but you had to import calendar so another library another module for standard library and then do something like that that's not beautiful but it works in Python 2 and Python 3 now you can do something like this so dt.timestamp and you get the number also now you don't don't have an excuse that you should use shouldn't use time zone of airtime because in the standard library there is daytime time zone UTC for the standard UTC stuff and now even from 3.809 there is zone info which uses the TZ library from your system in the standard library so you don't have to import PyTest or something like that and now the dates they just work beautifully and you can really work with that correctly and explicitly with the time zones I know time zones are difficult but another thing is time time so if you have some expensive operation you want to measure it time time gives you the number since epoch and then again but the problem is if during run expensive operation your system clock will be the correct time using NTP then end-minus-start can be a negative number so in that case use perf counter that gives you the number of seconds each time you call it and this is the number of seconds since some point in the past but it is always increasing or process time this is for CPU time or for real time all of this and even more wait for a few weeks and the second edition of the Fluent Python book by Luciano Ramayou is going to be published and it's a book of 1000 pages that contains really plenty of information or ideas on how to do such beautiful stuff with Python it is actually the best second book on Python that is available for disclaimer I am one of the technical reviewers so the question is should we return to do you want to keep the backdoors in our code? Here's the plan when someone uses a feature you don't understand simply shoot them this is easier than learning something new and before too long the only living coders will be writing in an easily understood tiny subset of Python 096 that's all thank you very much thank you very much for your interesting talk I think there was a discussion about blue versus black packages so if if we could maybe know what you think about this issue it's your choice stick with one version like with black or blue or black the problem is that sometimes they even change their mind because they update from time to time so I would really stick with one version of black and then update it once a year when you also update the version of Python and you also update some syntax of your code but what do you think about blue and black? what is your opinion? I think the blue one is better because it's newer it has a different opinion on some stuff that has been criticized in black okay thanks a lot for your talk today we're very happy that we had this amazing opportunity to listen to you thank you very much I am available at the booth of Treyport, happy to chat with everyone and later meet you in person thank you very much