 So, welcome everyone. As Sradek Vokal said, if you have three seats on your left or right, you are doing it wrong. So, please squeeze into the middle so we can fit as many people as possible into the room. So, I'll repeat my wish. If you had room on your three seats on your left or right, please move to the middle of the room so we can have as many people as possible in this room and it's always easier for people that are entering to have a seat on some side and not try to move into the middle. So, I still see some backpacks are also attendees so please remove your backpack from the seat if you are not expecting someone else and if you are not really expecting someone else and help us to squeeze as many people as possible into the room. Squeeze, people, squeeze. I know you can't do it. So, I would like to encourage you to ask questions during the presentation or after the presentation. You can get really cool DEFCOMF scars to get warm in these cold days. I would also like to encourage you to vote for the talks. You can use the web application for the schedule. You can also use the mobile application for voting. I would like to also encourage you to use to write blogs and tweets using the hashtags. Could those few people who have free chair somewhere in the metal middle just do a shift so we can really when we say we are full we are really full. I see there is a free place. Okay, so free backpack for someone. Cool, thanks. Okay, so welcome everyone. I would like to give the stage to the next RedHeader. This time it's Petter Victorien coming from RedHead's own developer experience team. Petter has a lot of experience porting stuff to Python 3 so he's gonna tell you what awaits you in Python 3. Go ahead. Hello, I didn't expect so many people. So, I'm Peter. I'm a member of the Python maintenance team at RedHead and I would like to ask you, who of you has written some code in Python? Everybody. Who as you have tried to port something to Python 3? Everybody. Who of you hated the experience? Wow, not so many people. That's actually great. Okay, so I guess you know what Python is. I guess you know that Python is a really nice little language to use. It's also a very old language. Actually, the first version of Python came at the same time as the first web page or the first version of Linux. And from there it kind of evolved. It got some new features when people were doing something in a very roundabout way. Python evolved and enabled them to do it some better way. But all this time, all the old features that weren't really the way to do it anymore stayed in the language. So, the language is really backwards compatible. Programs that were written for Python 0.9 will probably with some slight modifications still work today on Python 2. But eventually there came a time about seven years ago, when people said enough, there's too much stuff in the language. We need to clean it. We need to make it so that there's again one obvious way to do something. And that way is performant. That way is the good way to do it. So, someone who comes to the language and tries to learn it doesn't have three different options to do something. Yeah, so this happened in 2008, around the same time as these projects showed up. As an example, I'd like to tell you one thing about function definitions. Since the first version of Python, you could define a function like this. Somewhere pretty early, Python grew the ability to define functions with a variable number of arguments. So, you could define function like this and call it with how many arguments you wanted. This wasn't in the first version and it looks kind of funny. If you don't know what this does, then you need to go to the manual and find it out. And because it's a star, you can't Google for it very easily. So, things like this are actually not encouraged. There's a very high resistance to adding something like this to Python. Maybe you know that instead of ampersands, like in the other languages we have and, which is a word that I still can't Google, but at least it's a word. So, with functions like this, Python could no longer need it to have print as a statement. In the first version of Python, this was some operation that takes a variable number of arguments and that's something functions didn't do. So, print was a statement. In Python 3, they said, well, we have functions now that can take variable number of arguments. So, let's make print a function and make it work like all the other functions. Make it so that somebody who learns Python doesn't have to learn a new special case for the print statement, but make it so that when they learn print, they know all about what functions do. And it may not seem like a big change, but when you look at the way print printed to a different file that's standard output. In Python 2, it was, again, very weird. Again, something that you really need to know the docs for and something not obvious at first glance. Python 3 used named arguments like with every other function. So, this is something that hits everyone that starts to port to Python 3, but for anyone coming to the language for the first time, it's a good change, I think. Another example of this phenomenon is the list return in functions. Since very early in the history of the language, there was a range function that gives you a range of numbers in a list. So, a list with 0, 1, 2, 3, and so on. And this is heavily used. It's used in almost all the four loops. And it turns out if you have a giant list of numbers, it takes up a giant amount of memory and you might not need it if you're just using one of the numbers at a time. So, they added the function x range, which is probably not something you would Google for. But it returns an iterator, which happens to have the property that if you iterate over it, it only allocates one object at a time. You don't have the whole list in memory. In Python 3, they said this is the good behavior. This is what we want to have in all the four loops. So, they renamed x range to range. And if you want a list, you just call the list constructor on it. A similar thing happened with the dictionary methods that give you a list of all the keys in the mapping list of all the values, list of all the key value pairs. Again, you don't need the whole list if you're just iterating over it. So, they added iter keys and so on. And then they said, well, we can't really replace iter keys or keys with iter keys, because sometimes you want it to behave as a list. Sometimes you want to ask, is this key in this list? Turns out these should behave as sets or, well, please, the keys should more like be like a set that's linked to the actual dictionary. So, they added that. So, now you have view keys. These are already in Python 2. You can use them. It gives you an object that's linked to the original dictionary. And you can ask if a key is in there. You can use the set operations on it and so on. And in Python 3, they took these and renamed them to these. So, again, if you need a list of the keys, you need to call this on the result of the keys method. Back to the questions, what is after the views. I guess we have to wait for that one. As you'll see later, this process is still continuing. We're solidifying the basics so we can still build on top of them. And I don't think this is a bad process to do it. It's just that once too much craft accumulates, you need to have some more or less disruptive change to get rid of it. So, the most disruptive change of all is probably the strings. I would like to take a moment to see what the string really is. I would like to start with showing you an image. You have a picture. And everybody knows that if you have a picture and you save it as a file, you have all these different formats you can encode it as. You know, JPEGs, PNGs, GIFs or GIFs, whatever. Some of these are small. Some of these are big. Some of these, like the JPEG, don't really capture everything in the image, but sometimes it doesn't matter. And when you have a bunch of bytes in the file, you always have to load the image. You always have to decode these bytes to actually get something you can display to the user. And it's the same with any other file type. If you have an mp3 music file, if you have a WAV music file, it's pretty much the same thing. The only exception the programmers in 1991 thought is plain text, because there's one mapping of the characters to the bytes. Everybody knows that. It's what we look at our programs as when we say, you know, open this file. Especially in Unix, everything was a text file. And, you know, there's one way to put Hello World in a text file. Nobody uses the ECBDBC or whatever that non-ASCII thing was. The problem is plain text is a very naive term to use. These days you can't really use ASCII. Even if you wanted to straighten the quotes, even if you wanted to use the now-prepared English word with just one dot, you can't really ignore emojis nowadays. Sorry. And it's the same problem as with the images. You have a bunch of different encodings. Some of them can encode all of the text. Some of them cannot encode all of it. Some of them are pretty easy to use. Some of them are deceptively easy to use if you stick to the common cases and if you don't, they're horrible. Yeah, so, yeah, it's the same problem as with any file type. There's a difference between the text object itself and the representation of the text you would put on a disk or share over a network. So, Python decided let's split these two representations and let's have a text and the byte that represents really a different kind of object. Now, I didn't actually print the actual byte object here because it uses something that, well, when you print it out, it tries to look like a string for reasons that I don't really agree with. But conceptually, bytes is a list of numbers from 0 to 255 and text is a collection of characters. It's the same thing with numbers. You can have big numbers. You can convert them to a list of bytes and that's how you would represent them in memory. So, this all actually works pretty well. If you embrace this philosophy and always separate your bytes and your text, you will write better programs that have fewer bugs than the users try to put emoji in them. The problem with this is when you're dealing with low-level protocols that are explicitly based on the plain text myth and those are unfortunately pretty common. And the problem with these being low-level protocols is that the authors of the libraries that deal with these but Python 3 is really weird. It makes me do so much more work and they didn't want to port their libraries very much. Some of them still are very reluctant. So, the text bytes split doesn't really create new problems. It just pushes them to a different level of the stack. Of course, it makes you actually handle the edge cases which you didn't really have to do before if you didn't care if your program crashes or not or it gives you incorrect data. But the problem with the libraries is those low-level libraries have to be ported first and nobody wanted to do those for a long time and only after those were ported could we port the other things that are much easier. Sorry if you're maintaining some kind of requests library or if you need to handle all Unix filenames. That's life, I guess. Anyway, Python 3 is not all bad and not all unnecessary work. It also has some nice things. It has some things that you can use just by making your code compatible with both Python 3 and Python 2 which is a thing most libraries nowadays do because it's pretty possible to do it. Of course, the resulting language as a subset of both 3 and 2 is strictly worse language than both. Now, once you can drop Python 2, Python 3 is really a better language but if you have to support both, I'm sorry. I feel your pain actually. Anyway, if you run your program under Python 3, you have a solution to this problem. Here we have some code that catches an error and tries to log it, but the log function for the errors is not very well written. It doesn't open the file for writing, so it gives an error as well. If you run this in Python 2, you only get the second exception. This is a simple case, but if you have giant code base and many cases of these in a single exception and, of course, the error cases aren't very well tested, it can be a hell to debug. In Python 3, it gives you all the exceptions. It just remembers what happened before and gives you both of them. I don't know how easy this looks to you, but I've been saved by this a couple of times already. Another thing Python 3 added to make correct programs easier to write is keyword only arguments. In Python 2, you could have a function that takes some numbers or letters or whatever, and it has a flag, and then people would call the flag without the name, which when you read just the call, it doesn't tell you what the true stands for. This is why Python has keyword arguments, and you can call it like this. Everybody knows what it means, and usually in the documentation you would have, please use keyword arguments for this, otherwise we won't be able to change the API later, for example. So what Python 3 did is added a star. It's the same star as with the many keyword arguments, except the variable name is missing, which means you don't take any positional arguments for this function. You have to call it this good way, and in the next version we can add a new positional argument, and nobody's code breaks. Another thing that I don't know if you would use a lot, but I think one of the good things in Python is when you have some kind of sequence, you can unpack it into variables directly. All the dynamic languages do this, but C doesn't, so it's a good thing. If you remember the star from last time, they added it here as well, so now you can put the first thing in the sequence in one variable and the rest in another. And it reuses a concept that you were really supposed to learn because you can't Google for it. So the concept is that if there's a star in front of a variable, it means there's supposed to be some kind of list here, and you can call functions with it, you can define functions with it, you can unpack sequences with it, you can even make new lists with it, like this will embed one sequence in another. And it all makes sense if you just learn the one concept of what the star means if you're not using it as multiplication. A similar thing, if you have two stars, means there's a dictionary. So I don't know if you had to do this, but if you have a dictionary with defaults and a dictionary with overrides and you want to get the result, you need to do this idiom, which is found on Stack Overflow, and if you work it through, it actually works. You need to take a copy of the original so you don't modify it, and then you have to override your copy, which works, but now you can use the double star, which you should already have learned from function definitions. So it maybe doesn't look as weird as if somebody who didn't know the function definition meaning would think. I don't know, what do you think about this? Is it readable? You don't think so. So which one is more readable actually? The context is well defined because the double star means dictionary. If there was a single star, then it would be a set. It's the same with functions. You have star args and double star keyword args. Yes, that's a good point. That's something you will need to learn. The order is obviously what I have there. It's as if you made one dictionary first and then updated with the other one. Another thing that was added, I don't know how many of you do data manipulation, but if you were doing some hypothesis testing, you could find this formula in literature. All of these variables are either matrices or vectors, and you would use a library called NumPy. The way you would write this is this giant thing that doesn't look anything like the original equation. There's a lot of parentheses. Also, scientists are not very good programmers in most cases, so this is thoroughly confusing. And matrix multiplication, I don't know if you can see the operator for matrix multiplication. You can't because it's so common. You just put the variables next to each other and that's it. This was another case of Python adding an un-Googleable character to the language. You can see this thing. I don't know if it's readable, but it at least looks pretty similar to the original. And this helped very many people, but mostly people who wouldn't be at this conference and who didn't really have too much input into the language design, even though they were using it all the time. So after 20 years, they finally got their multiplication operator and explained to everybody that this is one case where it's really needed. There is no matrix addition that's different than normal addition. This is really something they needed. But you're not data scientists, so let's do something webby. So let's say you have some function like this. You get some web page from somewhere, you process it and return some result. The problem with this is this takes some time and it blocks your application. Your program could be doing something more interesting than waiting for some server to get the web page. What you can do is rewrite the function totally, have this await keyword, and this means that while this get is executing, then some other piece of code can run. It's a bit like threads, but unlike threads, you're explicitly saying at this time something else can run and that something could modify the global state, for example. The difference from thread is that I don't have to put a lock around my print because if I did it with threads and didn't put a lock around print, it would try to print many things into the same file and it would garble the file. So this is something that became popular with WebJS and it's a nice way to think about programs where you need to run more things in parallel. It doesn't solve multi-processing, it's still one single thread, so if you want to use all your eight cores, then you need to spin up multiple processes, but it's good for the case where there's a lot of waiting for the network. The problem with this function is that you can only call an async function with await or other special calling conventions and otherwise you need to run it in an event loop. So it splits the word of functions into two different kinds and you can only use one with its own kind, which doesn't look very nice, but you need to consider the situation we had before where we had Twisted and we had G event and we had threads and all of these were incompatible with each other. So what Python at first was introduced the async Iolibrary, which serves as a common ground where all these frameworks can plug in and all can interoperate, and then when it became clear that the syntax of Python itself wasn't up to all the edge cases, they added very well thought out two keywords that make this simpler. I don't know how many of you would use async code. Wow, so that was actually useful, great. So now something more simple. When you were starting with Python, you made a little module and you imported it and Python created a little pyc file that you put in your gitignore and told your editor to not show and so on, but then you remove your minemodule.py and you try to import the module again and boom, Python finds your forgotten pyc and it looks like the module is still there and it's doing the same thing it was before, but it's not, you know, it's deleted. What's going on? So this was one of the problems with this pyc. The other problem is that pyc is specific to one Python interpreter, so if you imported it with Python 2.6 and then with 2.7 and then with 2.6 again, it would regenerate the file all the time so the cache wasn't very useful. So what Python does now is, I don't know, probably you've seen this pycache which solves both of these problems because the pyc is not found when you're looking for the module itself and it has a little tag that, you know, allows more of them to work at the same time. Another thing that was introduced by some companies who had giant Python 2 code bases and were having a hard time porting to Python 3, well they didn't really try, but this is for them. So you can, in your function definition, you can specify what tapes it takes and then you can run a special static analyzer on the Python. It's something like pylint. It checks your code and checks if everything's okay and if you have type problems like, I don't know, the bytes text split and you put all the expected types in, it'll help you write more correct programs. It's not to be used in the standard library, so you can completely forget about it if you don't need it, but it's useful if you have a stable API. You can see now in programs like NumPy or even some other libraries that in the doc string you would have the parameter name and its type, which is not a really good solution to the problem because you'd have to parse the doc string and everybody writes different doc strings. So now we have the typing module that consolidates all of that. And I can see some of you are bored, so I'll show something that should be pretty new. It's similar to the thing I showed before with the range function. So Python says there should always be one way to do something. But for string interpolation, for putting things inside strings, originally we had the percent operator, which is not very well designed. It takes a tuple and it also takes one value that's not a tuple and in some edge cases that's not very helpful. So strings grew a format method that functions as a normal function. But that is still not enough because you have to repeat the variable name here and you have to repeat it here. And if you use named arguments, then you would say result equals result if you have that in a variable. So if you just want to put some variables in a string you have to repeat every variable name three times and people got so frustrated with this they actually wrote libraries that work around it by exploiting some very deep magic in Python code, which is a good thing because that's how the language evolves. It allows you to do these things, so it allows you to experiment and people experimented and said, this is really good, this helps me, this makes me write more readable code. And that's something I would like to have in the standard in the language itself. So after much discussion and much opposition and much fighting, Python 3.6 will have format strings with this little f where you can just put whatever expression and it prints it up for you and puts it in the string. It's important this isn't 3.6, which is not out yet, so you would have to compile Python yourself, but at least it's something new for all of those of you who knew everything already. So all these new features, can you actually switch to Python 3? Well, it turns out there are some statistics on how many of your dependencies are ported. These are statistics for all of the projects on PyPI, which is most of the pure Python libraries. 26 looks like a very low number, right? Most of them are projects that somebody wrote and put on the internet and nobody cares about them, not even the original author. So for some definition of actively maintained, we are up to 64%, for some definition of most popular, we're up to 80%. In Fedora, we actually now have a script that checks it and the number of ported packages is slowly creeping up, but even for me as a maintainer, it's more important that the number of blocked packages, the ones that don't have all the dependencies ported at, is slowly getting down, so that's what I personally am focusing on more. But still it's about 40%, it might seem like you'll know in a low number, but keep in mind these are packages like printing an image of a dog to your terminal, right? You probably don't need this ported to Python 3 right away, and if it drags in a different interpreter, well, who cares? So as you might know, in Fedora, we don't install Python 2 by default on Workstation, server is still blocked. If you want to use Python 3.5 in Fedora now, you need to either use Rohit or Copper, because we missed the release date by a few weeks or months. Well, just a really short time. Sorry for that. It'll be there. Yeah, and if you have anything you want to ask me, at the time. Is it possible to use it somehow for list comprehension to achieve like parallel processing? Yeah, so the question was, is it possible to use the async... Where is it? Is it possible to use the async for list comprehension, for parallel processing, pretty much the map function, but async, right? Yes, it is. This await will, or this session, will most likely return either a coroutine or a future. So you can use it without the await and just get the future, make a task out of it, just schedule it to run, and after you get it sometime to run and when you need the result, you can do the await. And there's a function that you put several of these futures in and it gives you the results as they come in. So, yes. It is full featured. I'm just using, you know, a very simple example here. The docs are still not really that comprehensible if you don't know that this is what you need. But with the new keywords, it's getting, it got much easier to explain it. So hopefully it will get better over time. Wanna scarf? Wait for the mic, please. All right. Would you recommend the 6th library in order to port code from Python 2 to Python 3? Or would it be better to just, you know, do everything by hand and save maybe some work? What's the best approach to do this? If you already have dependencies in your library, then I would definitely recommend 6. Because if you don't use 6, then you will reinvent the things 6 does and not reinvent them well probably. And it's a pretty small library. So if you can afford the extra dependency, definitely use 6. There are some other libraries too. There's Python Future, which is like 6, but it makes your code look like Python 3 code. I personally don't recommend it that much because there's a lot of magic behind the scenes. So 6 makes you, you know, know what you're doing. So I would definitely recommend that. And Python Modernize, yes. Python Modernize, or there's a thing that OpenStack uses, Sixer, which does even less magic. Yeah, Victor Stiner. Yeah, Victor Stiner is an amazing guy and he, yeah, whatever, he wrote to use it. Any more questions? I can repeat the question. Yes, yes. And that's, so the question was in Python 3, when you have accept, when you have accept and you store the exception at the end of the accept blocks that removes the variable from scope, so it deletes the variable. This is because now exceptions store tracebacks. So now whenever you would use that exception object, it carries its own traceback with it. This wasn't the case. And carrying the traceback has references to all the local variables of the functions and carries a huge amount of state. So in order to not keep this huge amount of state alive, for the normal case, it's better to get rid of the exception at the end of the block. What you can do is just assign the exception to a different variable and that'll work. Yes. It was just undefined after the accept. Right, so if you need to store the variable, put a different name here and assign it to the name you want. So the way it's implemented is it's as if there was a del at the end of the block. Right. Any more questions? So thanks, Peter.