 Dank je. Ja. Inderdaad, woon je code op de generatoren genomen? Dus ja, na deze presentatie heb je behoorlijk behoorlijk bezoek als kandidaten voor regevering. Je weet hoe de regevering in meer beheerbare en elegant pyfonic code is. En je hebt eerst de kwaaimtens met de standardlibraries Itertools module en de wondervol, meer Itertools package. Maar eerst laten we mezelf introduceren. Mijn naam is Jan-Hijn Buurmo. Ik ben de aandacht van de basic Python voor Java developers tutorial op real Python. En ik wist de eerste baby steps van Python heel dicht, meer dan 30 jaar geleden, wanneer ik een programma voor de Dutch National Research Institute voor Mathematics en Computer Science werkte. En in m'n karier werkte ik meestal in C++-developer. Maar ik heb altijd een eind op de ontwikkelingen van Python. En ik was zelfs nog steeds kunnen gebruiken in m'n werk. Mijn volgende employer voor meer dan 10 jaar nu is Ordina. Het is een benelux-based IT service provider. En toen ik er als Java unit manager was, kwam ik met de idee van starten een speciale Python-practice. En deze praktijk foberst op Python en software-artistiek. Ik denk dat het heel speciaal is om een dede Python-practice te hebben, waarin in een board-spectrum IT service provider-company je dat niet veel ziet. En de Ordina-Pythoners zijn ook een participatie-sponsor van de PSF. In de laatste paar jaar doe ik wat ik meestal, en dat is Python-programma. In deze presentatie ga ik jullie in een familie van bepaalde functies. Ze invoeren loeps. Ze containeren fragmenten van een bepaalde code. En ik zal je laten zien hoe je de verschillende verantwoordelijkheden binnen deze simulant en tangoord code kan ontdekken. Ik zal op het topic van generatoren functies en hoe deze generatoren functies kunnen aanbieden om je deze en tangoord code in verschillende functies te ontdekken. En finally, ik zal je uitleggen met de utility-generatoren en functies van de Ittotos module en de More Ittotos package. Dus ik zal je een fictieel verantwoordelijkheden vertellen. Niet een echte verantwoordelijkheden over het Python-team die de functies van een product-owner ontdekken zal ontdekken. En deze product-owner is verantwoordelijk de Fibonacci-sequence fanclub. En de product-owner wil je een functie maken om een list van alle Fibonacci-numbers minder dan een bepaalde threshold, zoals N. En dus eerst, wie weet wat de Fibonacci-numbers zijn. Nogmaals. Dus ik moet het niet uitleggen. Ik zal het wel doen. De definitie van Wikipedia staat, in Mathematica, de Fibonacci-numbers van een sequence. De Fibonacci-sequence, in die elk nummer is de zon van de twee precedende mannen. Dus dat is de essent. Je begint met twee nummers. Normaal, de meeste keer, 0 en 1. Er is een debat over de begrijpenvalu's. Maar laten we deze eens gebruiken. En de volgende valu is simpel de addition van de vorige twee. Dus in dit examen, 0 is de eerste, 1 is de tweede Fibonacci-numbers. En de volgende, de derde, is de zon van de vorige twee nummers. 0 en 1. Als je ze aanbiedt, dan krijg je de volgende valu, die is 1 weer. En de volgende valu is de zon van 1 en 1. Zeg het samen, die is 2. En 1 plus 2 maakt 3. 3 plus 2, 2 plus 3 maakt 5. 3 plus 5 maakt 8, et cetera, et cetera. En dat is hoe je deze Fibonacci-sequence opbouwt. Dus de volgende stap is om dit in Python te implementeren. Een beetje googelen zal je bij Python zijn documentatiezijn, die een implementatie van precies deze functie heeft. Deze implementatie, wat je hier ziet, heeft een andere naam. Het heeft type annotaties en een docstring, maar het is functioneerlijk equivalent naar dat examenfunctie. Dus wat doet het? Hoe werkt het? Eerst kreeg je een eindelijk makkelijke resultaties, een container. En subsequently, twee variables zijn initialiseerd naar de eerste twee Fibonacci-nummer, de 0 en de 1 die ik aanbiedt. En je kunt zien deze variables A en B als Fibonacci-register. Register A holdt de current Fibonacci-nummer en register B contains the next Fibonacci-nummer. En dan enter je een loop, in dit geval met een exit criteria, dat de current Fibonacci-nummer moet minder zijn dan de threshold-value, de gevolgde threshold-value. En je hebt gewoon de current Fibonacci-nummer op de list. En dan begint je te calculeren de volgende set van Fibonacci-register. En dan gebruik je dat hele mooie feature namelijk sequence-unpacking van Python om deze calculaties in een go. Dus je maakt de nieuwe current Fibonacci-nummer tegelijkertijd de value van de oud-next Fibonacci-nummer. Dus je maakt de value van de oud-b naar de nieuwe A en de nieuwe B krijgt de aarde van de vorige twee values. En finally, when the loop exits, then you return the result list. So you present that to the product owner and the product owner is happy. He says, oh, that's nice. Can you make another function for this Fibonacci fan club? And now he wants us to make a function that simply returns the end Fibonacci-nummer. So it's an indexed number. And it's zero-based counting from zero. So Fibonacci-nummer zero is the first Fibonacci-nummer, Fibonacci-nummer index one, en one is the second Fibonacci-nummer, enzo on. We just make it zero-based. He's facilitating us programmers to make the stuff zero-based. So let's go to this new feature. You see the implementation right here. Again, you see the initialization of these two Fibonacci registers, but you don't see the initialization of the container that holds a list of values. En you enter, in this case, another kind of loop, not a while loop, but a counting loop. You're using the range function to simply count the number of iterations that you want to go through. And the underscore is just a dummy variable that holds the countervalue that you don't need, that you don't use. And within the loop, you repeat this calculation of the next set of Fibonacci registers, A and B. En once you're ready, you have gone through the number of times of loop of the loop, then you simply return the current Fibonacci-nummer. So we could, in our faults, we could try to mimic how the processor works in this case. So if you would pass n equals zero as the threshold or as the index, then you initialize A and B to zero and one. The range function does not cause any iteration of the loop, so A and B remains the same. You return A, so you return the value zero, which is Fibonacci-nummer zero. And one more example, if you pass one, then you assign, then the loop is executed one time. So then the A gets the value what was before that, the B value, which is one, and then you would return one, which is indeed Fibonacci-nummer on index one. So, you demonstrate this to the polytoner again. The polytoner is still very happy, gets even more enthusiastic and says, okay, well, can you make two more functions for me? One function that returns the first n Fibonacci-numbers and another function that returns the smallest Fibonacci-nummer greater than or equal to a certain threshold again n. So, more features. I'll quickly present you the two implementations of those functions. Here again, you're returning a list. You have the A and B registers again, a counting loop. You append the result as long as the loop count holds. And after that, you return the list. And finally, a bit of a different function, smallest Fibonacci-nummer greater than or equal to the threshold. You initialize the registers again. You enter the loop while the current Fibonacci-nummer is less than the threshold. You calculate the next Fibonacci-nummer. And once the condition of the while loop does not hold any longer, then you have a value which is equal to n or greater than n, and you return that number. So, you have now a set of four functions. And I could go on, of course, because the polytoner is never satisfied, but nonetheless, you see a collection of these four functions right here in a sort of matrix setting. And I guess you, I see, I think you all see a pattern here, right? You see repeated codes. The code looks very similar, but there are also statements that are different. And if you normally see that kind of codes, you automatically, at least I, start automatically, start acting in a kind of refactoring mode. I want to refactor this because I see duplication of code and I want to get rid of this code. And what is this pattern that you see? The pattern that you see is that you have some function declaration definition. It accepts some sort of parameter. Sometimes you need to initialize the container. In our example, it was always a list that you sometimes need to initialize. You initialize those two famous, now famous Fibonacci registers, as I coined them. And you have some sort of loop, a for loop or a while loop. And in this loop, sometimes you need to fill the container with the current Fibonacci number. Sometimes you simply calculate and then next you, sometimes you don't do that and you only need to calculate the next set of Fibonacci registers. En at the end, you either return the current Fibonacci number or you return the built up container. That's how it, that's the sort of the pattern that you see. Okay, time to start refactoring code because you want to keep it dry. You don't want to repeat yourself. So you start looking for ways how to refactor the code. Extracting the common part into a function or a method. So a simple check, who knows what refactoring is. We see many hands, not all hands, so I can give you a short definition recap of what refactoring is. You restructure your code without changing its external behavior. And you usually do this to improve the design, structure and implementation. But again, you make sure that original functionality of the code remains the same on the outside. And you do that for a reason because if you do that, then you aim for making your code more readable, less complex and better maintainable. So if you attempt to do that in this particular case, it seems very hard. The code that you want to extract is mixed up with control flow constructs and the values are collected in a list and stuff like that. So it seems hardly doable. Of course you could try to extract that one single statement that calculates a new set of registers, for example, or the initialization of the two registers. But that's a bit of useless. You replace one line by another one liner. So that's not so useful. As a kind of apparently final resort, you could try to combine the functions in one golden super function. So you simply make one functions with multiple modes. So let's start with trying the first two requested features. En we will use in this case a mode flag to designate which of the two features is actually requested. But the warning up front, what you're going to see is very ugly code. So put on your peril sensitive sunglasses for a moment. Here it is. It starts already with the ugly type annotation, right? And as the author of Black, Lucas Langa told us in his excellent Python US keynote, ugly type annotations are an indication of ugly code. And I think this example can only confirm that statement. I don't want to even explain what is happening here in this code. It's quite unmaintainable, not so readable. And you should also not take this contrived example too seriously. It merely demonstrates that the resulting code is not maintainable and this is not the way to go. We have to solve it in another way. So back to the drawing board again. And the solution lies in the fact that you have to take a step back and look at your code in a more conceptual way from a larger distance, from a higher flying altitude, so to say. And then you realize that there is a data producing part in your code, the code that is calculating and provided the Fibonacci numbers. And there is a data consuming part, the code that either collect these values in a container or is checking some condition that identifies the moment to return the current Fibonacci number. And the data producing part is the part that you can isolate. So, for that we need to make a short context switch, enter generator functions because that provides our way out. You just make a function. If you want to make a generator function, you just code it as a normal function, but instead of that it contains a return statement, it returns yield expressions that provides the return values, the values that you want to give back to the caller one by one instead of all at the same time. So, as a demonstration, I can show you a very basic generator function. You see it here, it's called gen2 and actually it mimics the range function when called with an argument of two. The range function with argument of two gives back first zero and then one. And this is just an alternative implementation of that, but also larded with some print statement so that you can see what's happening when. And so you yield the value zero and just before that you see a print that's indicating that that is happening. Likewise for the value one, that is yielding and finally you see a print statement when it's about to return. How do you use this function? You could use it through the iterator protocol. You create an iterator of this function generator like this. I assign it to the variable iter. And then if you want to get the first value, then you just call next on the iterator and you get the value zero back and you see the print. So it only starts doing something when it needs to provide something. And it immediately stops as soon as it has provided that value. And if you want to get the next value, you just call next again and then you get the value one and you see the print out again. And if you want to get the next value, uh-oh, there is no next value, then you get the stop iteration which is perfectly fine. It's fully according to the iterator protocol. It's signaling that your iterator is exhausted. There's no more value to give back. But you can also use it in a for loop, right? Like you're used to. And it looks like this. So you just get the values back and it stops neatly after it has provided the two values. How you're going to code the Fibonacci number using the generator? Well, that's what you see here. You see the registers again, A and B initialized. There is an endless loop. That's quite a difference from what you've seen before. And in the loop, you simply yield the current Fibonacci number, the A register. And in the loop, when it continues, you calculate the next set of Fibonacci registers. And this is all there is to code the Fibonacci number generator. And you could see this as perhaps the most canonical implementation of a Fibonacci function when you code it in an imperative programming style. And how can you use this generator? For example, if you want to get the first Fibonacci numbers using this generator, then you could zip the result of your generator together with the result of the range function. And you would throw away the value that you get back from the range function because you're not interested in that. But you use that because the zip function stops as soon as one of the generator stops. So once the range function stops, then also the Fibonacci generator stops. And that's the way you can print the first eight Fibonacci numbers. So we're set, problem solved. My talk could end here, but there's a little bit more that I can show you because we have these packages. Ittotools module from the standard library, the more ittotools package from PyPI. And you can read out the text later. I continue quickly to show you how you can implement that first requested function using the Fibonacci generator combined with these neat tools. So for that, I first need to point you to the take wow function, utility of the ittotools module. And the take wow function takes a predicate function. Predicate function gets one argument. That's the current value that it gets back from the generator. And it takes an iterable and that's the generator, the Fibonacci generator. And then the resulting function has become a one-liner because you build up a list and the list takes the values, the output of the ittotools take while, the argument of the predicate function is just a comparison of whether the current Fibonacci number is less than the provided threshold and the second argument is the Fibonacci generator. So that's all there is to create this first function. How would the second function look like? Well, therefore, I'm going to introduce you to the I-slice functionality. Very shortly, you can compare the I-slice functionality of the ittotools module to slicing a list in general, where you have this square bracket with columns and you can provide a start and a stop value and a step value. It's in two forms. The first argument is always the iterable and then either you provide only a stop value or you provide a start and a stop value and perhaps a step value. And the difference with standard slicing a list in party list is that you cannot provide negative numbers. That should be positive. And there is another function from the more ittotools package, which is called one, and one simply makes sure that your iterator or your iterable contains only one element. Not less, not more. If it's less or more, then it raises an exception. But if it's one, it extracts that value and returns that value from the iterable. And then you have a one-liner again because you simply return what the one function returns and that extracts the one number from the I-slice function or generator, which is called with your Fibonacci generator, indexed at index n and stopping just before n plus one. So you extract one value and that's the implementation of the n-Fibonacci number. The first n-Fibs, you need I-slice again. Yeah, that's here. So you just call the Fibonacci generator with I-slice that says, well, give me the first that number of Fibonacci numbers. And finally, the smallest Fibonacci number greater than or equal to n. We have a drop while function. It's comparable to the take while function, but it does exactly x the other way around. So it takes an iterable and it drops values as long as the condition holds. And it only starts returning values after the condition does not long hold. And the first function, you can guess it already, I think, gives you back the first value of a returned generator en if there isn't any, then it can return a default value. And combined to each other, it's again a one-liner, if the line was long enough and you see the result here. So you take the first value of a generator that drops all the values as long as the Fibonacci number is less than n. And if there is no value at all, then it returns none. So that's it, sort of almost. I encourage you to read through the documentation of the iter tools module and the more iter tools package. It might probably save you some work if you have a good idea about all the functionality that it has to offer. And it often makes your code more expressive as well, better maintainable, I guess. And there's a lot more to discover from these utilities. A final example is the chunked or eye-chunked function that partitions your generated data into chunks. This can be convenient for loading massive amounts of data into a database, for example, where you can partition that amounts to very short, to shorter, still large, but shorter chunks to keep it all maintainable and to keep it also doable for all your packages en modules that you're using for that. So, you've heard about the following. You are now able to recognize the pattern of certain loop constructs as candidate for refactoring. And I've introduced you to the generator functions to extract the data-producing part. And finally, the message is that the iter tools module and the more iter tools package might save you a lot of coding. So that's it, there are some resources and links that I've used to come up with the definitions. And I would like to thank you. Oh, by the way, all this code has been, all the, not the interactive sessions, but all the functions that you've seen, the old variations and the new variations, have been tested using PyTest BDD. It was a fun exercise by itself. You can find out how in my GitHub repository. So, thank you for the presentation. There is still some time for questions, I guess. Yes, thank you very much. We have some time for questions. So, if you have a question, please come to the microphone. Ooh, silence. Yeah, question. Yeah, a little less of a question and more like a direction for like additional reading if people are interested in this. Because first of all, thank you for presenting this. It's just like a little thing that I keep doing where I basically don't write loops anymore because of generators and things like that. And it gets rid of what I think is actually the biggest problem with Python that loops don't have their own scope. So if you don't write loops, you don't have that problem. And nice and simple. So, two hints for further reading. One is in the standard library, there's loads of other things to go with this which you obviously don't have time to go into now. Like the filter function, map function and plenty more, which you can combine with these generators. Yep. And another package that I would like to recommend is the tools package with a Z. It's like a super set of more eta tools which also goes into like lists of dictionaries and working through those super useful stuff if you're interested. Oké, thank you for reins. En of course there's a lot to debate about whether you should filter a map on and not list comprehensions, but that's a separate discussion. Yes. Thank you. I have a question as well. You were shown us lots of beautiful one-liners. Do you like writing along one-liners and do you think that's good for maintainability? I don't know. Well, you have to be careful actually. I think it's a good question. It looks like I love it. I was surprised by the end result that finally all these functions were one-liners. But that was not the ultimate goal. My goal was to make maintainable code. And indeed, I think I understand the question behind the question, you have to also be careful with one-liners because you compress a lot of information into one line. So I guess that the first time you see those one-liners you have to think a little bit, oké, what's happening here? So sometimes it's perhaps better to create an intermediate variable that contains the generator or an intermediate generator, right? But so the answer is perhaps silently I like writing one-liners but it was not the goal of my presentation. So if there are no more questions, I'd like to ask you to give one more final applause to Jan Heijn.