 Hello, everyone. I'm very excited to be here. Such a great conference My name is Mia and I would like you to welcome me or to the standard pipe to the standard library to First, let me tell you something about myself. Oh So I am a software engineer working at at a comma at a comma is a Canadian check company that specializes in creating data products I have over five years of experience in the IT industry I try different stuff like tech support testing analysis and backends development I'm based here in Prague and I am a co-organizer of people which is Prague Python meetup and I'm co-organizer of our local Picon conference all right, so Let me ask you at the beginning how many of you have ever heard of Python standard library raise your hand Okay, everyone. How many of you have ever used it? Okay, is there anyone that hasn't used it? No Okay, so looks like all of you are familiar with this but just to ensure that we are on the same page of me Just quickly mentioned. What is the standard library? So the standard library is a collection of modules and functions. They're included with Python it offers numerous functionalities and These functionalities includes things like interacting with operating system running servers scientific computing debugging data manipulation and many more Now let's talk about why so you might be wondering Why should you know anything about a library and why should you use it instead of for example implementing the features by yourself? So by using the standard library, you're not reinventing the wheel when when it comes to finding solutions the solution that are there they've already been optimized and By using these some Solutions you can help avoid to encounter a lot of bugs that have already been fixed Now just to manage the expectations. Let me just mention. What is this talk about? So this is a brief overview of lesser known features of the standard Library and the aim is to discover the unknown unknown So these are the features that you probably didn't know that they exist So the library is huge So I started thinking what exactly to cover because we don't have time to cover Everything so I started thinking about which modules are well known meaning that most of you have probably used it and Which are less known that are not much used and also started thinking about things that we do as developers like Every day so some things all of us do every day and but some of teams we rarely do so I would I will not talk about Less known modules which do some Less known things which because this is a topic for some small audience audience that does something like very specialized and Also, I don't want to talk about like top five functionalities from the standard library because if you just Google Top five functionalities you would probably find it there Instead I would like to focus on the other two quadrants. So let's start with the well known modules Which do less known things. So let's see what's hidden in the models, which probably all of you have used So just a small disclaimer that all of code examples that I will show they're just illustrative The specific use case when to use the standard library or not, of course depends on your specific Use case. So let's start with fun tools raise your head if you have ever used fun tools Okay, so it looks like most of people so it sees one of the most frequently used modules inside the library and It includes where to wear a variety of tools working with functions So stuff like modifying function behavior or creating function like objects. So let's start with an example So this is an easy function which adds two integers together But this function should handle two cases first cases when both of values are integers and the second case is when there are strings So when they're integers the function should add them together But when there are strings it should concatenate them with an empty space in between So you could write something like this And if we try to run it with different values, we see that it works in first case It added it in the second case. It prints us. Hello world is there in the third case It says unsupported it that type which is as expected However, there might be a better approach on how to solve this problem by using the standard library So here we have a single dispatch decorator above the function at you can see it on the line for and then we Specify the behavior for each type in a separate function So if user enters integers the functions define on the line 10 is run and That one adds two integers together But if user enters strings then the function on the line 15 is run and for all other cases if user adds enters any other data type then the value error which is defined on the line 6 is raised So single dispatch it's used for function overloading Just to mention function overloading means creating several methods which is name time Where they different from each other in the in the type of into input parameters and number of input parameters And it's commonly used when you work with different data types as input to your functions Well, for example, you're parsing input data from some files and you have different data types Now you might be wondering why the approach with using single dispatch might be better than the first example where we just had a lot of if else Statements so the advantage is that it's easier to modify is because each function is Independent so you can just modify one function for example only the one that adds integers And you don't have to modify other ones and also the code is cleaner and more readable However, it dispatches only for the first argument So the downside is that if you have multiple arguments in that case You cannot use single dispatch, but you would have to use some third-party libraries Okay, so let's see some more examples From the fun tools module So let's say we have a function which adds to number together, but very often we add only by two So you might write something like this where you basically define the default parameter But now let's say we suddenly start very often Adding by three and you add only by two or three So in this case you might go like this So you just create another function where you have a default parameter three But basically what you are doing is your duplicates is the code, right? Because they're kind of the same So instead of what you could do you could use partial so in this case We define a function called add which is on the line for and it acts it acts as some sort of a template for other Functions which are in our case the function add to two and add to three So we use partial to pass add into these functions where we define an argument in each case And in this case the core logic is kept in one place which is our function add on the line for and You can have multiple partial functions that are really they are able to reuse that code So if we run it we can see that it returns the same result So partial it's used to create a new function with some sort of the argument of the original function So it's some sort of a template we could say it can be used with any Collable including built-in functions methods from other libraries arcs and quarks So now you might be wondering, but what is the advantage or why you use it instead of just using the default parameters and duplicating the code so the main Advantage is called the reusability. So you're adhering to the dry principle. You're not repeating yourself and A common case is when you need a call when you need to call a function with the same argument multiple times, so One example from work So imagine you have a function where you call various API endpoints and there is an authentication header Which you must pass each time when you make a call So if you set up its value or refer to it at each place where you call the API and The authentication measures suddenly changes what would happen is that you would have to update all of these places, right? But if you use partial you can just centralize this logic in a way that you have it at one place and Where you define the header and then you just reuse it at all places where basically you just call your various API's But it has one downside and that is it's not really Intuitive to new developers. So for example, if your team is very junior or they come from different They have different background they come They use other languages for example, which don't have partial functions Then that case maybe you would want to sacrifice reusability for better readability Okay, so let's see one more example from partials. So we have this Fibonacci function It is recursive and it has some sort of expensive operations So is there any way we can improve it and make it faster and more efficient by using the standard library? So one option is we can use cache. So there is the LRU cache decorator from func tools So basically this decorator Caches the result returned by the function so the function doesn't need to execute the code each time and It can just return the cache results instead when they are available and from version three nine We can also use the cache decorator So least released recently used decorator is stored the results of function calls it checks for the key in the cache dictionary when the key is present the wrapper returns the value and Updates the cache hit info But if the key is missing then the wrapper calls the function it pass arguments And then it updates the cache miss info and returns the result in case the cache is full It evicts the old items and just replaces it with a new once and now you might be wondering What is there any difference between LRU cache and cache so cache is available from version three nine And it is the same as LRU cache max size none. So basically it doesn't have any max size It doesn't evict the old value. So it might be faster So I have benchmark average execution speed with and without cache just to mention that this is not a precise method So if you run it on multiple machines and multiple times, you would probably get different results But this is useful just for some like rough comparisons And you can just like focus on the ratio here, which are like typically consistent. So here we have three examples We have our Fibonacci function without caching with LRU cache and but with cache only and you can notice that the cache Versions are noticed noticeably faster. So the caching does speed up our code and you can also notice that the performance of the LRU cache and Cache only version basically are the same All right, so let's explore some more well-known modules Who here has ever used iter tools raise your hand? Alright, a lot of people by a bit less than fun tools So it or tools provide various functions to create Iterators for efficient looping and they're commonly used when you're handling large data streams So let's start with an example Let's say I need to calculate the Cartesian product of two lists So here we have two lists and two nested four loops where we iterate over both lists Instead of calculating him by ourselves and just like iterating over two lists We can use instead iter tools dot product and as you can see it return returns the same result So product returns the Cartesian product of two iterables Let's see some more examples So let's say I have a list and I want to filter elements that are in another list So I could use the built-in function filter which does exactly that and Now you might want might be wondering but what do we want to do the opposite? Like I want to filter elements which are not in the list so something like not filter So we can call iter tools to the rescue we can use filter false Which is exactly the opposite of the built-in filter So filter false it filters element from an interval returning only those for which the predicate is false and Basically, it's the opposite of the built-in filter functions Let's see some more example with Python built-in function So we have two lists and we would like to combine them together You can also notice that the first contains three elements and the second one contains six of them So what does happen when we run this code? When we run this code we can see that only the first three elements are merged together Why well because it doesn't take the elements for which there is no value in the first list So basically it just discards them but what if we want to zip all of the elements and we want to for those that we don't have values in the first list for Example we want to fill in with some other value like none or something else in that case We could use the zip longest function Basically, how does it work is that it fills the value for the missing elements and we can also set up our own value in this Example I set up here none, but we can use any other value Okay, so so far we have talked about fun tools and iter tools And I would like to talk about one more well-known module Which is also commonly used and has some interesting hidden features. How many of you have ever used collections raise your head Okay, a lot of people's but still less So let's say I have multiple dictionaries and I need to find some key So we could do something like this But what happens if for example, I have ten dictionaries and not only two then we would have a large if Else if else statement and while this approach is functional. It might not be the best approach So let's say we can improve it with a standard library. So we can use chain map Here we add all dictionaries to the chain map class any groups all dictionaries together So basically when you're searching for a key in the chain map It will search for it in all group dictionaries and it will return you the first result it finds Now you might be wondering how about creating a new dictionary? I just using the update method to add all the data in this case We could also group all dictionaries into one and then iterate over it Well chain map works in a way that it references already existing dictionaries and doesn't copy any data while the update method does So the chain map groups multiple dictionaries into one and it provides a single dynamic view So what happens is that when one of the dictionaries gets updated the update is visible in chain map as well Speaking of dictionaries, what happens when we attempt to search for a key that doesn't exist yet The key error happens, right? So and what do we don't we want to prevent it we don't want it to appear because for example We have we don't have the key there yet, but we will have it. Is there anything we could use from the collections module? So it happens that collections have a subclass of the dictionary class that returns a Dictionary like object which is defaulted the difference is however that we can use it with non-existing keys And it doesn't return the key error instead. It supplies the missing values with a value that was passed in this case We instructed to supply the missing values with a non-value So if we run it and try to access value for that key that doesn't exist in the dictionary then You can see that it results returns us none So default it it is a container like dictionary it returns of default value for a non-existing key it is commonly used for grouping or for counting elements in a collection and It can make your code simpler more readable in case that you don't need to check if the key already exists or not and You might be wondering but what if my structure is nested dictionaries. Can I still use default dict? Yes, you can you can have multiple nested dicts with the default dict as well Okay, now let's move to the second part So what are some less known modules which do well known things? Some things which we do on a daily basis for which there are tools in the standard library and knowing About them might be useful for us. So testing I will not ask how many of you write this because I hope that all of you do There are certain frameworks for testing APS or web applications You can find a lot of materials on the internet about testing pie test and many other frameworks But I would like to focus instead on the last Explored topic and that's kind of quick and dirty kind of testing without using any frameworks So let's say you have some small script. It's not It's not an enterprise Solution it's not big API just some small script on your PC running just calculate something and You want to have some kind of test just to make sure when you change is that it still works And ideally also some documentation So is there any way how we could add some tests and documentation in a really easy way? So I discovered the doc test module It works in a way that you can add examples in the doc string and you can notice that these examples are both tests and documentation for your script So if we run it, we see that nothing happens, but this is normal because there are no errors in our examples But if we change something for example the line 10 We have changed it from minus 15 to 15 which is a mistake and when we run it again We can see that our test failed expected 15 got minus 15 So test mode it's used for very simple scenarios It's mostly for quick and dirty kind of testing and documenting very simple scenarios These test cases are readable to humans. It allows your test and to document your code in the same time Comparing things it's also something we very often do and it happens that in the standard library There's a lot of tools that we can use to compare things. So let's see some examples Sequences so here we have two strings and let's say we want to compare if they're same or not and If they're not we want to determine how different they are So to compare sequences, we can use the jflib module So we here we create an instance of the differ class and we compare two strings and We when we pretty print the result we can see that all the characters are the same except the one on the line 8 Which is missing in the second sequence and One on the line 9 which appears to be in the second sequence, but is not in the first one And we can also calculate the ratio of comparison between these two sequences So basically that means how similar or how different these two sequences are So here we can see result which is 91 percent. That means they're very similar So dflip it's used for comparing pair of sequences of any type string Tuples list as long as the sequence elements are hashable and it uses the Radcliffe Algorithm if you're interested in that you can read more about it And we can also measure the similarity of the sequences where their values between zero no match and one Which is identical match where? The ratio over zero point six means that the sequences are close matches So these were sequences and how about other things we very often work with for example files Can we compare contents of two files? So it happens that there is a module for that in the standard library as well here We have two files and we can call the cmp function from this file cmp module And when we print the result it returns true That means basically that the files type size modification time and content are identical. So the files are equal And that was fun, but you might be wondering how about directories. What if I have for example two directories? I Accidentally cloned and I'm not sure if they're the same or not So for directories, there's a method dire cmp which compares to directories when we call the report method We can see the difference. So in this example, we can see that the file File one dot pi is only in the first directory, but the file file two dot pi is in the second one So the file cmp defines functions to compare files The dear cmp class constructs a new directory comparison object to compare directories and there are multiple functions to define what to compare and how to show you the results and It's very useful for cases where we have for example different versions of the same project and The last great feature from the standard library. I would like to talk about is a context manager So let's see an example Let's say we have a file which we need to open we want to do something with it and then at the end Of course, we need to close it So for this we could use the context manager decorator where we write a function Where we define our operation that is supposed to happen at the very beginning before we use it And at the end after we have finished our work and then we can use the width statement Which is on the line 15 to use it So if we run this one, we can see that first we open the file Then we've brought to the file which was an operation inside a width statement So that's is the line 16 and at the end we close the file So context manager defines a factory function for width statement Context it provides a clean easy to read way to manage resources that need some setup and some kind of tear down And you can ask multiple constate managers with blocks to use them at once or in a single statement by separating it with commas For opening and closing files. There is the built-in function open But other common use case for which you might use it are acquiring and releasing a lock Working with network connections temporary files changing and restoring global settings And now you might be wondering but what if my function is acing does this work in the acing context as well The answer is it doesn't but there is another decorated for it in the library as well So if your function is acing then you can use the acing context manager So here is basically the same example, but only acing and here is our main function where we run it So basically if it behaves exactly the same But the only difference is that one is intended to be used in the same context while the second one is the acing All right So we have covered the second part where we talked about less known modules which can help us with everyday things we do and At the end I have two takeaways. I would like you to take out of this talk You will probably not remember all of these examples and all of these details, which is perfectly fine But I would like you to remember the following two things The standard library is packed with very powerful tools and Always verify if there is an existing tool for your task And if it suits your need before attempting to create something from scratch I hope you have enjoyed the talk and that it has inspired you to embark on your own journey of exploring the standard library Thank you very much for your attention. I Will take my privilege as session chair to ask my question If you have more time for this Presentation which other modules or things you you should have you would like to share like Yes, so my first version was My first version was 40 minutes. So I had to cut it I really like the module paths because there is a lot of usable tools Which we can use there and the good thing is that there all of them are operating system agnostic Meaning it doesn't matter if you write something for linux or for windows or for Mac, but you can use it anywhere Okay, thank you Say let's let's give an applause again for me after this