 Let's go Hi there, my name is Sebastian bit of ski and I work as a Python consultant freelancer and trainer So as part of my job, I joined different teams and work on different projects As I work on those different projects. I noticed that they often suffer from one of two problems They are either over engineered or over simplified. Well over engineered or stuck in the 90s The over engineered ones involve jumping on every shiny new toy that you find on the first page of Hacker news and Then everyone gets excited because well, it's fun to work with something brand new What is not fun is when this tool gets deprecated Maybe its creator get bored and they stopped working on it And now you have to either fork it and maintain it yourself or you have to replace it On the other hand, we have projects that are still stuck in the 90s Some people think that because it was fine to use bar scripts for everything back then it's fine to keep using bar scripts today It's fun if you're a hardcore Linux user and you love writing bar scripts But to be honest, when was the last time you wrote one? Okay, writing them from scratch is not that bad Maintaining a bar script that someone else wrote for you is much harder and I don't mean that bar scripts are bad They're actually pretty great for simple things. It's just you should use tools that are more suitable for the task at hand Neither of this situation is good You should update your tools, but at the same time you should choose Those tools that are proven and you know that they will be around for a few years So that's why I've decided to make this presentation I want to show you some useful Python tools that will make your life easier tools that I recommend to my colleagues and to my clients We'll talk about things that a lot of beginner programmers struggle with at some point First I will show you how to install different Python versions on your computer using a tool called pyenv Next I will talk about Python dependencies I will explain you what are virtual environments, how they work and why you need them if you work with multiple Python projects I will also compare the build-in VNV module with some external tools like the virtual and wrapper and Finally, we will talk about tool called pip x This is a great tool if you want to install some Python package globally But you still want to isolate it from other packages on your computer and before we move on I want to highlight one thing those are all just tools and There are a lot of other tools that do similar things if you're using something else. That's great My goal here is not to convince you that those are the best tools No, I just want to show you how to solve some basic problems when you configure your development environment If for example, you are using conda and you are happy with it Then first two bullet points on this list are actually already solved for you and that's perfectly fine But if you don't know what to use, let me show you some recommendations First let's talk about installing Python on your computer Depending on your operating system your computer might already come with some version of Python installed If you're using Mac OS, it comes with Python 2.7 If you're using Linux, then the Python version depends on which distribution you are using and in Windows 10 You don't have any Python Let's say that your operating system comes with some Python version This Python version already installed on your computer is often called system Python and And no matter which system Python you have I strongly suggest that you don't use it in many cases as we saw for example with Mac OS It's terribly outdated Python 2.7 is no longer officially supported by the core developers and hopefully you're not using it since a long time You might be tempted to update system Python to Python 3 But you probably have some problem programs on your computer that depends on this Python 2.7 You'll be surprised how many programs or even parts of the operating system Still requires Python 2.7 at least on Mac OS If you update system Python, those programs will stop working So if you change the Python version that comes pre-installed on your operating system Your risk that your computer will stop working And that is not fun I have done this in the past when I was young and I didn't know much about programming or operating systems And instead of a fast update I had to reinstall everything from scratch So my advice here is to leave the system Python alone and pretend that it doesn't exist So no matter what operating system you have you will have to install Python and There are many different ways how you can do this You can go to the python.org website and download the installer for your operating system You can use a package manager like homebrew or Mac OS or apt-get on linux And you can even compile Python from the source files However, my favorite way of installing Python that I have been using for a few years already is to use a tool called PyEnv PyEnv is a tool for managing Python versions You can use it to easily install new Python version and to quickly switch between the versions that you already have installed It might not be a big deal if you only use one Python version all the time But if you're working with multiple Python projects that require multiple Python versions, this tool is extremely useful For example, when I do some Python tutorials or courses, I usually try to use the latest version of Python But when I work with my clients, they usually don't have the latest version of Python, so I have to switch between them I've been using PyEnv for years and it works very well If you work with different programming languages, then you might recognize similar tools that work in the exact same way as PyEnv For example, there is RBN for Ruby, NodeN for NodeJS or GoN for Go While PyEnv will work on macOS and Linux, if you're using Windows, check out PyEnvWin It's a port of PyEnv to Windows It might not have all the features that the standard PyEnv has, but it has all the essential ones that I will talk about But then again, if you're on Windows and you are a happy Konda user, there is really no point in switching to PyEnv One of the things that Konda can do is to create a virtual environment with different Python version Which already covers two out of three tools in the stock So you can install PyEnv with homebrew, you can clone it with git and then manually set up a few things or you can use the PyEnv installer tool and While this last option requires you to download a bar script from the internet and run it in your terminal Which is a big security. No, no, no, it's actually very convenient It will install not only PyEnv, but also some useful plugins like the PyEnvDoctor That you can use for example to check that your installation is set up correctly Once you have PyEnv installed, you can use it to install new Python versions To see a list of which versions you can install, you can run PyEnv install dash dash list And you will see a huge list of available Python installations. At the top we have the standard C Python versions But then we have something else like Anaconda, ActivePython, Miniconda, PyPy and so on So all those different types of Python can be also installed with PyEnv If you ever wanted to try PyPy, now you can very easily do this Once you select a version from this list that you want to install You can type PyEnv install and the name or the version number And then you can go make yourself coffee. I mean it. The installation usually takes a few minutes There are some additional libraries like OpenSSL that you can pre-install to make this process faster If you don't have them, PyEnv will try to download them each time you install new Python version So I suggest you take a look at the GitHub documentation Once this is done, you can run PyEnv versions to see if this new version was installed correctly And we can see this on the list. So it means that we are all set Now we have to tell PyEnv to use that version and for that we have to choose one of three different levels Where we want to change the Python version. So we can say global, local or shell What does it mean? Well, nine out of 10 times you will probably want to change the global Python version And to do that, you can just run PyEnv global 3.9.0 and from now on you are using Python 3.9 Then we have PyEnv local. So imagine that most of the time you are working with Python 3.9 But you have this one project on your computer that still requires Python 3.7 And for some reason it won't work with any other Python version So instead of changing the global Python version to 3.7 each time you work on this project And then changing it back to 3.9 when you stop working on this project You can set up a local Python version So that's where the PyEnv local command comes in You can run PyEnv local 3.7 or 3.9 And this will set a version for the current folder and all its subfolders So whenever you go inside this folder, Python version will change to the local version But when you go outside you will be using the global one This is very handy when you work with multiple projects that each requires a different Python version So instead of changing the global Python version back and forth You just have to run PyEnv local in each of those folders And now when you go inside any of the project A or project B folder PyEnv will automatically change the Python version for you And finally we have PyEnv shell that changes the Python version of your current shell session You might want to use this in a situation when you temporarily want to change which Python version you are using For example, maybe you want to run some code that still uses Python 2 So all you have to do is to run PyEnv shell 2.7.18 And now you are using Python 2 But once you open a new session in your terminal, you are back to using the previous Python version So that's how you can easily manage Python versions on your computer. Let's talk about the dependencies in your projects Let's start with peep Peep is a package manager for Python Whenever you want to install a new package, all you have to do is to run peep install and a package name However, peep has one big problem Whenever you ask it to install a specific version of Python package It will uninstall the previous version from your computer and install the one that you ask for Let me show you an example of what I mean Imagine you are a Django web developer and you want to build a Django website Well, you can use whatever different framework. Let's stick with Django for now So you start by installing the latest version of Django by running peep install Django and that installs Django 3 Everything works great. You build an awesome website and the website is so awesome that A client or your boss or maybe your colleague comes to you and says Hey, you're building awesome websites and I have a Django website. Can you help me fix it? well As it often happens with clients their Django website is not using the latest Django It's still running. Let's say Django 2.2. So you install that specific version by running peep install Django equals equals 2.2 peep does what you asked for and after a few seconds you start working on your client's website using Django 2.2 So far so good But later that day you discover a bug on your personal website the one that you created with Django 3 You quickly fix the code. But when you try to test if it's working correctly You get an error message saying Django 3 is not installed Like what I mean we installed it yesterday. So what happened? well When we told peep to install Django 2.2 peep first check if we already have Django installed and we did but it wasn't version 2.2 So peep uninstalled this version from our computer and installed the correct one We just run into a problem with dependencies management We have this problem because peep installs Python packages inside the site packages folder and puts each package in a separate folder named after that package So Django 3 is placed inside side packages slash Django But when you want to install Django 2 it will also be placed inside side packages slash Django So peep has to first remove what's inside the Django folder and then install a different version of Django If you only work with one python project on your computer Then you're probably not affected by this problem But sooner or later you will need to install different versions of the same package And you are going to run into issues with peep uninstalling the previous dependencies The problem with peep is that it installs all the packages in the same folder So how about we tell peep to temporarily install package to a different folder And then we tell our python interpreter to use that different folder instead Well, that's exactly what the virtual environment does Virtual environment is a folder that contains a python installation And any additional package that you install when you activate the virtual environment two things happen First we tell peep to install any new package into that folder And then we tell python interpreter to use packages from that folder So now if we try to install some python package peep won't uninstall the previous version from the global site packages Folder because it's not using the global site packages It's using the local site package specific to this virtual environment So how does it look in practice? Well, first we need to create a virtual environment Python has a built-in module called vnf to manage virtual environments. So we don't even have to install anything We create a new virtual environment with python minus m vnf and then the name of the virtual environment So in this case i'm creating a folder called dot vnf inside the current directory Dot vnf or vnf are both a common convention for naming your virtual environment folder This makes it obvious that whatever's inside this folder. It's related to virtual environments But it also has another benefit some code editors like pie charm or vs code will recognize this folder as a virtual environment And they will automatically start using it Okay, we have created a virtual environment now we have to activate it Inside our vnf folder. We have a bin directory and there we have the activate script Since this is a bar script. We have to run it with source If you're using a different shell then there are also different files that you can use for example There is activate dot bot for windows users Once you activate it you should she use Sorry, you should see a difference in your prompt So now you will see that in parenthesis you have the vnf, which is the name of the current virtual environment So if you see something like that, that's great because it means that the virtual environment is active Now we do everything that you would do as you're building a python project If you install a python package with pip, it will be installed inside this virtual environment And if you run the python script, it will have access to python packages from this virtual environment And if you want to stop using a virtual environment, you just need to run the activate command When you call deactivate it will revert all the changes that activate command did So you will go back to using the global python version and global pip packages So a typical workflow using virtual environments involves creating a one virtual environment for each of your projects So if you're working with two different Django projects, you first create one virtual environment You activate it you install Django 3 and you start coding And then you create a different virtual environment. You activate it you install Django 2.2 and you start using it Vnf module is perfectly fine for managing virtual environments But I want to show you another tool that I have been using for a long time It's called virtual end wrapper and it comes with a lot of cool features that makes working with virtual environments much easier You can install it with pip and it works on macOS and linux for windows users You can use virtual end wrapper win that works pretty much the same But then again, if you're a happy conda user, you can stick to managing that for virtual environments Virtual end wrapper stores all the virtual environments inside the dot vnf folder in your home directory And it provides you with some convenient method for managing virtual environments So you can create a new virtual environment using mk virtual end command So if you run mk virtual end Django 2 up, it will create a virtual environment But it will create it in your home directory in this vnf folder not in the current folder If you want to activate a virtual environment, you just have to call work on and the name of the environment So work on Django 2 up You don't have to type the whole path to the activation script Virtual end will figure out which activation script to use based on the name of the virtual environment And if you forgot the name of the virtual environment, you can just run ls virtual end And this will give you a list of all the existing environment With vnf, it's impossible to get the list of all your virtual environments because vnf doesn't store this information With virtual end wrapper, since all your virtual environments live in the same folder, you can very easily get that list And then finally to remove a virtual environment, you just call rm virtual nf and the name of the environment So now you're probably wondering which one is better vnf or virtual end wrapper Well, both tools are really great and it really boils down to how you like to use your virtual environments I use virtual end mostly because I Don't like to type this whole source vnf slash bin slash activate Where of course half of the time I'm in like subfolder of my project and I get an error saying that folder vnf doesn't exist So I had to figure out what's the actual path. No, I just want to type work on Django 3 up and be done with it And also I sometimes create a virtual environments that are not connected to any specific project For example, I have one virtual environment with a bunch of data science library That I use every time that I need to run jupiter with pandas or numpy And also virtual nf has some cool features. For example, you can create a temporary virtual environment That will be deleted when you deactivate it. So this is quite handy when you want to quickly test some python package But the nf module also has some benefits If you store virtual environment together with your project, then your code editor will automatically pick it up And when you delete this project, you automatically also delete the virtual environment Plus vnf is a built-in module. So you don't have to install anything All right, we covered how to manage python packages in our projects. What about global packages on your computer? Some tools are much more convenient to use when you install them globally Instead of installing them in each of your projects, for example the code formatter black the codelinter flake 8 Or even the virtual n wrapper that we just saw We want to use those packages across all our projects or even when we are outside of any specific project For example, we would use the virtual n wrapper to actually create a virtual environment before we start working on a project But again, if two tools require a different version of the same dependency, we have a problem You install one tool and everything works fine. You install the second tool It changes some versions of this first tool and then the first tool stops working We could install each of those tools into a separate virtual environment But that's a lot of hassle to use it like that each time we would have to activate it Run maybe one command and then remember to deactivate it I mean, you can do this, but it's a lot of typing and in general it's a waste of time We can easily solve this problem using a tool called pip x pip x installs python packages inside separate virtual environments But at the same time those packages act as if they are installed globally So you don't have to manually activate any environments to use them How does it look in practice? Let's say I want to install black to format some python code I run pip x install black and after a few seconds I get a success message saying that black has been installed on my computer And now I can use those three commands black black primer and black d Now I use black as if it was installed with pip It doesn't matter if I'm inside a virtual environment or not I will always use this global package. I don't have to activate anything I just run the same commands as I would run if I didn't use pip x And if I want to install a different version of black inside my virtual environment, I can always do this This black from inside of a virtual environment will take precedence over the global black And it will be used as long as I'm inside the virtual environment So what else can we do with pip x? We can list all install packages with pip x list. So we'll see which commands we can run We can uninstall a package and this will also clean up its virtual environment We can upgrade all the packages with one command called pip x upgrade all And there is also one very useful command called pip x inject It will install a pip package inside the virtual environment of another package You might use it for example if you want to install a plugin for pytest So let's say we want to add a pytest cof plugin to display the test coverage of our code We can't just run pip x install pytest cof because It will install pytest cof inside the separate virtual environment and pytest won't see this package Again, we have we we have this kind of a problem because pip x installs everything into separate environment So we can call pip x inject and this will install pytest cof plugin inside the same virtual environment as pytest is using And that brings me to the end of my presentation. So let's quickly summarize what we saw First use pyenv to install new python versions Once you set it up, it's really easy to install new python versions and switch between them Then use virtual environments when you work on different projects And you want to isolate their dependencies Always make sure to create a separate virtual environment for each of your projects python comes with a built-in vnv module But if you prefer a tool with some additional functionality the virtual n wrapper is a really great choice And finally if you want to install some tools globally you can use pip outside of a virtual environment But you risk that if there is some version conflict, they won't work together So a much better tool to use is pip x That will install each package in a separate virtual environment But for you there won't be any difference in how you use them And even though this is the end of my presentation. I realized that I only scratch the surface of how to efficiently build python projects So if you enjoy this talk and you want to learn more I have made an online course called modern python projects where I talk about this and other tools that you can use to build some really nice python development setup Plus it talks about things like writing tests documentation Setting up continuous integration It has some examples of how to build pipi package cli tool how to deploy a python application And there is even chapter on how to set up vs code to work with python So if you want to learn more you can check it out at modern python projects.com And that's really all I have for you today. Thank you very much for listening and I think we have some time for questions Yep Awesome. Let me get to the first one Is it possible to install python using pyinv without root access? That's a good question And I think yes I have never used computer without the root access So i'm not sure how they work But basically how pyinv works is that It will Change your path environment variable and it will put itself at the beginning of the path So it will have a folder in your home directory where you will install python versions And because of modifying the path it will use those versions. So it's Completely independent of the system python So if you mess up something you can always just remove this folder and you will go back to the global python Can we create a virtually nv using some other already creator virtually nv? I know that konda has this feature I don't think that vn has it. I'm not sure about virtual and wrapper So I know that I know for sure konda has this feature. I don't know about the others There are a couple of more. Let me pick one or two How does v env or virtual env wrapper know which python version to pick when creating an virtual env They will pick up the one that is currently in use So, I mean they will pick the version of python that is being used when you type python in your terminal But I think there are some options to point it to a different version Yeah, I'm pretty sure konda for sure has it vn also has it you can provide a different python binary Probably okay. We are out of time. So thank you so much Sebastian for the great talk Thank you so much for having me