 Hello, I'm Marco, I'm an independent software consultant and I also have a few packages on PyPI and this talk is an introduction to the kind of workflow that I use, it works well for me and I think it can work well for most people and it's easy to change if needed. Before I go on, I just want to list a few additional resources, the really important one is the last one, which is the link to this talk, so all the links are included so if you have a link to that one, that's all you need. So the content of this talk is how to build a Python package and we will look at pure Python one because of the time limits and also my experience is mostly pure Python one. And in my opinion, if you're going to publish a Python code, which I think everybody should, I think everyone has something to contribute, it makes sense to do it well, which means that it's properly tested and documented and everything is top-notch. So when we start working on a package, some code, we first need to think about the layout and the first thing we need, which every Python package needs, is a configuration script setup.py, which goes into the root directory. Next thing that we need is to put our code somewhere, the code that we want to distribute. So my imaginary package is called PyStart and the directory I would use is PyStart, which is the same name that would be used in import statements, so it's important that you pick name well in that regard. You could probably do it differently but quite complicated. Another thing that I like to do is to separate tests and documentation in separate directories because that way they're easy to find and they're also easy to include or exclude from a package as suited or wished. So once we have basic layout, we need to look at the setup.py again. So the configuration script, which is composed from two parts. The first part is metadata about the script itself and the other part is packaging information which tells Python how to handle the script, what to build, dependency and things like that. So the metadata section highlighted here is mostly about what this package is, what it does, who's it from, under what license and in general those elements are fairly self-explanatory, so not going to details, but I would like to point out three things. So the first one is the long description, which in this case is included, but my advice would be to put it in external file because then you can reuse the same description on PyPI, on GitHub if your code is stored there, and GitHub itself doesn't care about much, which format you use, it supports nine different formats, but PyPI does, so write the description in restructured text. Next thing to think about are classifiers, which are here at about a half of the slide, and those are meant to be easy machine processable, but they're also used by developers to quickly decide if this package is of interest and it's important to keep it updated and to think about what kind of things to put there. And the third thing is license, so it's very important to pick one, because since burn convention every code is automatically copyrighted and without license this means that it's effectively locked, so it's not of no use to anybody else except you until you give it a license. If you need help with this, you can talk to me later if you want otherwise from somebody without any legal training whatsoever. Anyway, this is a metadata section and the next thing is to include the code itself. So we do this with the packages attribute parameter, which lists the relative path to the code of our package, in our case that's PyStart, and in principle that could be everything you need, but a lot of packages and a lot of interesting packages also have some kind of external dependencies, and those are handled with installed required section, which is a list of all the external dependencies with their names, the same names as they're used on PyPI, and you can also specify version, which really you should, so you can limit it to the matching ones that will not break compatibility and also ideally is liberal enough that you can also pick up automatically bug fixes or security fixes for dependencies. So at this point we have a package that's buildable, that can be installed locally, but for which we don't actually know yet if it works well, and for that we need to add tests. So there are many talks here at the conference and previous conferences about testing, and I will avoid going into this, this is a good idea, I think we probably all agree that it is, and Python comes with many testing frameworks, you are free to choose whichever you want, unit test is part of the distribution itself, but me personally I prefer to have PyTest because it's a really powerful and flexible framework, and once you install it, it's also easy to set up, it will find tests just about anywhere as long as the files that contain them start with test underscore and tests themselves are functions that start with test underscore as well. So once you install it and run it, you will get a display something similar like this, in this case there are just two tests for this package, and yeah, hopefully eventually we will have a full test suite covering all the code, but with tests like this we only cover the interpreter that's run by default, and ideally we would also test with other Python versions, we would want to support more of them hopefully, and for something like this we might want to use TOX, so TOX is a tool that lets developers test, build different multiple testing environments with different set of dependencies or dependency versions, and run testing all of them and collect results and display them, it does actually much more than this, but for what we need this is enough, because we can think of Python interpreter version as just another package, so it can be easily covered with TOX, sorry. So once we install TOX, we need to also configure it, and we do this with TOX.ini configuration file, here's a sample one, and it contains two sections, so the first one is TOX, where with parameter endless, we specify which environments we want to use. TOX itself comes with predefined environments for all Python, interesting Python versions, so in this case we use Python 2.7, 3.5, and PyPy, we can also define our own, but most of the time we probably don't need to do that, so with this we set up that we will test with 2.7 and 3.5 and PyPy, and then we also need to define how this test should be set up and run, so we do this first by listing dependencies, which in our case is just to install PyTest, and then we also need to specify how to run those tests, which in our case is just to run PyTest, and it will do everything else automatically. So once we've done this and run with the TOX command, we will get this very similar to this one, where we can see that it ran in all these three environments, ran tests there, and at the end reported success in all of them. So now we have something that works pretty well locally. We can run tests for multiple Python versions, but wouldn't it be nice if at the same time we could run tests automatically every time we make a change that we would like to keep? And for me this is kind of similar or more or less the same thing as making commits that are pushed to GitHub. It's those ones that I intend to keep, and we can achieve this by using an external service called Travis CI, which is a continuous integration service that is also has a hosted Travis CI or website that allows open source public projects to use it for free. And to use it, we first need to create an account, which we do by logging it with our GitHub account. We then need to authorize application, so it has access to our repositories, so it can read them. And once we've done that, we will get a list of our repositories. We need to switch on those that we find that we want to use with Travis, and after that we only need to configure it with Travis configuration file, which is a YAML configuration file in which we first specify for which language we want to use, in our case Python. And then we need to specify also the interpreter versions that we will be using with Travis. Now, this section is slightly more complicated because we can test, and ideally we should test with Travis with different Python versions, but we are also using docs for the same thing locally. So the way to handle this is to instruct Travis which Python interpreter it should use with the Python keyword, so for example 2.7, and also instruct docs which testing environment it should use at the same time, which we do through docs environment variable where we use the name of the environment. So after we've done this, we need to install whatever is needed to test our package. In our simple case, this is just installing docs which we install PyTest and do the rest, and then we need to also run the command that we need to run tests, which is just executing talks. So after we've done this, we will get a page similar to this one. This is for one of my projects where we see the status of the test that we run every time we push another change to GitHub. And we also get this batch on the top of the screen. If we click on this, we can get embedded snippets that we can use to include into our read me file or online pages so we can also tell other developers how, in what state our package right now is. So at this point, we have something that works pretty well. It's hopefully tested, it gets automatically tested every time we push changes, but it doesn't have a documentation yet. And without documentation, it's probably not really inviting to others to use because how would we, I mean, without some kind of knowledge on how to use software, it's probably less appealing than other potential packages that might have more documentation. So writing documentation is important also for other reasons because with documentation, we can also think about how the code should be used, how should it work, and we also tell what it actually does. And for writing documentation, I use those things, which is the same one that Python Project uses, and I publish it on readthedocs.org, which is another free service for open source work that you can use. Again, we need to create an account, and we need to authorize the access, which we do once we log in by clicking on this Connect GitHub, and again, the same screen for authorization, and afterwards, once we authorize, we come back to the screen, and we need to import the project from a list of projects. Usually it seems for some reason when you come to this page for the very first time, it's empty, so you need to re-sync with that sync button up there, and afterwards, you just click on the plus sign of the project that you would like to add documentation for. So once you've done this, you can add information about the project itself, usually the defaults are fine as they are. They're picked up from GitHub, name, and repository. You can change the settings if you don't like them, but if you do, please use the format of the documentation to Sphinx HTML, because this is the one that we actually use. And once you've done all of this, you get to a page like this that's sort of a status page for your project, which tells you in what status the documentation generation is. Again, you get a batch that you can use, and you also get short URLs that you can use to point to your documentation once it's built. So this is the online part, and now we have to also set up the site on the locally. So first, we need to install Sphinx with Sphinx Automate, and then we move to the documentation folder that we created before and around Sphinx QuickStart, which will ask us a few easily-answered questions, or if you don't know the defaults are, again, fine, and this will generate a configuration file in which we might want to check if those two extensions are turned on. The first extension, Autodoc, is used to generate documentation from doc strings, which can be really useful for generating API documentation. And the other one in this Sphinx is for if you want to link to documentation from other packages. So once you have all this, and once you also wrote some documentation in your structured text, you can create a local copy with Make HTML and browse it afterwards. You don't need to check it in, because Read-A-Docs will automatically pull changes from GitHub and build its own version anyway. So now we have package that's probably pretty good. It has tests, it has documentation. It just hasn't been published yet. And in a way, it has been, because it's possible to install it from GitHub directly, but that's not very flexible. It's also not very manageable for users, and it's possibly more difficult to find, so we should publish it on PyPI, which is Python's one-stop shop for Python packages. And to do this, we already have everything we need. So we need to register ourselves. We need to create an account, which we can do either on web, or we can do it while we register a package itself. So to register a package, we would run this command, so Python setup.py with setup.py of our package and with the register command. And it will ask us for credentials to PyPI, or if we don't have them, it's at this point we can also create an account. And everything else about the package will be read from the setup.py itself, so we don't have to do anything there. So once we've done this, we just need to push the package itself, and we do this with another setup.py command, in which case it's creating source distribution and real distribution and upload it to the PyPI. We might also want to do some other changes, so if we create a PyPI configuration file, we can specify our credentials there. It's probably not a good idea to put password there even though you can, but if you use all the versions of Python, that's before Python 2.7.9, it's a good idea to put that repository... Oh, there's a typo, sorry. A repository line inside, because it forces setup.py to use secure connection to PyPI and not expose your credentials. Newer versions of Python handle this automatically, but on other versions, it's a good idea to have that in. So at this point, we've done everything. We have a package that's getting tested, it has documentation, we know how to publish it to PyPI, and since we know all this, it would be kind of silly if we had to do this every time we want to create a new package and publish a new package, so it's better to automate it. And we can do this with a tool called Cookie Cutter that was created by Audrey Royal Greenberg. It's a common line tool for creating... scaffolding for a project from a project template, and Audrey already provided a great template for Python packages. So first we need to install Cookie Cutter, then we need to point it to the location of the template file. In this case, it means it's on GitHub at Audrey's repository Cookie Cutter Py package. It will pull it from there, and then it will ask us a bunch of questions that we answer so that it can fill the missing or unknown parts, and afterwards we will have the whole scaffolding... the whole package ready to go, so we can just add the code and the content that is needed. So, yeah, that's it. We didn't have time to cover... We couldn't go into depth about the tools that were mentioned. I strongly urge you to look at their documentation. They are really powerful, and you can do much more. There are also alternatives for each one of them, so if for some reason you don't like some of them, you can easily replace them with a different tool. But on the whole, this thing works pretty well for me, and I hope it does for you, too. Thank you. Okay, thank you, Marco. Any questions? In the back. Hello. I wanted to ask you if it's possible to do the TOX configuration without using PyTest, like using another test runner, like the default Django test cases or... Sure. So, right, yeah, so I use PyTest, but in principle, the only thing you would need to change for everything to work is in the TOX config, so you use whatever test runner and testing framework you want. But then, if you wanted to keep everything else the same, you would just need to change the TOX configuration where it installs PyTest and then run PyTest. You would need to change it to install your framework if it needs to be installed if it's not a unit test, and then you would need to change the execution part so that instead of running PyTest, it would run whatever test runner you're using, and everything else would pretty much work the same way, I think, without a problem. Okay, thank you. Thank you for the question. Any more questions? Hello, thank you for the talk. So, do you think a proper project should have some kind of automatic code quality control such as, you know, Flake8, PyFlake, such like that? Yes, I do. So, this talk originally would also have a section on coverage, which is not PyFlakes, but it would point into that direction. So, ideally, you would try to automate everything you can. So, the problem with running things manually is that it's easy to forget, and anything that can be automated ideally should be automated. So, there are a bunch of tools out there that are usually free to use for open source that you can use to integrate these things like this into your workflow. And join me once again giving Marko a big thanks. Thank you.