 Thanks everyone. So as Noah said, I will be talking about the challenges for maintaining a popular open source project and I want to start with that this is my own kind of personal experience so that doesn't necessarily apply to all of the maintainers and it's It's there are huge differences between what people do like inside of or like as part of the role or just in this bad time But so this is kind of my story and I think it's worth hearing that at a community conference like Europe But first I think it's been a great event so far and I would like to give a big hand to all the organizers and volunteers for this great event So just a little bit about myself I'm a maintainer of Pytest and cookie cutter both project are fairly popular and widely used in the Python community Also, right speak and tweet about these topics. I will have my social Handles on the following slides so you can follow me if you want to hear about these kinds of topics And I'm currently living in Berlin Germany and work for Mozilla As a senior test engineer This is the URL to my block. I mostly write about Pytest Or some go every once in a while I'm Hacker brought on Twitter and GitHub And if you are interested in Mozilla what we do our products and projects then come find me after the talk And we can we can have a chat So the agenda for today is first I want to kind of touch on what it means to me personally to maintain a popular open source project And I will be using cookie cutter as an example, but we have face similar challenges or problems and Pytest as well Then the challenges of a growing community Then just to highlight some of the frustrations for maintainers And I will be very careful about not starting to rant about it, but I just want to give you some Kind of insights in what happens on a daily basis and kind of how you can I think it's called to die the death of a thousand cuts or something like what happens on the daily basis and how that can sum up to actually Burning out So I just want to kind of draw your attention to that and then in the end I want to give you some practical advice on what helped me personally I think what also have other people from the cookie cutter project and maybe how you can use it for your own Projects and life So the cookie cutter project is a common line utility and The idea of cookie cutter is that you have a common line interface and you feed it a template It has a certain structure And then this template is being used to create directories and files and also source code for you So we use ginger to for templating so those of you familiar with flask for instance will know how it works And then it's also working on all of the major platforms. So windows mac and linux and on newest version of CPython also pipe by This is the URL to the project. It's hosted under Audrey's github handle Audrey R slash cookie cutter We use a permissive psd3 license. So essentially you can do whatever you want to Just don't blame us if you If your servers start fire or something We have a whole lot a big number of contributors actually So it's around 200 or 180 is last time I checked and there are more than a thousand templates on github alone And those are just public templates. So we have no since we don't have any telemetry or anything We don't really know how many people actually use cookie cutter How many projects there are but this is a very easy to to find number by just searching on github for projects that have a certain file and We also had multiple talks at community account functions. So there is like some sort of community around the project already These are some of the most popular templates. There is one for just creating a python package that you can then push to pipe EI there is a huge you probably are Template for Django, which is kind of you use cookie cutter that template and you have a production ready Django app So that's really popular and it's also featured in the two scoops of Django books for Maudrey and Danny both Also maintain the project and then there's also the cookie cutter pie test plug-in Template which I mostly maintain on Bruno and about a bunch of other pie test talks and it's kind of the Preferred way for how you would start creating pie test plugins Yeah, the project in itself is community driven. So it's not like we have a Like a company or anything sponsoring the project. It's really just volunteers You can you can talk to us on we use skitter for chat and everyone is invited to contribute This is how you install cookie cutter using pip from papi This is how you would call it. So we have support for some abbreviations in this case It's github and it's just pie test Dash and a cookie cutter pie test plug-in and then cookie cutter will get clone the template and you will be asked a bunch of Questions or to provide information about so that's kind of these prompts are Set up by the templates author and then based on this information the project will be generated So in this particular case we created a pie test plug-in called pie test emoji So I want to touch a little bit on kind of my story with cookie cutter I think I started contributing maybe four years ago and first I just created a bunch of templates for project that I was interested in and I kept kind of copy pasting the same files the same Set up py code every every time so I created a bunch of templates and then I heard about this new Testing framework which wasn't really new but it was just new to me called pie tests And I saw that cookie cutter had a unit test coverage of 100% and the tests were really simple, I would say and I Took this on as a challenge to convert the entire test suite of unit from cookie cutter from unit test to pie tests Because they were really separated so I could just go ahead and gradually migrate all of the tests That was very very received by the community And yeah, I also helped like with other problems on the issue tracker and and chat and all of these kind of typical things that you do in an open source project I Also developed a number of features and bug fixes and then kind of things get a bit Interesting because we had a number of people complaining that they can install cookie cutter And we had no idea what the problem is because we don't have any Crazy dependencies or so it turned out that under certain conditions on specific platforms on Python versions Payamal was broken then we switched to ruamil yamal, which is a fork of payamal Supposedly better maintained But the the astus had was just broken or something so people couldn't install payamal and ruamil yamal so they ended up not being able to install cookie cutter and Yamal is really only used for a very specific feature for cookie cutter So it was really frustrating kind of to being blocked by a feature which not everyone uses So as we do as software engineers when we encounter a problem, we just create a new problem in this case. It's called Poyo I Read the yamal specification and I had nightmares from that so Poyo really only supports what cookie cutter Does with yamal so it's not compatible with Jason. It cannot write to yamal It's really just getting the user configuration that we use for cookie cutter and and kind of replace Payamal and ruamil yamal what we used before Then I also created a project called ginger to time which is a ginger extension for retrieving the current date time Which is oftentimes used in templates to for instance when you use the MIT license It always contains the author name and then the year So a lot of the templates would do some like please enter the current year And then people got really upset that why do I need to provide this? So I created this extension and you can just use a ginger Flag of whatever it's called just now and then we'll insert the current year into the text And then there's also the project called Pytas cookies We had a number of people kind of reaching out to us. Hey my template doesn't work or I can't use that template So I got frustrated and created a pie test plug-in So people actually write unit tests for the templates and don't constantly create issues on our entry tracker because Template authors didn't test creating projects from their own templates I'm showing this graph not because I want to brag about my commits But because I find it really interesting that for the longest time github really only values commits as contributions and Funnily enough the contributions of all of the maintainers have nothing to do with kind of the commit history Like we've we put in so much time and energy into the maintaining the project And what's really only visible to the community and what's being tracked by github was commits And then so if you look at that you can see kind of how I started that was just in the beginning Migrating all of the tests and then it's just going down and down and down because I just merge Pull requests mostly and that I think that the merge commit doesn't even Contribute to your commit graph. I think on github. Yeah, maybe it does nowadays But it's it's just one commit but the effort of like reviewing code is it takes just so much time and Then part does cookies and ginger time and so like part of my commits were even like a different project So it wasn't easy really visible to to what I did on cookie cutter itself So eventually I asked the team if they want me to become a maintainer so I can use some of the tools that github provides like labels and Milestones and all of these things you can only use if you're a collaborator on the project And I felt like I had the time back then to actually help the project So they promoted me to to be a maintainer and I also started doing the releases and pushing to pi pi Yeah, and then the typical stuff like reviewing pull requests and triaging issues which takes a lot of time And I also spoke about cookie cutter at Europe I've been before a pie data villain and and Glasgow and some other local groups And then what's also really interesting because this was always happening just in my spare time I was starting to explore ways so we can actually get funding for cookie cutter because a lot of big companies use the project so I was kind of Keen to see if there's maybe a way so that I can I don't know be paid to work on cookie cutter Maybe a week a day a week or something Turns out it's really hard if you have an open source project with people from Los Angeles South Africa the UK and Germany to somehow have a Legal entity or something where you can actually receive money or do taxes or and I like dived into all of this Stuff talked to the the awesome people from the Django girls Foundation how they did it and they were facing all sorts of struggles to set it up so this was really exhausting and no one knew about it outside of the core team, but I think it's worth highlighting that if you actually want to kind of Be more serious about an open source project project You will be facing a lot of struggles if it's not supported by a company that pay you pays you to to do it actually Yeah, and emails so I have tried to find out how many emails I got in this time I think it was at peak times. It was maybe 40 emails a day and those emails might be just Someone updating their pull requests or something really minor But it's still like you get the notifications all the time and they don't it's outside of business hours So it's it's not like a job where you can just put away your your work laptop and then forget about it But you just keep constantly getting emails about all sorts of tiny things So that doesn't really help with maintaining a healthy work life work life balance. I think So I want to get to some of the challenges that we faced because the community grow grew so quickly So first, I think it's important to highlight Not only for cookie cutter, but for all projects. You will always attract users at a faster rate than contributors And then also you will only like the rate in which you attract new contributors It's also also way faster than people who actually have the time are privileged enough to do to have the time available to Step up and actually maintain a project So the reality of cookie cutter is that there are thousands of users which we don't really know because again We don't have telemetry or anything but since there are so many templates and we get so many requests that it must be thousands We have as I said 180 contributors and the core team is five people and No one is paid to work on the project or has like resources available to actually do it For a specific time or anything So what are the learnings from this I think so maybe I should also say about how this section of the talk is structured So I want to kind of highlight the challenges and then provide you with some of the learnings So maybe you can Take some of these for your own projects. So after each topic, there will be some learning section So I think what I what's really important is that it's You should try to make your project really easy to contribute to Because there is no way that you can do all the work yourself So you rely on people to step up and provide patches and all these things for you Then it's really important for yourself But also for the wider community to adopt a code of conduct and also enforce it because there will be people with toxic behavior Who are abusive to other community members? But most most likely to you as a maintainer because you always are the contact person for all frustrations So it's good to have a code of conduct in place and then Also have the strength to enforce it Then always have empathy towards others but also towards yourself. So be kind to newcomers be be welcoming because otherwise you will never Kind of help the user become a contributor if you don't really show empathy and and try to make grow them into the contributor stage and Then write good documentation because if you kind of put all of this this knowledge in these learnings into documentation You will spend less time on actually Repeating the same kind of stuff and then lead by example I think that's really important like if you if you are a toxic maintainer There's no way other people will be better than you so be kind have empathy and be a nice person overall Project scope that's an interesting one for cookie cutter because people came up with all sorts of interesting ideas how they use the project So we get those emails Yeah, at work. We do this blah blah blah and it's part of our deployment pipeline and this and that and You will be facing like issues About use cases that you just never intended before and it's important to listen to those people But it's also important to Kind of be specific about the project scope and say well This is just a common line tool that creates files directories and source code But it's not like setting up your Kubernetes cluster or anything It's like just be specific about the project scope and document it and Then in the case of cookie cutter is also really interesting that a lot of the people who encounter problems with templates will raise Issues against the tool Yeah, so that that happens a lot and I think it's probably Could be better in the cookie cutter documentation kind of to explain the concepts that there are templates And there is this tool and this belongs here and this belongs there but also maintainers of templates kind of need to do a better job of Showing how to get in touch when there is a problem with the templates Yeah, so about the project scope Encourage your users to build tools on top of your project. So we offer a Python library So we have a comment line which uses the Python library So if if there is a use case which is outside of the scope of the comment line utility Maybe people will just use the library to create a new tool and that's perfectly fine So I think that's important to encourage people to do Then always ask contributors to develop automated tests write good documentation for new features because you don't want to merge Some code and then have to kind of write the documentation afterwards or write the tests or anything so each change needs to be kind of Like contain all of the things that you require a change to have and then again, right? Documentation describe the project and its scope Breaking changes. So I mentioned kind of the the platforms and the Python versions that we support and there are also people who kind of push cookie cutter to different Package managers so there's you can insert cookie cutter via condor via homebrew. There's a debion package. There's an arch package like people created all of these different distributions But then when something breaks, they will still come to cookie cutter and like I have no clue about debion So I wouldn't be in a position to even like Understand what the problem really is so again, this is kind of Maybe a documentation problem, but That's not something that we kind of Anticipated when we when we started the project So all of the changes that go into the project need to be carefully considered and thoroughly reviewed because if there are so many people using your project you want to be careful about not breaking anything because It's it for one. It's it's the good thing like the right thing to do not breaking people's stuff just because you're sloppy or because you're a Bad person but also because people will raise issues with you if something breaks. So it's kind of Protecting yourself is kind of part of the thing Yeah, so be careful about adding new features test your projects and the on the supported platforms and Python versions Right documentation and also say no like if there's someone with a very very specific use case you should Say no and please build a tool on top of our Python API for instance workflows as with many projects Cookie cutter we have asynchronous communication. So Danny Audrey based in California. That's nine hours time difference to Berlin South Africa is the same time zone as Germany And the case just one hour but still like a lot of the communication will happen as a chronously over GitHub issues or emails Which makes it kind of hard to decide on specific things like if there is a problem and you want to have a discussion It's really hard to do that If it's not happening in real time Yeah So you kind of want to have guidelines For how you want to merge code and show that the code is actually Maintainable idiomatic Python code and tested and documented so that in case someone isn't available or just I don't know Doesn't isn't as responsive as usual. You kind of want to have this document so you can rely on that and then also for for the longest time it was a bit of a problem that github didn't really have many tools to help maintainers like I think a sign is for pull requests or Yeah, I like it was really hard in the beginning to Think you also the code reviews were made in a way that you get an email for each of the comments on the pull requests rather than Submitting one review with all of the comments. So if we had a large patch and we had Someone reviewing that I would get 30 emails like and that was just not really nice So some of the learnings are set up meetings To you over chat or video or something if you really want to have a discussion and then but be sure to document your decisions on an issue Or a pull request or the documentation so that the community understands why you made the decision and you don't get bothered afterwards Why did you do this and then walk towards making yourself redundant? So you want to kind of don't want to be the bottleneck for the project So be sure that the code isn't super super complex and there is just a single Person in the universe to to understand that code He kind of want to be sure that everyone can technically step in and do your job in the project And then for for the github Like I'm pretty sure it applies to more most other Platforms as well is kind of try to reach out to other maintainers So we talked to folks from I don't know the Java strip community and we talked about everyone is having the same problems So kind of if you if you group up together, you can actually read reach out Maybe to the to the product folks and suggest features Which I think helped to some extent and Then what's really important maintainers are humans Some people seem to forget that but for the in the case of cookie cutter Which we're just volunteers some like when we review code that's outside of our working hours for me that was mostly at night and Didn't really help me maintain a healthy sleeping schedule So that's just a very bad idea and people seem to forget that this is the case for for a project like this And then every maintainer had their own reasons why they started Contributing to a project or why they decided to step up and take on the responsibility of managing the project and this changes over time Just to talk about myself like in the beginning. I was working for a software company. We only worked on proprietary code So contributing to open source was a way for me to build up a portfolio so that I can show some code when I'm Apply somewhere else So my motivation was never to kind of receive 40 emails a day But I just wanted to demonstrate that I can actually write good code and then because I was spending so much time on the project And I really liked the team and I found useful myself. I kind of grew into this position of a maintainer And then I felt ownership and responsibility over the project Which is why I just didn't like step away even if I sometimes were stressed out or so But I think it's important to keep that in mind that like this is really just people volunteering So the maintainers for this section sorry the learning for this section are pretty much stick to best practices so that if you know how to write Python code you should be able to contribute to to cookie cutter don't use Super special things or crazy dependencies or anything make it easy to put people to contribute Then document the process of bringing on new maintainers, which is really important So a lot of the projects don't really have this documented what do I need to do to actually become a maintainer and I don't really believe that we have this super while explaining for cookie cutter as well, but it's It's I think it's important so people can kind of know what they have to do if they really want to be more involved in the project And then really important do not tolerate toxic and abusive behavior from community members or users This is really important and it's it's hard if the same people keep kind of sending you a nasty email But so it it's important for yourself and for the project to exclude those people if they really violate the code of conduct So this brings us to expectations I think everyone using a project or contributing to a project or maintaining a project has a certain set of expectations And it's really hard to manage that So you can't really please everyone and I think it's it's just important that you also document kind of What you can kind of expect as a user as a contributor For instance saying yeah, well, we don't offer 24 seven support So if you break your deployment, then that's your problem and don't don't send us emails And as you might have noticed in a lot of the sections I mentioned documentation and for me I think that's the major takeaway That if you have good documentation for users for contributors for maintainers for template offers Chances are that people will be less likely to ask you the same questions over again and like if the documentation is good people will also read it so I'm a big advocate for for writing good documentation. It's really hard, but I think it's it's worth doing So I want to touch very briefly on some of the frustrations And then kind of highlight why this is a problem the first one is bike shedding So I already mentioned that we had problems with Yamal and then when we had an issue this debating over What shall we do to kind of solve this Yamal issue? The first comments was where well, why don't you use Tomo? Then comments, why don't you use Jason? Why don't you use hjson? Why don't you use config parser? Why don't you use blah blah blah and Kind of we as maintainers we just wanted for people to be able to install cookie cutter like we don't like we support Three key value pairs. We don't care what form of that is you can technically just use a text file and parser so having people stepping in with a lot of Opinions is is really hard in those cases to not get frustrated by that and the the Yamal thing is really just one thing Why do you use six for Python 3 compatibility? Why don't you use future? It's much more lightweight. It's just like two kilobytes instead of five or something and like Well, we want to support all of these platforms and we don't want to maintain a compatibility layer ourselves And six has proven to be working So we don't care. It's it's 40 kilobytes or something. It's a common line utility that you install once It's not something that you constantly download and install as part of a process department process and Then people oftentimes said we need this at work and when someone opens an issue and starts with this sentence I turn off immediately mostly because People who use our tool at work are probably the most demanding and they have the least empathy for that We are just volunteers and that's that's really frustrating if people Use our project during the working time and then they demand support from us in our spare time, which was at night for me So just be mindful about the fact that people are volunteers And it's not the case for every open source project But I think a lot of the very important Python projects in our ecosystem are maintained But just one or two people and mostly in their spare time So it's important to keep that in mind that even if you rely on the tool as part of your job Kind of take this into account and kind of think about the wording Yeah, this is a very common one like someone used our tool in an unintended way and somehow we broke their new release or something and I can't really say I'm sorry I would rather say well you should be a better software engineer and don't rely on Like the latest master or something like you should pin your dependencies or maybe I don't know Have a vendor or a tool or something, but like this is your problem. This is not not my problem really Then again, so Poyo is kind of this library that I wrote to parse the Yammer configuration and This is a quote. So someone said and this Poyo introduction is a great example how direction of cookie cutter can seem immature to my eyes from times to times and That's interesting given that Audrey has Multiple degrees from MIT and worked at Microsoft and Danny worked at NASA and they both wrote the two scoops of dango books And I feel like I'm experienced enough to write good code So it's interesting when some random keyboard Cowboy from the internet tells you that you're very unprofessional in the way that you lead the project But it's in this particular case that person kept repeating the same behavior So at some point we actually pointed out that this is violating our code of conduct because they're really abusive towards the core team And we we didn't have to exclude them, but they kind of decided to step away from the project So that was good, but without the code of conduct We wouldn't be able to actually point out that they are actually doing something wrong So my recommendation is have use a code of conduct that you feel us is kind of Matching your project and then enforce it for your own sake and for other community members as well Yeah, can't you simply Don't even need to say anything about that really it's it's Yeah, no we can't Something that someone said so we I created a proof-of-concept poor quest where I was adding Types to cookie cutter variables. So currently it's really just a variable and it's text and a lot of templates they would like to have say a Boolean variable or a float or a string so that when you render your project you actually have typed variables and can do More advanced things on that and I had this need as well So I created a poor quest which is what was kind of ready to merge But it was more or less just a start for a discussion If we actually want to take on this extra work of maintaining that going forward And then we made a decision as a core team that if we do this We need some real sorts of sponsorship because the people requesting this feature where companies building Editor plug-in extensions or whatever based on on cookie cutter So they had kind of the real interest in this feature And while we needed it as well. We kind of felt like it was It was just fair towards the team if the companies would actually help us and in maintaining the project So we we started a discussion on how we can we actually get funding and sponsorship for the project So some people just said well not sure how much sponsorship is needed But here is a site for allowing the crowd to contribute as I know my organization would be interested in helping So then he created a patreon. He gets two dollars a month. I have a patreon I get I think fifty one dollars a month and I get this money from Russell Keith McGee who used to be the president of the Django software foundation He's a contributor to cookie cutter and I don't want his money Because he's been giving so much to the community that I really don't feel like he should be using the money from his Spare-time projects or like he doesn't get any money. So it's his own money, which is going into cookie cutter Carol she's a PSF or used to be PSF director then a couple of my friends donate to my patreon. I Think there are two people who are Which don't have an association with the project itself or friends of mine. So it's It's kind of interesting to see how many people suggest for us to kind of make the work of setting up an open Collective or a patreon or a Kickstarter, but then in the end No one actually Contributes to any sort of sponsorship of funding There is a lot of unseen work So every time at Euro Python I ran a sprint that was mostly for pythes We also made the effort of ordering stickers kind of just advocating for the project then exploring all of these ways to get sponsorship Then submitting talks to community conferences. So I try to speak every time Every year at a couple of conferences. So that's a lot of work just writing the proposals and submitting them and then Also kind of working on the slides and that's not pop or it didn't It wasn't part of my job back then Today, I'm fortunate enough that Mozilla actually supports my work on on this and then The suggested solutions that come from the community or other folks oftentimes require even more work from antennas So the sponsorship is a good example. Like why don't you just set up an open collective? So then you spend a weekend of on setting up this open collective and in the end no one actually donates. So it's kind of Also quite frustrating if you if the solutions that are suggested to you Require you to put in even more work, which is kind of what you want to avoid And then people don't respect boundaries. So I don't know why but People sometimes don't create issues on our issue tracker, but they send emails directly to us And in some cases it's interesting because I don't have my email public on Github Yeah, I have the email because in Germany you are required to have an imprint So I have my contact details on my personal blogs or someone would be it's really easy to find out my email But for some other people who got contacted they don't have this anywhere public outs outside of the git commit So apparently some people make the effort of cloning a project and looking at the git log And then using that email to contact people and ask for support and I think that's that's really Shady and you shouldn't do that like please respect boundaries Especially if the people of the people work on that as part or not as part of the job really So don't do that and then a Realization that I made or the cause of these years is that at some point maintaining open source projects is Really just work Which you're not paid for and you don't have paid time off and you don't have Health insurance and you don't have working hours, but it really feels like work So I think it's important to set boundaries for yourself. So You can use tools maybe to Delay emails to be sent to you outside of working hours or something For me that's mostly unsubscribing from the notifications on github And then I just check the repository if I feel like there is a need for me to or if I feel like I have the resources available to Actually work on the project Yeah And then I think it's a general lack of empathy towards other community members, but also maintainers Which is I think the underlying problem So I want to get to the takeaways for the maintainers So first set expectations. I think it's really important that projects have a clear description. What does their project do? What platforms do you support officially? What does the project not do so when I started adding it to Poyo? I added this line. This is not compatible with Jason and it cannot write yaml like once I added this to the Documentation I the number of issues that were created decrease drastically So just like those two sentences pointing out that my project is not the fancy new Yammel parser for Python helped quite a bit in reducing the amount of emails that I got Then work towards making yourself redundant. So use tools like continuous integration use documentation generator Right simple idiomatic codes like stick to best practices Nowadays we have black to auto format codes I really like that about goal that I just don't have those stupid discussions around whether we should use single quotes or double quotes like use those tools and Stick to best practices because it will make your life much easier because you don't have those discussions Which are really pointless and then say no like if people have some really exotic use cases for your tool don't feel obliged to Maintain this as part of your tool but encourage people to build tools or project on top of your project if that's possible Then lead by example, I think it's just important as a maintainer to kind of showcase what kind of behavior You would like to have for your community so don't be snarky or abusive or anything just Like we see sometimes on a Linux mailing list maybe be nice to people and have empathy and Then do not tolerate toxic and abusive behavior So adopt a code of conduct and enforce it. This is really really important if you want to attract more contributors and keep yourself safe from from toxic members and Then allow yourself to take breaks and even walk away So I think it's important to keep in mind that as a volunteer you always have the option of just taking a break and be absent for Days weeks months and maybe even forever. That's that's a perfectly valuable solution try to to kind of Document the process of bringing on new maintainers If you care about the project so that the project in itself doesn't die But since it's also open source, you're not really the owner of the source code So just kind of keep that in mind that it's it's out there Just put in put like mechanisms in place that you can go away So for instance, if you rely on PI PI releases be sure to add more index maintainers to the to the project and Then take care of yourself. I think that's that's the bottom line for me as a maintainer to other maintainers maintain a healthy work-life balance and open source is work at some point don't Sacrifice your sleeping schedule for reading emails from random people on the internet with opinions and That's that's it. Thank you so much. I will Publish my slides later to that's my speaker deck profile. I will push them later on. Thank you Okay, I'm the speaker said we don't have the Q&A so if you're in question, please come here perfectly and By the way, I was thinking they are speaking of speaker give us many cheap or advice for the contributor and the maintainer and I'm really useful. So if you have a question and the boy, please review the video on YouTube again And this time and we are giving a warm welcome to Mr. Raphael again. Thank you