 You probably already know me, I'm going to talk about running EuroPython 2020 as an online conference versus running it as an in-person conference, what we had originally planned. So a bit about myself, I'm Mark Lemberg, I'm based in Düsseldorf in Germany. I studied mathematics, I have a company, we're running Python projects. I often work as an interim CTO or a senior software architect for clients. I'm a Python core developer but don't really commit anything much anymore. I help with discussions and these things. I am EuroPython Society Chair and have been for a while now. I am a founding member of the Python Software Foundation and that's about it. So I've been running EuroPython as a Chair, I think this is the third year now. So we have developed a lot of experience running in-person events. Now as you all know what happened this year, we started the organization late in November, as we usually do. We first do an RFP request for proposals from the different venues, which is a commercial kind of process that we run to find venues that are affordable. And then we had the results for the RFP available in November and then we started to work with Dublin. We prepared everything, we configured the website, we put the content there, we had a logo designed, everything was really put in place, everything was ready for starting the ticket sales in March. And then as so many other conferences, then we had to face the COVID-19 situation. Things developed very, very quickly around that time. So in February we still thought, OK, we're doing the conference in July so it's not going to be a problem, we can still go ahead with this. And that's also what we told people. Then on March 12, things got a lot worse. Ireland became, you know, they started to put some restrictions up. And then we contacted the venue and discussed with them what we can do about this because of course like Naomi mentioned yesterday in the keynote, we had the same problem. We had signed the contract already and we would not have been able to get out of the contract without actually paying, I think it was 80% of the total fee that we had signed up for. Which in our case was a significant amount, I think around 200K or something. So there was no possibility for us to do that. Then of course, even though the government in Ireland had not really banned these events just yet, it was becoming obvious that it would. So we discussed with them what to do and then we decided or they offered us to postpone the event to next year around the same time. And so that's what we did on March 17th. And then we thought, OK, what are we going to do now? Are we just going to drop the event? Are we going to continue? We're not really sure what the community felt about this. And so we ran a poll and this is the result of the poll. So as you can see here, I don't know if it's readable, it's around we had 308 responses around 78% said, well, they would like to have a virtual conference. So we had a look at the results and also the other data that we got. So preferred dates was clear that they basically wanted to use the same week, probably because they had already made arrangements for this. And essentially, they also wanted to have more or less the same kind of content. I mean, a number of days, number of tracks. So we took all that into consideration and then we decided on March 31st we decided to go online with EuroPython 2020. At that point, we had no idea what we were getting into because we were not aware of how we could run this. We just said, OK, we're going to try to figure out how to do it. We then had to spend a couple of weeks shutting down the in-person organization. So basically telling all the vendors that we are not going to do it this year, we're going to do it next year, which took some time. And then we started to research into the different online possibilities. Now, unfortunately, at that point, two of the board members, of the eight board members that we had at the time, they stepped down because they were not interested in running an online event. And so we were just left with six people. So that was another challenge that we had to face. So let me just summarize some of the challenges because that's what this talk is also supposed to be about. The purpose of the talk is not so much to present what, you know, that we are just such a great organization. The purpose is more to just so that we show other conferences how this might be possible for them to do as well. And we want to provide as much help as we can and provide all the resources that we have created and all the research that we have established and put online so that they can benefit from our research. Because, you know, like in open source where you write code and you write, you know, make the code available to others to use, we want to do the same thing for the conference organizations that run conferences in the Python space. So the challenges that we had, of course, were we had to redefine the budget. The budget was already set up for the in-person event, same for the pricing structure. We had to figure out what to do with the sponsors because, of course, the sponsor would expect to have a sponsor booth available and lots of attendees coming to the booth, talking to them directly and so on. Now, for an online event, that doesn't really work and so we had to figure out what to do with the sponsors. What we did there is we basically replaced the booths with online rooms. We gave them Zoom rooms so that they can, you know, talk to people directly. We gave them chat rooms. We put in some extra features for them to use online so that it becomes, you know, more attractive for them. We also reduced the prices a lot for them and we needed to do lots of research. So what we did is we, for redefining the budget, we had absolutely no idea what to expect, how many attendees would come to the online event. So what we did is we basically played safe. We used the numbers from the poll that we had, which turned out to be around 250 paying attendees. So that's how we structured our budget and based on that budget and knowing all the costs that we will have, we then reduced the prices a lot by 75% because we did not have to pay for the venue rent, no catering and all these things. The big cost factors were basically not there anymore but of course, we still did have costs. The conference cost around 30K and we had to cover that in some way. So we put that into the budget. We came up with new sponsorship packages and then we had a look around for chat systems and audio visual rooms that we could use. Now, a question came up and there was a discussion on Telegram about this as well. Why don't we make it free? I just want to bring this up because there was a discussion and there was some back and forth and people wanted to have this free or maybe half it at a very low price. And we thought about this a lot and what we ended up with there is we wanted to have the volunteers that we get for the conference to have them feel that the work that they're doing is actually worth something so that we are creating a value. And so we decided to use a value-based approach for pricing. So we thought what's the value that we give to conference attendees by putting all this content online, making it possible for all the speakers to provide all this content to the attendees. And so free was not really an option because free does not really have any value. And there's also something that I found because lots of conferences turned free when going online. I really didn't understand what they were doing because they were completely ruining the pricing structures of events. If they want to, if they have run a conference with the same content and now it's free and versus where you have to normally pay 700, 800 or maybe the commercial ones for 1,000 or 2,000 euros to attend, there's no relation between those two things. If you do it online, you do it for free and then you go there and you have to pay 1,000 euros, it doesn't really make sense. So I think the industry there, not only the industry, also in the open source world, the conference organizers are actually not doing the right thing by making these things free. And also specifically for us, the European society uses the income from the conference to provide grants to other conferences and to projects. And so this would have completely killed the revenue for us. In fact, we would have to pay those 30K out of our budget. And so that was not really an option. So we decided to use lower prices, a lot lower prices, but still higher than what you normally see for online conferences. And that was like, we were betting on this that people actually will see the value in this. And I think looking at the attendee numbers that we have right now, I think this worked out. So I would recommend this to other conferences as well. Because it's well possible that this status that we have right now will continue for some time. Right, so online means we need a lot more volunteers. We had to do lots of research. We did experiments, trainings, rehearsals, trial and errors. So we only had six members left who were actually active. So we started to downsize the event to stay on the safe side. We put all this documentation up online. We first, internally, we came up with a virtual conference concept. And then we were also researching to all these online conference tools. And because there was a lot of research going on, we wanted to share this with the other organizers. We put these online and we made a blog post which has the links to these. Actually, I'm gonna post these links as well. So there's a lot in there that can be used by other conference organizers, especially in this online conference tools one, but also in the virtual concept one. So you're free to copy what we did and maybe improve upon it, right? Use the things that didn't really work that well this year and maybe improve them. So what we decided for the chat system is we wanted to have a system that is, you know, it's kind of, it groups a number of channels. And the best system that we found to do that was Discord. So we used Discord. We had never used Discord before, so we needed to get in some expert. We've fortunately, we found Jason who is a Discord expert and he helped us a lot setting up everything. For the audio video, we ran a couple of tests. We then ended up deciding to use Zoom and Jitsi. Jitsi is a free server that you can set up. You can also use a shared server that's made available by a company. But then over time, you know, we thought Jitsi is nice, but it's not really that good for larger events. You can maybe have maybe 30 people or so in a Jitsi room. What the company says is that maximum is 50. After that, it simply doesn't work anymore. So for the larger things, larger talks, we had to use something else anyway, so we decided to use Zoom. And now, because we have so many Zoom licenses, we basically removed Jitsi completely. We scaled down, we just have two conference days, not three, and we have two sprint days. So the CFP was already ongoing. We had to repurpose that. It wasn't really hard to do that. We just simply extended the CFP and we made everyone aware that whoever had already submitted will now have to do the talk online. And most of the speakers actually agree to this. We also decided to use the chance to open up a bit more, make the conference available to more time zones. So we added extra slots for the Indian, Asian Pacific time zones in the Americas. Now, having more content because we actually did get a lot more content in that we had then we had already originally expected, we started out with three tracks that we wanted to do. And then we had to open up a complete fourth track. Of course, we had to find more volunteers to help run all this, because unlike for an in-person conference where you can just send the speaker somewhere into a room, and then they can just do things there. And it's easy to get on-site volunteers. We did not really have that possibility. So we had to find more volunteers. And fortunately, we found a very, very good team. We have 18 people now helping with the track hosting and a few other people helping with other things that behind the scenes. The team that we built was basically built not by doing a call for volunteers, but instead we used basically word of mouth. So we asked the people that we knew and they asked the friends that they had. And this worked out really well because we very quickly got a nice team of excellent experts with lots of different expertise. So we could assign the different roles very easily. So how do you run online conferences? Well, we ran a lot of experiments with this. We joined other events to see how they did it. We tried various flows. We found that you need two people to run a track. One role is the session chair, which is very much like for the in-person conference. The session chair introduces the speakers and the talks and helps moderate Q&A in these things. So the session chair is more talking to the attendees and the speakers. Whereas the room manager has a more technical role. The room manager will then take care of onboarding the speakers, making sure that microphone works, screen sharing works, all this technical stuff. And also manages finding the speakers before the session so that they can get on board on time. Room manager, we also have for the in-person conference, but the room manager for the in-person conference is completely different. This has a completely different role. Now, the online session hosting is a lot different than what you do for an in-person conference. And that's something that if you're running in-person conferences and want to switch to online, you have to be aware of. There are many technical challenges. The most difficult one is getting the internet connections working and making sure that everything is done. You have to have backups. You have to figure out what to do in case the connection goes down. For example, we ask for slides so that we can run them and then people can join by phone. Unfortunately, we did not so far have to do this. We also ask people to send in videos of recorded sessions in case they were going offline during the session. There are lots of personal challenges. Talking just to a webcam, it doesn't feel like an in-person conference. So you have to imagine that there's this crowd sitting there listening to you, but it's difficult. For example, the applause that we always play. This is something that we found in other conferences where they did not have this. This was really the major lacking thing from our perspective. And so we added this. It was simple to add. So it feels a bit more like an in-person conference. You need a lot more organization. You need to practice a lot. You need to train everyone. We had lots and lots of training sessions. We ran two and a half weeks of speaker training sessions, sometimes two sessions a day. I think it really paid out because we don't have a lot of issues. Also, this part we had a document, of course. So for the host, we have a hosting guide and we currently don't have that online. We are gonna clean this up a bit and then turn it online so that others can use it as well. We sent the speaker guide to the speakers where everything is basically explained. The flow is explained, what they have to pay attention to. Tools are explained. All these different things that are important to know as a speaker, both of these guides are more than 30 pages. So just to give you an idea of how much you actually have to think about. And this, again, proved to be very beneficial because the conferences running were very smoothly. So let's do a reality check. So where did we end up? Originally, we had, like I said, we planned in the budget with 250 people, but of course there was a hope that we get more people. So we hoped to have something like four to 500 conference attendees. And it turned out that we actually sold 650 tickets or around 650 tickets. Plus, we made the sprint tickets free because we wanted to attract more developers. And also, you know, our mission is to help Python projects. So we wanted to help open source projects there, attract more volunteers. And so we originally had expected around 200 people for the sprint only tickets. And it turns out that right now we have 390 signups for the sprints only. Plus, of course, the conference attendees also can also go to the sprint. So we're going to see how many people we're going to have tomorrow. And then maybe we have to, you know, increase the size of the sprint teams a lot. What's important to know for other conference organizers is that the dynamics of ticket sales are a lot different compared to an in-person conference. There's no preparation that you have to do as an attendee for an online conference. You can simply hop on. And so most of the ticket sales happen in the last two or maybe even just the last week. So that's something to be aware of. Don't get worried, people will show up and will sign up. Right, and because I wanted to make this a bit more interesting on the technology side I put up this slide. I wanted to explain to you what technology we're using. So, of course, we have the Django website over here which is what we always have. We have that for the in-person conference as well, of course. It does the registration of tickets twice. The payments, it manages registration of talks, the CFP and also provides all the content that we need. Then you have the main Discord server down here which we decided to use as a platform. So everything is linking from the Discord server to everything else. So for the attendees it's easier to find things. Initially we didn't want to set up a lot of channels but then we found that it's probably a good idea to have one channel per talk. So we ended up with, I think it's around 170 channels that we have right now which is a bit overwhelming but we found that and people also appreciated it that we have these breakout channels so that people can continue to ask questions, get in touch with the speakers more directly. So it's a good format. It's something that if you run an event like this you should consider as well. It's only important to have some consistency in there so the naming of the channels should be consistent and you always need to mention the names of the channels when the session runs so that people can easily find it. Then we have basically a Zoom setup for all the audio video. So we have in total 18 Zoom licenses running now for the rooms, three or sorry, four of those rooms have an add-on for the webinar which is what's running now here in this track for example. This is basically meant for running events where you have lots and lots of attendees and you want to have a little bit more control over what the attendees can do and what the panelists can do versus in the standard rooms basically everyone can do everything depending on how you set it up. We have seen other events that used rooms and they had major issues with spammers. Fortunately, we did not run into those here yet. This is also why we asked not to publish the links to the Zoom rooms because spamming in these Zoom sessions is really a major problem. You have to be aware of that and act quickly. So we have three for 500 attendees and one for 1,000 because before the conference, like I said, we had 650 sales so we thought that maybe we need 1,000. Turns out that you don't need the people spread over all the tracks quite a bit so the 1,000 investment here was not really necessary. So if you run one of these events using Zoom, I think it's perfect if you just use the 500 one. Down here we added some bots. The registration bot was very important because this essentially gives you permission to access all the channels on the Discord server. We initially wanted to do this manually and then Chuck went ahead and wrote a bot for this based on the registration database that we have and so this made it possible for people to self-register and this worked out really well. I mean, it's just doing an amazing job that bot. We also put up a stream bot which takes snapshots of these sessions taken from YouTube streams and then post pictures of these snapshots to the peak channels that we have. I don't know how many people use this. It's something that I wrote a while ago and I thought it would be a good idea to have people see what's going on in a different room before actually joining it. Right, so then the next big issue is that of course we had to make sure that everything stays up online and because we are all joining from different remote locations, some have good internet connectivity, some don't, sometimes the internet connectivity breaks so we decided to, we don't want to risk that these sessions go down because if they host and there's only one host per room that you have in Zoom, if the host goes down and there aren't any co-hosts to take over then basically Zoom shuts down the session and we wanted to not have this happen. So what we did is we set up a couple of desktop VMs at Hetzner and those are actually running all the, there's one VM running all the different sessions so it's host for all the different sessions and then we have for the main four tracks we have additional VMs which are used to run the sponsor ads for example and maybe show other things so that turnip would be really useful and of course we have, each of us also has a personal setup very diverse, so Windows, Linux, Mac OS all kinds of different Linuxes. These are typically set up as a co-host so we have some extra backup and some of the co-hosts then also do the playing of the sponsor ads directly from their system. Others go into the VMs and then play the videos like this. Something to be aware of is that if you want to run this as a co-host or anything that has to do with this it's recommended to have at least a dual setup dual monitor setup. It's even better to have three. Right, so going to production. So the major challenges as I mentioned was internet connectivity. It turned out that at an online event you have to do lots of things in parallel. You always have to be very, very much attentive to what's currently happening. You have to watch several different channels you know, on Discord you have to be aware that the YouTube stream can go down you have to check whether the Zoom session is still running whether the VMs are still okay. Everything is, you know, very... Well, it's not very complex but you have to know how to do things, right? So you have to train a lot. And what you have to do is you cannot plan for everything so you have to accept improvisations. You have to be aware that you cannot just, you know, expect everything to work which is fine. You know, you just have to be aware of that and not panic too much. Right, so what did we achieve? It's actually quite amazing we're running this conference online even though we had no experience running the conference online before. We still managed to get more or less the same kind of content size as we have for the in-person conference. So basically we addressed the... In the poll what people asked for. And everything is working quite well, I think. I haven't heard any, you know, negatives or people, you know, having major problems. So this is definitely a format that works. If you know how it works even better if you can, you know, if you have some experience, we now have experience and we'd like to share this experience with others. So if you have any questions, please come to us, ask us. We're there to help. Right, so thank you for your attention. I can take a few questions if there are any. Yeah, so we have two questions. So here's the first one. Is there a preferred method to give any feedback in hopes that it can be passed on to future online conference organizers? Is there... I didn't understand the first part. Is there a preferred method? Ah. Ah. The documents I showed, they are public and you can comment on them. So if you go there, even though the virtual concept that we have, it hasn't been updated in a while, we're going to update it after the conference with what we actually did. You can comment there. You can put extra hints there of what you think would be good or not so good. We are going to have a feedback form after the conference, like we do for every conference. And we're going to send this around probably on Saturday and then you can put any comments that you have into that feedback form as well. So those are the two ways to do that. Okay, so here's the last one from Gil. If this works well, would a hybrid conference format be a possibility? For example, people choosing to watch from home with a lighter Discord moderation plus live streaming without the Q and A from Zoom while the conference is happening in a venue? Well, we have been running the in-person conferences with live streaming already and people had the telegram chat. So, you know, we kind of had this kind of setup before. We're going to probably keep this. We are thinking about maybe using Discord for next year as well. But, you know, the hybrid version is not really something that I think is not going to work well because the remote people don't have enough contacts. But let's see, I mean, maybe we can, you know, make it happen anyway. Maybe there are some ideas out there that we can use. Okay. So I think we have one more question, but I think we are out of time. Now we have the next speaker plating. Yeah, we can actually take it. Yeah, we can actually. Yeah, we can take it to the break group. And there you can like ask your questions to Mark. Thank you very much. Yeah. Thank you.