 Hello, everyone. Oh, go ahead. Welcome, everyone. So we have Andrew Bogot and Brookstorm here with us to talk about Wikimedia Cloud Services. So let's get started. Hello, yes. As already introduced, I am Andrew Bogot. I have with me Brookstorm. We are SREs on the Cloud Services team. This talk will be an overview of the services that Cloud Services can provide for you. For the most part, we'll be covering what you can do, but not so much how you can do it. So once you know what you want to do, you should feel free to follow up at the end of the session, ask us questions, or find us on IRC, or ask fellow attendees about how to actually do things, explaining the how would cause this 20-minute session to be a 20-hour session instead. So Wikimedia Cloud Services provides hosting and storage and data access for projects that are involved with the Wikimedia movement, but are not themselves the Wikis. So bots, tools, websites, anything that's not under an actual MediaWiki install is most likely running on Cloud Services. All of the resources we provide are freely available for any work associated with the movement. And also, all of our systems are maintained in public. They use open source software, and our code is on public Git repos. So to start with, just a few numbers. The percentages and accounts on this chart are up and down a lot, month to month, just like page views are on the Wikis. But you can see that the numbers are big, right? There are a lot of people participating in our projects, and a great deal of the content that is contributed to the Wikis is passing through Cloud Services and makes use of Cloud Services. If you spent any time at all using Wikimedia's Wikis, you're probably already using or have seen some things hosted on Cloud Services. A few quick examples. If you use an offline reader, the content is generated by Qwix, and Qwix builds its files using Cloud Services. If you've used Wikisource, there's a download link in the sidebar, which relies on Wikisource export, which is used to create about 20,000 EPUB and PDF files every week by people who want to read Wikisource text. There's the Wikidata game, which is used to quickly insert properties into Wikidata. And there's deployment prep, which is the staging zone for edits or changes or code contributions to MediaWiki. Things get rolled out to deployment prep and tested before they are released to production Wikis. Here is a quick rundown of some of the execution platforms we provide. I'm gonna present these in order, starting with tools that you can start using today without any setup, and then I'll make my way up to the more complicated, more powerful options. Anyone with a Wikimedia account and MySQL skills can sign on to query and query the database replicas. The database replicas are stripped of personal editor information that contain all the page content and history for most of these. Not the content usually, just the metadata, but yeah. Yes, sorry, correct. The content you can get through the APIs. Yes. Which you can do with pause. A segue, another platform that you can log into right now using your Wikimedia account is pause. Pause provides an interactive web-based notebook to run your code. You can write snippets of Python with ready-made PyWikibot and account integration, which is great for research or one-off batch jobs. Pause also supports R and bash besides Python, but the PyWikibot integration makes Python the most effective tool. It's also possible if you really need to run a command line shell in your browser in pause. If you want to write persistent software, then you probably want a tool forage count. That is a system that gets you a shared, sorry, a login shell on a shared platform. The signup process is pretty quick. Once you have an account, you can write your software in whatever language you like. Getting set up is more complicated than pause and you need to be a little bit familiar with the Linux shell, but you can do a lot more on tool forage than you can do in pause. There are a couple of very common tool forage use cases. The first is writing a bot, which is just a piece of software that runs all the time and then either accepts messages or interacts or connects to IRC or whatnot. We have some automatic frameworks that will keep your program running basically forever or run a job every day at four o'clock or something like that. They'll also have ready made access to the database replicas as in query or pause and also access to a tool specific database account. The other major use of tool forage are web services. This is another thing that we provide a lot of support for. You can write your little flash gap or your simple web service and then we automatically will provide you with URL and SSL termination. This is usually done by launching your web service in a Kubernetes container. And if just a login on a shared system isn't enough, we also have the VPS service where you can apply to have an entire cluster dedicated to your project. This is cloud VPS where VPS stands for virtual personal server. The upside to this is that if you have a thing that isn't easily containerized and you wanna run it on a VM or a fleet of VMs, you can do that. The downside is that once you have your own VM, you are also the sysadman of that VM. So this is the solution that's sort of like having AWS account or Google Cloud account where you just get a VM and you get a login to it and then you just set it up as you like. Okay, that is the code execution side. Now I'm gonna turn it over to Brooke to talk about what data is available for you to access from your tools and projects. Okay, so yeah, we provide a number of data sources that you can access from within cloud services to make them a lot more useful. The main data systems that we have, that have special uses in the cloud are the wiki replicas, tools DB and dumps. Wiki replicas are our real time system in the sense that their live database replication actually coming out of the production, MySQL and MariaDB databases directly. They're sanitized over the course of a couple of different steps for public use. And at point in time, they should be appropriate for that. They're a queryable database from Query and that's what you're hitting. And MySQL client access is automatically granted to all users of Toolforge and Paws. Slide. Tools DB is where we provide a, using the same credentials as for the wiki replicas in Toolforge, you're given access to a shared database instance that allows you to create your own database and write to it for your web service, which can be awfully useful times. It's sometimes difficult to just query and process it and put it on an app. Sometimes you do a need to share state. Next slide. And we also help administer the dumps, which is a service that you may be familiar with already because obviously it's on the internet and you can access it from there. And there's also mirrors in other places around the internet. It provides several formats like HTML, XML, and JSON. It includes the data and the content as well as the metadata. The thing about that is that inside Cloud Services, we also provide direct NFS access so you can actually interact with the dumps, tar balls directly on the file systems without having to download them first over the web, which can be very useful and it gives you a lot of data that you can work with if you don't need to query the wikis in real time. Next. Yeah, so that is a very quick rundown of the most commonly used services we offer. To get access to those, you, well, for pause and query, you don't really need an account. You can just go ahead and use your standard login account like for ppedia.org or whatnot. To use Toolforge or Cloud BPS, you will need a developer account, which you can create the tools admin.wikimedia.org. Once you have an account, you can join an existing tool, you can join an existing project, you can create a new tool. If you wanna create a new VPS project, you need to apply and then we have a fabricator workflow for that where we can discuss what you need and what you're using it for and so forth. Ordinarily, our turnaround time is maybe two or three days at best and maybe a week at worst for approving these account requests. But certainly this weekend, we're trying to be available and on call all the time. So if you want to apply for resources, please, or an account, please just ping us and we'll do our best to get that set up right away so you don't waste your hackathon. Yeah, and anybody who doesn't have these things already, you might wanna screenshot this slide because it's got some good links. And the next slide as well, which is contact information. We have several communication channels you can use to get in touch. We have mailing lists, we have IRC, which is mostly on Libera starting yesterday, but we're lurking on FreeNode still as well. Our user docs are on Wikitech, which is a Wiki so you can contribute to those docs as well as read them. There's also administration documentation, which is what we used to actually maintain cloud services. And this is an unobvious point. You are invited and encouraged to use Wikimedia cloud services, but you were also invited and encouraged to contribute to the cloud services infrastructure itself. There are several volunteers that have almost, many of the same root privileges that we, the staff have. So if you're interested in helping, please talk to us. And again, everything is done in the open so you can see what we're doing. You can see the code and get repositories and so forth. That is our talk. I think we're set up to field questions in the YouTube chats. So we're just gonna sit back and answer questions, but I'm also gonna back up onto the, how to contact a slide because that seems like the most useful persistent info. And then we'll just give you a flash of credits at the end. Thank you very much all for being here. Thank you. So we do have a question. And that is, can we host a Wikibase instance on Wikimedia cloud services? Is that the place to do so? Would an instance hosted this way possibly be a third from the projects? Do you wanna answer, Brooke? Wikibase, I don't know exactly what Wikibase requires in order to run, but you can definitely host a Wikimedia instance and any of its necessary plugins. We do provide, as a convenience, we provide a Wikimedia vagrant class that you can use on your instance or you can build it from scratch. And that is a place that you can host it. It depends as long as it's something that is valuable to the movement, not something that's just useful to you. Yeah, I agree. The answer is almost certainly yes. The only reason it would be no is if somebody were trying to host their own, if it was something that would be best done on fandom then we would send you the fandom. But if you're using content that is consumed by or relevant to the sort of general mission of the movement, then you would be welcome. That sounds like it would be a cloud BPS project. And as Brooke said, it doesn't sound like something that we have ready-made setups for. So we would almost certainly provide you with the resources but you would probably be on your own for setup and administration. Awesome, so the next question. What is the difference between Forge and PBS? Should one just run over the other? Oh, thank you. I was just gonna ask you to paste because I'm having a little bit of audio trouble. Okay, well. Okay, go ahead. Okay, I'll do this one. Tool Forge is intended as something of a platform as a service in the sense that you can go on there and there's a set of tools that allow you to take your code and put it on a web app or to run a bot or a cron job with as little effort as we can make it. Right now you still have to get a log into a shell and do things like that. And there's tutorials online on how you can get your services up. But for the most part you have things that are maintained by our team that are running it like the actual web servers, the actual database, things like that, that's all maintained by our team. On the other hand, on Cloud VPS you get a VM or you get a set of VMs, you get a quota where you can make VMs and you can make instances and then you can do what you need to do on them. So if you feel confident that your project is going to need and that you're capable of doing something like setting up your own web server, setting up your own instances, your own cron jobs and things of that nature, you have a somewhat freer hand and you have more options in Cloud VPS because you control the servers. On the other hand, you have to control the servers and we don't support them directly. We just help where we can. Yeah, but the vast majority of tools or projects people propose can be done on Toolforge and that's almost always a better user experience. But often what happens is that somebody will open a request for a Cloud VPS project and then we discuss and figure out a way to have them move to Toolforge or discuss with them whether it's appropriate or not. So feel free to engage if you have a particular puzzle in mind and we can help you figure out what the proper platform is. Definitely. Moving on to the next question. So what about hosting a bot on Wikimedia Cloud services? Does that require to request an account for Toolforge and is that account different from the Wikimedia developer account? I'll answer the second part and then let Brooke answer the first part. A Toolforge account is effectively the same thing as a developer account. Typically people apply for Toolforge account on ToolsMN and then they get both and then you're off to the races. Of course, if your bot is gonna talk to the Wikis then you need permission and whatever flag is necessary for your bot to talk to the Wikis. That's a thing that we don't, aren't very involved with and I don't know a lot about but that's the other flip side of the bot if it's an editing bot. I said that I was gonna let Brooke answer this but now I'm not sure. Is there a more specific question there or did that answer the question already? Then I'll let the person who asked the question reply to that if it answered the question. Let's move on to the next one. Any plans for providing an environment to host static file projects or docker containers directly? We already do provide the ability to host a static file which is nice. In Toolforge, generally, we have something called ToolStatic that I don't have a link to at the moment somewhere but in Toolforge there is a public HTML folder where you can basically just put HTML in your tool account which does put it online under your tool name dot Toolforge dot org. As far as docker containers, interestingly enough, when you're running a Toolforge web service, you're typically running it in Kubernetes. So you are running it in a docker container but it is a docker container that we created and you are serving an NFS volume to that docker container. We are not currently providing the ability for user-created containers to run as a service. And for services that are not suited to Toolforge, that is we don't have an alternative for that right now in terms of like a serverless type deployment or things like that, not at this time. We're working on other ideas but not yet. Yeah, literally just for file hosting, we have, I think, budgeted hardware for the next year to support some sort of object store, Swift or S3 probably will be Swift. So that'll at least provide you with a place to stick files that the public can see. But it's very early days, so I can't say a lot about what the user experience will be like for that. Yeah, I think the best we have is stay tuned. Yeah, but at least, I mean, it's not someday because we have already ordered the hardware that I think it's maybe for a middle of the, maybe like come Christmas or something I think is when we're getting the hardware. I think we've answered all the questions in the chat. We have a few more minutes. So if folks want to go ahead and ask questions, please do. Yeah, if there aren't any more immediate questions, I just want to put in another plug for our ISU channels. We, the staff are in there during our working hours and outside of working hours, but there really are a lot of helpful volunteers that know as much about the services we do. So that's really the place to go. It's great. It's generally very friendly and helpful. And that's, I would encourage you to use that as your first place for support for getting started with these things. The cloud channel is actually linked to a telegram channel as well. They're mirrored. How does that work? We don't have that on our slide though. Yeah, if you just visit wikitech.wikimedia.org, the front page has a bunch of links right there on the front to cloud services docs and communication tools and such. So that's another good place to start. If you're not already IRC, you can start there and that should get you going the right way. I know they can't see this, but that's a link I see. But yeah, it's always a good idea to go to wikitech. For questions first. So Vithra, do you want to send that back into the YouTube? To do a lack of preparation on my part, we are in our own chat and video and using Vithra to pass things back and forth. So. Yeah, let's put the link down there. Okay, well, thank you so much everyone for attending. I think we can stop the recording and get a break before the next session. Thanks everyone. Thank you. Okay.