 Hello everyone. Thank you for sticking around. I seem to always get stuck with the last time slot of the day So it's always good to see people hang around and make it through the day Before we get started a few things I'm not going to cover in this talk This is not going to be a talk where you learn how to use GitHub actions in your projects But you'll learn some core concepts behind them and how they work And that might spark some some creativity in your mind as far as going back and learning a little bit more and Implementing them in your own projects WordPress is also a very large project and we have some very unique challenges that we'll cover and What we choose to do and how we approach things may not be right for your projects but they're very interesting nonetheless and This talk is mainly focused on the WordPress core code base after Gutenberg is merged into it in SVN We mirror the WordPress code base to GitHub Which I'll cover Mostly it's a convenience and a courtesy, but I'll cover that in a little bit more detail and We share a lot of things with the Gutenberg repo as far as their GitHub actions and how they're implemented We work together on a lot of things and I'll happily answer any questions you have about that as well I'm from New Bedford, Massachusetts. I've been a WordPress user for a long time. I've been a Committer since 2018 Maintained several components in WordPress including the build and test tool component, which this talk falls under Since 2018 I've been a full-time Contributor to WordPress core for at Bluehost as part of the five for the future initiative If you're interested in learning more about that I recommend you you check out Hari's talk tomorrow And I've also served on several release squads in the past most recently the 6.1 squad I was a release coordinator, which went out. We're working on 6.2. So that's the most recent one. So right now You can reach me at Desros J pretty much anywhere the WordPress Slack or Twitter So to start off, I'm gonna take you on a bit of a history lesson. We're gonna go back to 2013 and WordPress 3.6 gets released and It's really a difficult time to get started contributing to WordPress There's a lot of fragmentation and the WordPress source code is in one repository All of our tests are in another repository. You have to manually run them. They don't it's not easy to do You have to know how to set it up locally You there's a implicit build process that happens It's a script on a private WordPress org server when there's changes to JavaScript and CSS files those will get built and committed back to the repository and It's It's not great because you have your production ready files mixed with your source code files And it can be very confusing to know which files to edit which one's not to and so work camp San Francisco happens at the beginning of the year and during the hack day a handful of contributors get together and discuss this and and how to make this more streamlined and make contributing to WordPress a little bit more approachable and And after that Daryl Cooper Smith publishes a roadmap called the new frontier for core development and the goal around this was to Solve these issues we wanted more people to contribute to WordPress and It's not easy to do. We need to make that easier if that's gonna happen I like to call us to great or reorganization I don't know if anybody's called that before but I'm gonna call it that now and I think that this is one of the most impactful changes in WordPress's history It's strange to hear that because it doesn't introduce any new user-facing features Most people had no idea that this happened but it brought everything under one roof and and and really Made it contributing the WordPress core a lot more approachable So everyone should recognize what this is here if you have a WordPress website You'll recognize this as the source code files that you would upload to your server to have a WordPress site And this is what it looked like before the great reorganization before the changes that were necessary happen This is what it looks like now There's a lot more files in this prime this top directory But if you have contributed the WordPress this should look somewhat familiar to you You can see we have a source folder directory and all the files that were in that previous view are moved into that source folder The tests that were in a separate repository are now in the same repository just in a folder test folder The tools that were also in another repository at this time. They were internationalization tools Those get moved into a new folder as well and we're able to adopt a real build process using grunt and NPM dependencies here So now all of that stuff that happened behind the scenes on that private server You can run that locally on your own and and make changes to these build processes do some testing add steps to it to introduce new things like image minimization or CSS preprocessing was added later and The the other repository won't go anywhere It's gonna stay there because a lot of people were installing WordPress from that repository and so we didn't want to break anything there after WordPress 3.7 is released They The contributors in the project start to take advantage of these different tools that that opens the door for One of the first ones that we start using is Travis CI If you're not familiar with what CI or CD is it's continuous integration or continuous delivery And these are platforms that allow developers to perform some automated processes or steps To accomplish tasks in their software So most commonly this will be building your software This will be testing it deploying code to production or staging servers And by integrating this service the WordPress unit tests are now going to be running on every change And so this is what the first iteration I had to do some digging But this is what it looked like when it was initially introduced into the code base We tested every version of PHP that was supported at the time We would run that build process that was now in the root of the repository and we would just run our unit tests Under these different conditions under these different versions of PHP And ultimately this just allows us to write higher quality software. We're gonna detect issues earlier We're going to know How many versions of PHP they sub they affect And before things get out in the wild, we're gonna better be able to problem-solve and correct any mistakes that we make This file continues to grow over time. We add things like JavaScript testing code style linting In 5.5 we add PHP compatibility testing, which will test that we're not using anything that's deprecated in certain versions of PHP and At the end of 2018 there's another Monumental milestone in the project's history You might have heard of it. It's called Gutenberg and Before work as 5.0 the Gutenmerge happens and we enter into the block era And this was important for the context of this story because the Gutenberg project was built in GitHub as a plug-in and it's still there to this day and still built the same way but releases to work as core are managed in SVN and so when a release is coming up the changes that are ready in Gutenberg are merged into core SVN and That's how we right now. That's how we manage that so I mentioned that the WordPress SVN deposit repository is mirrored to GitHub and Previously we didn't accept contributions to that It was only a courtesy if you preferred to deploy from git directly as opposed to SVN But we decide that now with Gutenberg a lot of our contributors are already on GitHub and so why not embrace that a little bit and meet them there So we introduced a feature where if you link you mention a track URL to a track ticket It will automatically pull it in on track and point to that pull request and show you some status and information about it And so this opens the door to a lot of contributors that may not enjoy using SVN and some straight up refused to use SVN And We're meeting those contributors where they are and opening the door to some people that might not be able to contribute through track and with every poor quest this also means that the Code changes you're suggesting are going to be run through all the tests and automated processes on Travis And that wasn't done previously if you were updating uploading a patch to a ticket none of those tests were happening So in order to better keep track of these contributions we add a way to link your github profile to to your wordpress.org profile and this allowed us to know You know everybody's username is not the same So this allowed us to track who is who and ultimately give them the proper credit that they deserve when the version of wordpresses are released So in 2020 Gutenberg is becoming much more mature The volume and frequency of contributions is much higher And we start to see a lot of problems with Travis Specifically we start to experience very long waiting queues So with more and more contributions pull requests Every merge commit or poor quest update is going to trigger a build on Travis and The way that Travis worked was it managed your your builds by concurrent jobs And so each build would have several jobs that represented different test conditions In our case, it's mostly the different versions of PHP that we support some JavaScript testing and so on and so we We had some times where we were waiting four or five six hours for our builds to actually run and confirm that You know our tests were passing and and and all the things that we were checking for So the Gutenberg team was already entirely in github and it made a lot of sense for them to move to github actions at that point They did that and we noticed a pretty good improvement However, we were where as more people started using pull requests in github for contributing That starts to come back to play into play we start to notice that backlog again Not as bad as before but again every pull request will trigger a build and then we end up with this backlog This is specifically bad during security releases when we're updating multiple branches of wordpress at a time So after discussing with several other contributors, we decide to do some testing with wordpress core itself using github actions we introduced several github action workflows and our our first phase was to introduce feature parity and Ensure that everything that's happening on Travis we could do on github actions in a similar way We plan to observe this for a while collect some data and then evaluate whether this was going to be a good tool for us to use And then we would eventually complete or not complete the transition and sunset our use of Travis and A service called app there, which we only use to test the build process on windows Machines which Travis did not support. I want to make this clear that the intention here was not to move wordpress to github This is something that always comes up But this was an experiment and we we have to be very careful when we choose the tools that we use We can't just chase the new the newest tools and what what's deemed the best tools We have to identify the project's goals and what we're trying to accomplish with changes We have to identify pain points and then we when we do that we can evaluate whether or not We're actually solving those pain points or making them better Sometimes when you change the tools that you're used it will make it easier for contributors But it will actually introduce new bottlenecks for maintainers if they can't keep up with with the influx of contributions WordPress is also 20 years old almost this year and that's 20 years of building tools processes lessons learned We have to make sure that we take all of these into account when we evaluate changing our tool sets switching off of track means that we have to spend the time evaluating all of those and Ensure that they're either rebuilt in a new way or that they'll still work and we don't have any issues that we we are introducing And it's all essentially at the at the when you boil it down. We're just trying to manage the risk that we're introducing With github supported only committers need to worry about SVN now as well. So embracing people where they were To accept these contributions But still allowing the committers to use the same tool set to commit the changes that made sense at this point We're also taking advantage of a more unified interface with this contribution experience one example of that is this instance here where when a PHP CS code style check is run We can actually contextually show annotations to the contributor Where their failures are occurring? So they would see this they would they wouldn't even need to run the tools locally on their own machine They can just submit their PR and then they would have more information here Normally, you would have to go through the logs of the the CI build and look at the the command line output And to determine what it was pull it up in your editor figure out what's going on But this makes that a lot more easier It also opened the door for some better contributor onboarding This is a workflow that we introduced where when a contributor opens their first pull requests to WordPress core They would get this welcome message with some detailed information as far as Some expectations we have set their expectations correctly and hopefully this will lead to better contribution experiences overall So we get all this done and about three weeks later There's a new pricing model announced by Travis CI and it becomes clear very quickly that we No longer can use Travis for our testing Within two days testing across the entire WordPress organization stops We use up all the credits that they they gave us when they switched to this new system And so we have no testing happening for any of the changes that are happening So obviously this increases the urgency to complete this transition So changing plans. We're no longer going to observe and collect We have a very limited amount of time that we did observe and and it's working for us We think that this will be fine to do so we're going to complete this transition now So that should be easy because we already have them committed to track, right? We already have them there should be easy If you've contributed before you know that WordPress has lots of unique challenges That's it's not that easy to do to just turn the switch So let's take a step back and talk a little bit about github actions and how they work some core concepts GitHub is GitHub this is github's own CI CD platform the concept at the center is a workflow and Workflows are triggered by events. This can be anything from a pull request is open a pull request is closed a Change is pushed to a branch a specific branch And you can also use github's API to trigger events manually Within workflows. There are virtual machines called runners that execute a defined list of jobs Jobs are sets of steps that execute on the on the virtual machine and Steps can be shell scripts that are executed or bundled sets of scripts that are distributed as things called actions There's a large marketplace of actions that are published and they do everything from caching to checking out repositories to configuring specific versions of different software like node or PHP And so we can take use make use of these actions that have been published and maintained by other people To accomplish what we need to in our workflows This is a simple example of a workflow that you could build When there is a push event to the trunk branch It's going to run a job that checks out a repository of the repository that it's in. It's going to set up node 14 Which is the version of node that WordPress currently uses It's going to install npm dependencies and then it's going to run the WordPress build script So as long as all of that completes successfully this work workflow would pass and you would know that you haven't introduced any issues This is a few examples of some ui as you might see This workflow uses a build strategy matrix to run a set of steps on Multiple different conditions. So here we're running our unit tests on multiple versions of PHP And then we also send a message to Slack based on the outcome of that of those tests Here's a smaller one where we run the build process on Multiple platforms Windows Ubuntu and macOS again to just confirm that we're not introducing any problems with the build process And that contributors won't have any any problems contributing So back to the challenges Let's talk about why moving to GitHub actions was easy at first but then not easy to complete that transition When WordPress 6.2 comes out in March There will be 22 versions of WordPress that receive security updates There are efforts on the way to reduce this number and we we just reduced it by eliminating 3.7 through 4.0 recently But until that happens, we need to make sure that these branches are being tested when security updates are being merged into them Workflows run in the context of the branch that they are in So if the workflows are not in these older branches 4.1, whatever it may be those old versions They're not going to run at all. So all of these workflows need to be back ported to each branch Testing changes over time as well Each branch is a specific snapshot of a point in history for the project There are specific test conditions to that point in time So it's not as easy as just back porting the same workflow because our PHP version supported is our main Example of that they will be different for every version of WordPress So we can't use the same test matrix there. We have to make sure that each branch is accurate to that point in time There were many specific Service-specific hacks on Travis that we use to make this happen. For example testing PHP 5.2 The images that they had on their service that contained PHP 5.2 Were deprecated and considered legacy and so we had to hard code them in certain ways in our Travis configurations So we needed to come up with ways to do that in our github actions also in 2019 This this will help with this is a local Docker environment was introduced into the WordPress code base It was highly configurable and it helped provide a consistent environment for contributors to get spun up real quickly To contribute on different versions of PHP different versions of my sequel and so on After this was tested Travis was actually updated to run the unit tests in these Docker containers So that helped eliminate all of those service-specific hacks that we had to include in our configuration file But this also only existed in some branches. We introduced it in WordPress 5.3 So it was only present from that version on We couldn't backport it easily either because the minimum version of node required to run the npm dependencies To to spin up the environment and install WordPress within it was node 6 As you can see the 3.7 for a branch of WordPress ran version 0.10 0.48 of node.js Each version of node.js that we need also has to be present on that WordPress build server that we we mentioned We can update the version of node being used in each branch But that also means we have to update the versions of npm dependencies that are being used This results in another problem that we we have to weigh carefully When you update the version of node you update the dependencies and then that results in differences in the built version of the software The main thing that we see this materialized as is differences in JavaScript files So we minimized each JavaScript file and CSS file in the build process So if we update the package responsible for that every JavaScript in every CSS file gets touched This is a big deal because when we do auto updates on these old branches We aim to not introduce new files and we also aim to keep the package sizes as small as possible We try to make as few changes as possible The reason is that the larger the package size the higher chance there is that there will be a failure during the auto update It could be a network error or a timeout. It could be a memory consumption issue But these are trade-offs that we have to consider carefully In this instance we decided it was more important to get this tooling updated and restore our automate testing and Then we would make sure to roll these out carefully and monitor them when this next update went out As WordPress has grown its build process has also grown as well Just like each branch is a snapshot in time of the PHP version supported It's also a snapshot in time of that build process that was present at the time Some tools and scripts are not present in these older versions And so they might build differently or they might have Different intricacies that we also have to consider as we update MPM Introduced the Docker environment and so on and finally WordPress develop was not the only word repository in the organization that had a Travis configuration of the 57 total Repositories under the WordPress org 20 had Travis configurations. So we had to look at each repository that did have one Evaluate whether it still needed it evaluate what updates were required in that repository and We had to achieve feature parity in those as well This was everything from the WordPress importer plug-in to the local Docker environment containers That's the repo that generated and published those All of these had stopped running within the two days of that change. So we had to evaluate these carefully So the transition is largely complete What's been done? We have more consistent tooling across all of our branches All of the branches were updated to use the latest version of node at that time, which was node 14 We are working to update to Support node 18, which is the current version in trunk But we likely won't update these older versions for the some of the same reasons that I mentioned earlier Until it's it's absolutely necessary The Docker environment is now in each old branch So if you wanted to contribute to a security release if you are on the security team You can now go back to these older branches and more easily test on different versions of PHP As well as run the tests in general The versions of MPM dependencies are now largely consistent across branches Tools such as grunt patch WordPress which allow you to easily Import a patch from a track ticket or a get a pull request is now in all branches So that will make updating our workflows in the past in the future easier We've restored all of our automated testing and scanning The workflow flyers are present in all of our old branches Our slack notifications are restored so that any failures that occur to get reported to slack and someone is made aware We also are reporting the test results to the host test results page now, which had stopped as well if you have a hosting company and several there's I think about 20 or so that do this You can test on your platform the WordPress core code base with every commit and report it back to a central location on WordPress org This can also be done using a github action And that's what we use we use that at Blue House to do that on our infrastructure and we report it back to that That location because if we start to see multiple hosts failing on certain changes We know that it might be problematic and we have to take a second look If you're interested in learning more about that you can come find me at the Blue House booth And I'm more than happy to give you more information about that because the more people that are running these tests and reporting it the better We have some things that are left. We have to continue refining this this contributing experience that this embraces One thing that we need to look at doing is replacing JS hint with ESLint These are tools that will lint your JavaScript files to make sure that you are following Our coding standards, but JS hint does not it's a little outdated and it still works, but it does not support Sharing annotations like we were with the PHP coding standards violations Travis used a The way that Travis reported the outcome of a workflow of a build There was one badge that reported a success or a failure With github actions. It's a bit different There is one badge per workflow and so there's really no canonical result that we can use to say This change is working or this change is not So we need to look at ways to do that We need to continue to look at making contributor onboarding better Automated contribution tracking. I'm working on something right now that will automatically close PRs when it looks like it's been fixed in WordPress core And what's next is largely up to you? Maybe you have some ideas of or pain points that you've experienced while contributing the WordPress We need to continue to think about ways to share this knowledge with our ecosystem Maybe there are ways that we can publish our actions that plugins and themes can use them in their own projects So I look forward to your ideas and hopefully you'll look think about contributing the github actions in WordPress core. Thank you