 Hi all, welcome to the Jenkins Online Meetup. Today is August 19th, and we have a Cajun team who will present GitHub app authentication and the checks API integration. Just to introduce the Jenkins Online Meetup, it's a regular online meetup organized by Jenkins contributors, where we present various stories related to Jenkins use cases, play games, development tools, and basically anything about the Jenkins project and the Jenkins community. And we invite everyone to participate if you're interested to know more or if you want to join a live discussion and discuss your topics. And we are always looking for speakers. If you want to know more about the online meetups, there is a meetup page on meetup.com. So you can subscribe to the meetup announcements here. And if you're interested to participate, et cetera, we also have an online meetup page on the Jenkins website. So here you can find information about what we do, how to participate, how to apply as a speaker if you want to present something. And again, any topics about Jenkins are welcome. Today, as I said, we have a presentation about modern GitHub integrations. So you'll have a presentation how you can authenticate Jenkins to GitHub using GitHub app authentication. And then we will talk about GitHub checks API and support for reporting static analysis and code coverage reports to GitHub checks. And if you have any questions during the presentation, please ask them using ZoomConnay. So you should see a Connay button on the Zoom panel. You can ask a question at any moment and then we will ask the presenters after the presentation or maybe answer the questions that's in pronunciate. Also, if you want to have a discussion about particular topics, once we finish the recording, we will have all the record discussion where basically we will grant everybody voice permissions. So we will have a kind of expert zone where we can discuss any questions related to the talk and maybe for other Jenkins use cases. Please remember both code of conduct and quickly just the big kind. And again, all these meetups are organized by contributors. We will appreciate any feedback and we invite you to participate in all the discussions. After the meetup, there is a GitHub channel specifically for GitHub checks API project and we also appreciate feedback from meetup participants. So there is a feedback form. I will share a link in the chat later and we will appreciate any feedback so that we could make our meetups better. Slides are already published. So again, I will provide the link in the chat. Okay, so I guess I can hand over the presentation to team and meanwhile, I'll probably launch the poll with a few questions. They may help us to get more information about your big background and interests with regards to GitHub and the patients. So again, we will appreciate your feedback and team, the floor is yours. Okay, you can hear me all right? Yes. Cool. Hey, everyone. So I'm Tim, I'm a Jenkins Core maintainer and a lead software engineer at Canos. I've been working with Jenkins for about the last eight years or so and I've been maintaining a couple of plugins for the last two years and Jenkins Core maintainer for the last six months or so. So today I'm here to talk to you about GitHub apps. So this is something that I contributed to the GitHub Brunt source plugin back in March or April or so. It allows you to, instead of using a personal user token for a real user, allows you to actually act as a machine user which the primary benefit to start with is much higher rate limits. So many people have been using GitHub much higher rate limits. So many people will be familiar with the 2000 rate limit and having to wait for their Jenkins to catch up, especially on busy organizations. So just about the minimum gain you get here is you go from 2000 calls an hour to 5000 calls an hour, which is really huge for a lot of places. But that also scales with your organization as your organization grows. So for an organization with a few hundred repositories to 1000s, you get up to 12 and a half thousand an hour. And if you pay, you get more money and you go into the enterprise plan, you can actually go up to 20,000 an hour as well. So that's kind of behind the scenes improvement but for ease of use and onboarding, you have user independent authentication. So you don't need to create an actual user. You don't need to create an email address. You don't have to have someone associated with it. You don't have to worry about when someone leaves. It's a machine user. You get fine-grained permissions management. So you don't have to just give it right access all over the place. Some people even give it owner access so it automatically gets right access. You don't have to do that anymore. And a great benefit, which Kizzi will be talking to you about soon is access to the GitHub checks API where you can put the results from the job directly into the GitHub UI. You can get your static analysis results and you can rerun jobs. Just integrates the GitHub developer experience a lot more and stops you from having to leave GitHub and go to your Jenkins. Here's a small tip that we hit recently on the Jenkins project infrastructure. If you're allowed to organization, you should definitely change the rate limit strategy. So there's a default one which has had for a long time just to normalize it over the hour. The other one is the throttle at or near the rate limit. You're unlikely to ever get near the rate limit with GitHub apps, but that the default strategy thralls you way too early with GitHub apps. We were seeing throttling with 6,000 requests to go and changing this meant that we were able to complete an org scan for 1,600 plugins in under 50 minutes. Contrasting that to what we previously had with a personal users token that used to take about four hours to complete an org scan because of rate limiting. So that was a huge gain for us and just a very easy tick box change if you are having any issues on live organizations. You can easily, so one thing you may be concerned about is how do I use this in my current pipeline? You might call out to GitHub to do things like add labels to pull requests. You might be using the deployments API to be registering where an app's been deployed to. You may be pushing code. So you may be automatically detecting stuff in your pipeline and maybe automatically changing files or publishing releases. So good news is it's very easy. Jenkins handles this for you behind the scenes. So GitHub app authentication, while not complicated, it's non-trivial. You need to sign requests with JSON web tokens. You need to have two different authentication tokens, an authentication token and an access token. So while not complicated, it's something that's abstracted away for you and all you have to do is just act like this is a regular credential in Jenkins. So you see it there, this is just the standard with credentials. You even look it up as a username password and in your password variable, you'll get the GitHub access token and you can use that just as normal. The example here is to making a check run. If you wanted to make a custom check run, but I'm using this to add labels to pull requests and to call the deployments API with no issue. And we're also actually publishing changes to GitHub as well by changing the files in our pipeline and committing back and that all just works. So that and there's a blog post which was the afterwards which has all this in it as well. So I'm just gonna give a quick demo of how you set this up and just point you at some of the documentation and some of the resources available. So the first thing that you need is, so this is using, it's called the GitHub branch source plugin. It's installed by, it's one of the plugins that's suggested by default. So if you install your Jenkins and you just say select and recommended plugins, you'll get this automatically and there's a documentation guide here for GitHub app authentication. It's on the front page of the read me and if you just go through here, you'll get all the details that you need. It tells you what permissions you need to set this up and all the permissions and all the fields that you need to fill out and just all the details. And I'm just gonna quit, I'm just gonna relatively quickly just go through it with you. So the first thing you need to do is you need to set up a GitHub app. Oops, that's not quite the right place. So I'm using this on my personal organization. It doesn't have to be an organization. You can use it on your personal account as well. I'm just gonna sign up a new GitHub app for the Jenkins Meetup demo. I'm just gonna put a homepage URL just of my GitHub profile. Don't need to worry about any of that. So you need to set up a webhook even if it's not activated and I am using this locally but so you just need to set a valid URL. So again, I'm gonna set this to my profile and it's not active. Actually, I will just leave it active just to show you. You need to pick the permissions that you need. So there's some minimum permissions that you may need which are detailed in the guide. You may also want to add a few more on as well. So the minimum to use GitHub apps is just commit statuses, read and write and pull request read and contents read. You're quite likely want to increase that though depending on how you're using Jenkins and GitHub. So what I'm using is contents write. I'm also using deployments read and write. I'm also using pull request read and write and checks. And for Kishi's demo coming up is you also need read and write here as well. So it's up to you. It just depends on how you're using it. It's easier to set up ahead of time but you can also update it as you want truly. You may want to set up some event subscriptions here. So things like pull request, check suite, check run, push. And you also get the choice of installing it on multiple accounts or just one. Most places will only need it. They'll have it as a private GitHub app and only install it on one organization. But you may have multiple organizations especially if you're using GitHub Enterprise. So GitHub Enterprise, different teams tend to have their own organization but with standard GitHub, you may or may not just be running on one organization. Just take this any account if you do need to install on multiple. Okay, so that's the first thing here. So GitHub's now telling me that I must generate a private key. Okay, that's good. So let's do that. You can also go through here and set like a logo and description information about it. So I've just downloaded that Jenkins Meetup Pem key. So the next step here is to just set it up in Jenkins. So you go here to manage credentials and then you go here and add a credential. You want to get GitHub app. First thing is actually, so I want GitHub app ID. So this is just the Jenkins internal ID. The next thing is GitHub app. So the next thing you need is the app ID. So that's on the general page of the app called app ID. And then I need to add the key. So I'm just going to get the one that was downloaded earlier. So if I just bring up the terminal here, it's called Jenkins Meetup. So if I just cat Jenkins downloads, Jenkins Meetup Pem and I'm just going to put it onto my clipboard. And if I just paste this in here and then I'm going to click test connection. So what I've seen here is that it didn't work. So there's something wrong. If I click show details, it's telling me here that private key is not in the correct format. So there's an issue that GitHub gives you the private key in PKCS1 format. It's quite difficult. There's no standard code in Java to work with it. There's some extra libraries you can add and it's a little bit complicated. Or you can just convert it. It's truly convertible. So we're just going to convert that to PKCS8 format. So let me go here. So my current key is this and it's going to be called new key. If I cat new key and if I put that onto my clipboard again on that and that's test connection again. And there's another issue. This is a common issue. You see that GitHub doesn't actually prompt you to install it into your organization. So it couldn't authenticate. Has it been installed to your organization? So you need to go to this install button here and click install. And now you can choose. So there's another benefit is you can select here which repositories you want it to have access to. Easiest thing is all but depends on your threat model and just how you want to see how your organization is structured and how your GitHub is structured. But for the sake of this demo I'm just going to click all and that's now installed. So let's try to test the connection again. Okay, so that worked. You'll also see this help text here alongside all of these tells you where to find it and help text here also tells you that you need to do the description. But I would just always recommend when you set this up just using the test connection button. There's an advanced feature here if you want to have a GitHub app installed across multiple organizations you have to specify which one it's connected to just because of a internal implementation detail on Jenkins. It's quite difficult to share that information. So it needs to be set on each credential currently. So I'm just going to test connection one more time. It's okay. So let's save that. And now for the end bit where we hope everything works. So I'm going to call it Tim J Org. The GitHub organization. And see that was pre-populated. And I've just picked the credential and the GitHub app has been verified that showing that it's all authenticated. You also know that it says 5,000. If I was using a personal token that would just say 2,000. One thing to note is you do have to specify the owner here. It needs to be, even if you've set it to be a different one in the credential, they both need to match. But let me just save that. And you'll see here that it's connecting GitHub app for Tim J Org. And that's all going through quickly. And you'll see they're all starting to show up and trying to start build. And that's all working. So that's the end of my demo. Thank you for your attention. And there's one thing in Q&A. Currently we raise PR in GitHub using the hub pull request command using GitHub app auth. So raise PR in GitHub. So I think I answered that during the slides. So with hub PR, you need to provide an API token in some way. And you just use that with credentials that I showed and inside of the closure you will have a access token available to you. And it should just work as normal. Tim, I don't understand why I would want to do that. Can you, I use the hub pull request command all the time but I don't understand why I would prefer to use the app, the GitHub app authentication token instead of my personal token for a pull request. I think it's in context of doing it inside of Jenkins. So you've got an automated process that changes some files or generates some files. And then you want a pull request created for review. Thanks. Cool. If there's no other questions, then we'll go on to Kishi. So just to proceed because GitHub app authentication is basically a foundation feature which enables many additional integrations in Jenkins. And now we'll see a few examples for user facing features which become available with GitHub app authentication. We've also got one more question here. Instead of curl, is it possible to use PowerShell command e.g. invoke risk method? Absolutely, it doesn't matter. Curl is just the example there. But all you need to do is just take the access token from the environment variable and pass it through to how the user thinks it's okay. Okay. Thank you, Tim. So let's move on. So I'll start sharing my screen. Can you see my screen? Yep. Okay, great. So hello, everyone. I'm Kojuchun. Today I'm going to share you with my JSOC project, the GitHub checks API plugin. And currently I'm an incoming graduate student at Northeastern University. And I'm now in the gap year and you can find me on social media and GitHub. So first I want to introduce to the GitHub checks API. So what is GitHub checks API? As Tim has said before, it's based on the GitHub app and it's highly customized way to integrate the CI tools to make reports for PR. So for check, you have status, which now we are using this to indicate the current stage of Jenkins Build. And you got the conclusion and the annotations. Annotations are great because you can make warnings and notices. Sorry. You can make warnings and notices on specific lines of a code. And those annotations are just like the comments you left when you are reviewing other people's code on GitHub. And so it's very convenient to the users. And the next thing is actions. The actions is kind of another extension point for GitHub. Since you can create many customized actions and the users when users click on that button, a request will send to the integration and you just implement the logic in your integration tools and to handle that. And these are the, and these are the, this is a screenshot I got from the GitHub official document. And you will see the status and some other information here. And you can also say this is the action, the rerun action I created earlier. Okay, so next I want to explain our motivation for the project. And so first we need to take a look at how Jenkins is integrated now is through the GitHub status API. And you'll see a short message and the status here is implemented by the GitHub branch source plugin. And so you can see it's very limited. And if you want to see more, you have to click the details and it will lead you to the bloatian view of the build. And we know that Jenkins has many plugins. For example, we have the warnings plugin. And so you can see many of the warnings from different kinds of static analysis tools. And you can also see the trend for those warnings. It helps you to maintain your project on code quality. And next we have the coverage API plugin. And so it gives you, you can show you a coverage, different kinds of coverage for different files in the very straightforward way. And so we're thinking maybe we can integrate the GitHub with them. So for example, there's a source code view from warnings plugin and we can migrate them to GitHub as the checker annotations. As I said earlier, just make this more convenient and straightforward for the GitHub users. And so now we have development plugins like the general API plugin. We developed this API plugin based on the GitHub checks API semantic meaning. And we provide this general API because we want to be prepared for similar concept in different platforms like GitLab or Bitbucket. The next thing is we provide an implementation for the GitHub checks API. And so you can find those two plugins in the GitHub repository. And these two screenshots is a check I made through our implementation. You'll see the annotations. This is just a demo check. So maybe not very meaningful just for sure. And we now have released the both plugins. You can search them, just type checks in the Jenkins plugin market and you'll see both plugins. And if you want to try this plugin, you just need to install both plugins on your Jenkins and use the GitHub app credentials for your multi-branch and GitHub organization project. And the support for freestyle project is on the way we have opened the PR and then we'll merge the features soon. So after I instill that, you'll get the first one is you'll get this stage indicators which can show you the current stage of the Jenkins build. So next we use those APIs in practice and first way creates to the warning checks which is powered by the next generation plugin. And so in this plugin, if you instill this plugin, you can also get the warnings from different tools to GitHub and you'll get the query gate. The query gate is shown here as a result of the check. And you can also get many statistics for the issues for different severities and you can also get the annotations. The annotations will just show along the code. So it's very easy for you to say what happens to code and so you can do your fixation very quickly. And now this feature has been able to on cya.jnkist.io. And so if you want to take a look at the result before you try it, you just go to the cya.jnkist.io and I'll show you some here. This is just like a client plugin and it's hosted on Jenkins organization so you will get the warning checks, for example, you'll see those warning checks, like Java check style and different tools and if you click details, you all go to the check page for this check and one thing I'm thinking is that since we just reported the result here, so I think many spaces, many functions which we have just wasted it. So I'm thinking whether we can make greater use of this page. If you click this link, you go to the solution view. Yeah, so it's didn't success because of the deployment. So the status here is as well. And use, if you want to see the warnings here, they are just the same as those checks here. So if you click this link, it also leads you to the build page. And so at the end of the day, you may find there may be many warnings that you don't care for, for example, maybe the Java doc warning. So you can just add the skip publishing checks in the pipeline and script of the warning checks and you can find more about the pipeline usage in warnings, plugins, documentation. So it will report the GitHub checks by default. If you have GitHub checks, you can see the plugin installed and configured, right? Yes, it's set as a default feature. And the next thing is the coverage checks which is powered by the code coverage API plugin. And from the coverage checks, you'll get the coverage trend and the coverage health score and different kinds of coverages. And if I show you more about this. So this is a coverage check. And here's a link to the reference build. So it's my Jenkins. So it just needs to do the PR4, the last build of this PR. If you go to the build of this, another build. It's the sixth build. So it's just compared with the last build and the link will lead you to the reports directly. And you can also disable those, this coverage checks feature is also set as default feature. So you can disable them in the pipeline. And you can also find more about the coverage checks pipeline script in this link. So, and now we are developing more features. For example, we want to add a pipeline support directive from our plugin. So the users can publish checks directly in the pipeline script without depending on a consumer plugin. And for example, a simple usage is, you just want to create a check. You just set the name, conclusion and summary. And but since we like the user scenarios now, so I didn't think much usage of this. Maybe the one I'm thinking is just to indicate the coverage stage of the Jenkins build. So if one stage takes very long time, so maybe this is useful. And you can find, you can left your comments and give us some feedback in this feature. I added a link here in it though. So we are glad to receive your feedbacks. And another feature is a rerun request. For example, the users can request a rerun for field checks directly on GitHub. This is the GitHub UI. And if you trigger the rerun on the Jenkins build, we'll just, the course will show as rerun request by me through the GitHub checks API. And so the effect is. So for example, I'll show you a demo of this feature. Let me check. I'll just click the rerun here. And the GitHub will give some feedbacks that you have successfully requested. And let me see, I'm using a forward directing tool. So maybe it's a little bit small, maybe disconnected or reconnected. Okay, you can see it in the console that receive the rerun request. The GitHub checks API. And they say, schedule the rerun for job requested by me. If you click, yeah, here it goes. It started with the link, if you go to Jenkins. Yeah, let's see, I just triggered two rerun build here. And yeah, you can see the rerun request by me through the checks API. Okay, so it is ends now. So conclude as finier, just as because I didn't change anything. And this is also still a progress. So you can give us some feedbacks in the progress. I just drop the link here. So the last thing I want to talk about is how to consume our API. So since there are so many options that you can configure for a check. So maybe our, the consummation of our API is somewhat complicated. And so you just create a checks details builder and I just add those properties you want to configure for your check. And you just remember to stick to the GitHub validation because if you don't follow that it will just fail the creating the checks. So for, if you want to create a check like this I just need to use our API in this way. And you just, here the details are just the properties of the check. And as you just create the publisher from wrong and you just invoke the publish method with the details. Beyond all this progress. So this API is for plugin developers, right? Yes. So for pipeline steps API would be different. Basically what you present several slides ago. Yes. Yeah, the pipeline is, the pipeline is as far as here. That is much, much more simpler than that. Then another thing is that I show you how to create actions. So should you make standard Jenkins by adding your customized actions if you want to create a button on GitHub. So when the button is clicked a request will be sent to GitHub app. You just with the actions add two actions. The last parameter is identifier for the check. So when you request, when the users click the button the identifier, which is the last parameter of the method will be sent back to the, send back to Jenkins, for example. If I... So, yeah, obviously these buttons are part of GitHub checks API plugin, not of the generic checks plugin. Is this correct? Yes, users can extend, can just create the checks and extend as a function. So if you click the format, and then click the report. So you see the payload here and then you go to the chatroom. This here, the requested action is report. So just the identifier is set for the check. Here, because it's a master branch, so the progress is empty. And so if you want to, and then you have to implement those actions, you just need to extend the GitHub event subscriber, which is provided by the GitHub plugin. And just remember to subscribe to the Chikrown event and implement the event handling logic there. You can find more examples in the subscriber folder of GitHub plugin. And you can also take a look at our implementation for the rerun request. It's still appear. It's all from my side. I see your questions. Thanks for the presentation. And if you have any questions, please ask in ZoomConny. We still have some time to discuss them more live. And after that, we will switch to the discussion. And if you have any feedback, please use the feedback from Bing, which is in the Zoom chat. So one quick question, the GitHub. There is a GitHub, almost the same, but it's very limited. I guess if you, the GitHub is just the, looks like the status API. So I showed you earlier, you can just leave the message and the result of the check. And in GitHub, I think it's called status commit status. I don't remember the name is status commit. Thank you. And yeah, regarding that, I guess this checks API, it's not limited to source control management systems and source code repositories. Basically, you can integrate with other services. So for example, yeah, if you want to submit this data to data doc, quantifications, to page repute, et cetera. Theoretically, you could use the same API. If you want to notify them, or maybe if you just want to send a message to email, again, in principle, you could use checks API for that instead of existing email plugins. Is there a try or are there some, any limitations? The rerun, we only show for failed checks. And it's also not merged yet, it's a real version. It's just a progress. Okay, so I have a question for you. Regarding GitHub authentication support, so in Jenkins, there are more than 20 plugins which integrate with GitHub, but not all of them support GitHub up authentication at this point. So if I need to get up authentication support for the plugin, what would be the steps for me? Yeah, I'll answer this one. You shouldn't need to do anything because so those plugins will be using a username password credential in most cases. So it'll either be a actual username and password, although that's getting deprecated by GitHub in November, I think, or it'll be a username and personal access token. And so it's a username password credential. The GitHub app credential basically pretends to be a username password credential. And when the password is asked for, it does the authentication behind the scenes. So it's transparent for both users and plugins. So I'd be very surprised if any plugin actually needs to know that you're using GitHub apps behind the scenes. There is a, you can specifically check if you do need to. So the GitHub branch source plugin does need to change the behavior at a couple of cases, but that's only due, so certain APIs aren't available to GitHub apps. The get me endpoint, for example. So the GitHub branch source uses the get me endpoint to check whether you're successfully authenticated. But get me doesn't exist for GitHub apps. So I think it uses the right limit API instead. But so you shouldn't need to know is the answer really. Okay, thanks. And it should be a good advantage for users of existing plugins. Okay, it looks like you don't have any questions left in the queue. So I suggest to thanks speakers and to switch to the discussion. So thanks to you, and we will publish a recording of this presentation later today. If you have any questions, please use the github link, gtc.com slash github checks API. And you can ask any questions related to what's authentication and checks API there. Okay, thanks all. I'll stop the recording, but if you have any questions or topics, the discuss please stay on line.