 Well, we are doing a knowledge transfer session for Windows Service Rapper. We have multiple participants on the call. Thanks Mike and thanks Budika for joining. Just to share some context for those who are interested. This year we have a number of Google Summer of Code projects and one of them is related to improvements in Jenkins Windows Services. We use a component called Windows Service Rapper. And this component basically provides us an opportunity to run masters and agents on Windows and to register them as Windows Services. So this year we want to add YAML configuration and also to improve verification of the configurations. Today we will do basically a code dive. We will take a look at the Windows Service Rapper repository and code structure. We will also talk about how this component is used in Jenkins. And then we will have some discussion about how we could improve the configuration so that we keep talking about the project implementation. That's good to you. Okay, so let's just start from the beginning. A Windows Service Rapper is a standalone project. Now it's hosted in a Windows Service Rapper GitHub organization. It's a transition that we've done a couple of months ago together this next term. Currently it's been used by multiple organizations, not only by the Jenkins project. So you can see that there are 4,000 stars and we almost hit 1,000,000 downloads. So yeah, it's used in Jenkins, but there are many other organizations using that. And actually it's really a good tool right now. What does it offer? Actually, a Windows Service Rapper is a binary which you can execute. So it provides versions for .NET framework, different versions. And also for .NET Core. So if you don't have .NET installed, you can take the native executable for 64 32-bit platform and run this. So what commands does it offer? It provides several commands like install, uninstall, for installing service, and also start, stop, and a number of other commands. So basically you can use them in order to install your service, your application as a service. Let me quickly show it to you. Just a second. I'll show you the demo. There's a lot of documents which we don't need. I will just take a demo from this folder. Documents. So we will just need this file. So it gives these files. Yeah, I'm running the demo on Windows. So we draw these two reasons. Okay. So we have a number of files. Actually, in order to run a Windows Service Rapper, you need two files at the moment. One file is executable. Okay. I have two executables. One for common and another one for the net core. And also we have a configuration file. So this configuration file is really simple. Well, it won't work actually, but it would install an abstract executable as a service in your Windows setup. And you can just run it. For example, I will just install. Let's see what happens. Do we see users account control screen? Okay. Maybe not. So anyway, it required me to elevate permissions. And after that, I installed it as my application. So then I can go to services. And here, for example, you will have an application called, it was called my app service. So yeah, my app service powered by Windows Service Rapper. And here you can find the number of properties like startup type, also login credentials, recovery options, and the dependencies to be used. All these options can be configured through advanced properties. So if you go here, you can find that there is my XML configuration file. And actually there is quite a long list of options here. And hopefully we will be able to get the same options in YAML once it finishes the project. So yeah, that's what Windows Service Rapper does. Same manner I can install this service. Well, let's try to start that first. Because yeah, almost like nothing good will happen. Okay. So yeah, it installed because we didn't force logs. And now if you open a Windows log, you can actually see that, well, my executable started because actually I have it as such executable. No, I don't. It should be just in the application log. Okay, let's say you install that and then switch to our demo. So I can show the code structure in two ways. One, I can just use GitHub. Another one, I can use Visual Studio. So what would you prefer? I guess Visual Studio would be a bit more professional. But yeah, we can just start from GitHub if you wish. So here we have a number of directories. So all this source code is located in the CRC folder. We also have a number of other folders. So doc is just all the documentation including cancellation guides, guidelines for reporting, also a configuration description I described. Then we have Ensh folder, which includes build YAML. So basically it's our configuration we use on Azure DevOps to build the project. In order to release the project, right now we use another pipeline, which is not published in the repository. We will move it here soon. We also have examples, but basically there are two examples. One is sample minimal. So this is the file you have seen. And another one is just sample all options. So basically it's, well, again it's a file which includes all examples and which includes some documentation. I'm not a big fan of this file. The intention was to just show an example, but maybe we could move for this entire file, for example, to access this schema, so that this documentation and center moves there and users can develop our configurations interactively. So it's something to discuss in the future. And yes, our main file is actually CRC. So what do we have here? Here we have a solution. So what it means that this is actually a multi-project repository and this solution defines the top level build. So for example, in Java, there is example like made in parent form, in multi-modular repositories here, we have approximately the same. And yeah, there are also .NET settings, but it doesn't really provide a lot of information here. It's just used for the build flow to other right versions and other components. And yeah, now we go deeper. So there are three folders. One is for tests. We will discuss them later, but yeah, obviously it's about tests. Then there is plugins. So Windows Service Rapper has a primitive plugin subsystem. This plugin subsystem is basically documented. Just a second here. The extension points. Did I remove that? Yeah, now it's here. Extensions. So in Windows Service Rapper, you can define all rights, not to inject additional logic, and the two extensions are included in the repository specifically. So there is the directory mapper and runaway process killer. So next turn wants to kill this repository and to replace by native features, but the directory mapper still makes sense. So most likely we will kill this extension engine. And our main repository. Yeah, sorry, I'm on top. And our main repository is core. So basically core is the logic of Windows Service Rapper. It can assist of two components. One is Windows Service Core. So it's basically a library which includes the most of components and extension API, et cetera. And the second project is Service Rapper. So Service Rapper basically defines main C-Sharp, the Internet startup logic. It basically uses the library. And it also has some features for logging. And finally this repository defines how we package Windows Service Rapper. So for example, here you can see the configuration we produce. So for example, here we have dependencies we inject. And also we have targets. So for example, we create a zip file for .NET Core. We create executables. You can see destinations where they go. And that's how it happens. And also we have a number of .NET executables. So for .NET executables, that is one curious thing that we merge the variables. So if you go to the built directory, let's go to our repository. So what I'm doing, hopefully nothing copied. But sometimes it's awkward. Okay. So here Windows, I have a broken mouse. I don't know what happened. Okay, here we have a development repository. And for example, if we just go to our SRC Core Windows Service Rapper, you can see two folders. We are interested in being. So here, for example, for .NET to do zero, we produce a number of details. So it really summarizes the libraries we use. So for example, we use a library for managing the zip files. We use log for net. It's a logging library. We also include our extensions. And we also include our Windows Rapper Core. There is a simple executable there, but this executable doesn't include these libraries. So this is how the .NET applications are usually being shipped if you built them by default with Visual Studio. But what we do, we actually merge all this stuff into a single VLL. So here you can see that we do the same basically for all platforms. So for all .NET versions, we just take all the assemblies, include them, and then we generate a single artifact. So for that, we use Unerge. Unerge is just one of standard tools for .NET. Yeah, we include it here. So what it gives us, if you go to Windows Service, we have artifacts. Here you have the finally produced artifacts. So here we have .exe files, and in PDB it's a symbol of files for debugging. So that's all we get. And these distributables, which we can distribute. So you can see that the size of them is quite small. Actually, this is the executable we use in Jenkins. Notably the executable for the last .NET version is smaller because we started excluding some code which is no longer needed there. So thanks to the next term for this work. So these are classic artifacts. And as I said, we also have .NET Core artifacts. They are a bit bigger. For example, if we go to the GitHub releases, you can see the recent releases. .NET Core artifacts is more than 30 megabytes. So why it happens? Well, basically we include .NET Engine there. So these artifacts can run on any X64 or .86 version of Windows. They do not require .NET. They are portable, but of course they are bigger. So if you use Java, you could do the same, for example, by using Quarkus or whatever to make it for distributions. Okay, so this is structuring principle. Before we proceed, do you have any questions? Are you still here? Okay, I'm still here. Sorry. Okay, so this is what we have in terms of the build flow. So all the build is basically defined by project files. So it's switched to Visual Studio because it's a bit more convenient here. Is the screen big enough? Maybe not. Okay. So here, by the way, it's better to answer the worries because otherwise I have to go and open the chat. So here, the build flow is basically defined and we produce the artifacts. So how do we test that? We haven't discussed the Windows Service Core yet, right? Yeah. So Windows Service Rapport was un-executable and the Windows Service Core is basically a main component. So it's a library which includes a lot of things. Notable things which matter in this project is server-descript. So service-descript is basically a configuration file. So you can see that there is an extension point called Windows Service Configuration. So this extension point just defines all the settings needed for Windows Service Rapport to run. You may see that it's a bit old-style, but basically it's structured or repositories, number of classes, some overrides. This is what we need to configure. All these options at the moment come from configuration XML files. There are also default settings which we can use. So you can see that not all settings are actually defined. So if you're not familiar with .NET, here we use a lot of properties. So it saves a lot of time on getters, setters, default initializers and other things. So I'm still waiting for Java to have similar syntax sugar like CSR. But yeah, this is what our configuration including defaults. And this is our service descriptor. So service descriptor is a class which basically takes XML as an argument and extracts all this data from XML upon request. And then this file can be consumed. So what it means, if you take a look at the score, you can see that there is constructor and then there is a lot of different utility logic like parsing, extracting files from XML, et cetera. But what actually happens, every time you need a particular argument. So for example, when you need to retrieve working directory for your Windows service wrapper, your code goes to XML DOM and extracts data from there. It's good from the point of view that basically all conflicts are supplied on demand, but it's bad because all conflicts are also verified on demand. So right now we don't have start-up time verification except a few options. So for example, if you need stop arguments, so stop arguments is basically arguments for additional stop executable which can be involved on the termination. So this code will be processed only when you start termination. And if there is an error there, basically it's too late because you want to terminate this service, your configuration is broken, so most likely nothing good is going to happen after that. This is the main weakness of this approach and hopefully you could change it during this project. Okay, so how is this file used? Let's go back to our main cshr. So yeah, there is a lot of logic there but basically we need to find the main function. And here what do we do? We basically run these arguments. We receive from CLI. So right now there is no special common line interface processing. So all of CLI is processed basically manually which again I think we could change. So here for example you can see the execution logic so we just initialize the descriptor. Here it reads our configuration. Right now the configuration is basically hard coded so you can change the file class. There is a pull request from Boudicca which changes so that hopefully you will integrate it soon. After that we initialize loggers. So we use log for net and we also connect log for net to windows logs. Why we need that? So if you're admin you can see system events in your administration panel. So we connect the service wrapper to that by default. And after that we just start initializing the service. So you can see that there is some common processing but interesting things start here. So there are commands like install, uninstall, start, stop and all of them have handlers. So these handlers can be executed and basically they do exactly what's needed. Above you can also see suspicious elevated flag. So you may have seen that when I installed the service it basically asks for permission every elevation. It's a new feature implemented by Nexton. Before that you had to run the executable as administrator. Now you don't have to do that because basically the executable itself restarts itself and asks for elevation. It happens thanks to this elevated callback. So basically it just receives parameters and executes them. Okay, so let's take a look for example at installation. And here you can see that basically in order to install the service we do a lot of classes because for example there is an interactive support for credentials. There is also support for defining different accounts but at the end it invokes standard libraries in order to create Windows service. So we don't write to registry on our own at the moment. It was a case for example a couple of years ago and now we mostly use service managers so that everything is installed and registered here. Okay, I don't think we need to deep dive basically if it does all the configuration you have seen here. So all these options. But after that it's just the beginning. So once we install the service we also need to actually run that. So when we start the service there is another logic included. The wrapper is executable and we try to execute that so you can see that it can also elevate permissions if needed but then it basically starts the service. And the service once started up it receives all callbacks and starts all the initialization project for process. And away it's written. Just a second. So start service. So just a second. It's a real nice system. It's a real nice schema. Yeah, so these handouts we execute including create for services. And after that we should really execute the thing and let's find it. Just a second. I don't remember this part of the code but we have a way to discover it easier. So we have our service descriptor. Here for example let's just take executable. That's fine to use this. So here you can see a lot of project. So actually it's located in main class. So yeah, it just starts with arguments. And here you can see that it starts the process with executable. It wraps all the arguments. It attaches a logger. So if you go to start the process logic here you can see that basically there is a bunch of arguments for process. And this logic just eventually yeah, so there are hooks. So hooks are used for extension and vacation like post termination, etc. But the most of the logic happens in the process helper. It's our own class which is located in utility folders. And this on class is basically invokes process and wraps it by providing cold environment by sending proper signals when you need to terminate the process. And this data is basically fed by our configuration because all these options can be configured in settings. And finally you will just start the process under wait for exit. So here you can see for example full installation. So we will need workspace, we will need arguments, we will need processes, etc. We set up environments for the process. Environments again, they come from configuration plus some additional options. And after that we just start the process or basically form a dictable stream and launch that and capture all logs. So here you can see that we actually capture logs from the process. In order to generate log files as a part of our execution this log file is then consumed, for example, by upstarting our login systems like windows events or Jenkins can also capture these logs and even add them to support models if needed. And that is handled by this handle. And finally we just wait until the process exists and we do it in a separate track. So that's it. Okay, so I think we shouldn't deep dive here. Do we have any questions before we go to other parts? Sure. Well, it's you who will be coding that. So if you don't have questions, then I am fine. Okay, so for you, the main classes of interest will be of course service descriptor because most likely we will need to rebuild it completely. I started doing some work when I was splitting the repositories. So now there is Windows service configuration and some companies already support sub-configuration files. So for example, download is located in a separate repository. So we already have a kind of structure for configurations. But we will need to warm up this structure to YAML files so that every section is implemented. And service descriptor most likely will do a major cleanup there. Another file which might be interesting to you is, yeah, this XML helper. So assuming that we have existing validation, most likely we will need to add some logic here in order to invoke gratification. But this file is relatively simple. It's just whatever the location which operates with DOM and has a lot of generics to simplify the things. Also has a lot of logic depending on dotnet version nowadays. Hopefully we will be able to drop dotnet to the zero support so the code becomes much simple. Okay, so yeah, the code is relatively small. Well, let's say it's much smaller than Jenkins core or many plugins like Git plugin. But it's still pretty complicated and we have some test coverage in order to verify that. So tests are all implemented in the test repository. And here, for example, basically we have a number of tests and test utils. And we use an unit framework in order to verify the test. So basically in unit is just another implementation of X unit framework. If you permitted we have JUnit, it's more similar. And here you can see that we just have a number of tests. So for example, here main test just verifies whether common iterations could be done. Like printing help. So here we can invoke it. Yeah, I thought that visual studio allows to invoke it from here. Okay, if not, it should be available here. Yeah, run tests. I'm not sure, but testing is a bit slow in visual studio. So here what you can see that we do testing for a number of configurations. For example, here we have main test. So let's stop the previous test run. Okay, let's rerun it. This interface is relatively new and it's still glitchy before that unit had its own separate test UI. So here you can see that I guess unit still has it. So here you can see that we just execute the test. We can also potentially additional outputs. I'm not sure what happens with that. But theoretically everything should be operational and we know for sure that these tests work. So they verify the results. Yeah, here's our standard output. Just didn't change the focus. So here for example, we expected that you run an existing command and we got exception. So the test passed. So that's the plan. Okay. So for more advanced tests, we have some tests for extensions. We have some tests for utilities. There is a lot of test helpers which automate our process and code configs, et cetera. So you can see that there is some test coverage. I don't think that test coverage in this repository is perfect, but at least we get enough smoke tests. And there are always issues. So why we cannot really test it entirely? Because for example, in order to test the service management, you need a Windows version, which actually supports that. So you need to run as administrator. It's technically doable now in Azure DevOps. It wasn't doable before in... So we were using Carpier. And if you try to do it in Jenkins, most likely it won't be possible as well. But yeah, right now we don't have integration tests which would verify the service part. So it's something you need to keep in mind because this part likely needs manual testing. But you can test that fine. Okay, questions? Can you explain the difference between starting in CLI mode and service mode? You mean in Windows services itself, right? Yeah. Okay, so basically let's go to main. So in main... So right now... Just a second. We actually need to run. So here, how we discovered that, we have a flag in CLI mode. Basically checks the number of arguments. So if you're supplying more than zero arguments, then you're running in CLI mode. And basically this is an interactive mode for admins. So for example here, help or whatever. So all these commands, they've been executed in the interactive mode. And basically this is management CLI. If no commands supplied, then it will try to run as a service or it will assume that it runs as a service. There was a patch which in recent versions were an external try to improve this behavior. So for example, right now if I just run without parameters, it will show me that I cannot start service from command line or debugger. And the service must first be installed or whatever. So basically it tried to start as a service. It tried to retrieve service metadata. It failed and you get an error. So this behavior is not ideal. It would be better to just print help. And yeah, the next term tried to do that a while ago. So in a version of 2.6.0, this didn't really work out and we had to reverse this patch because it cost regressions. So right now we're a bit, not that ideal behavior, but at least it works. Okay. So when we run with non-service mode, basically everything runs as CLI. So nothing specific. So here we still retrieve the service metadata, et cetera. But it runs basically as a common CLI application with all these commands, et cetera. With service mode, of course it becomes a bit more interesting. So here you can see that we actually start a new instance of the service tracker. What does service method does? So yeah, we start the service base. So the service base is the default API supplied by WinAPI, which defines all the service behavior callbacks because operating system will be talking to us. And let's take a look at our service wrapper. So our service wrapper is defined within the library. And it has a number of flags, but actually it needs to supply some information like conversion, for example, it needs to follow the process and other definitions for runtime. It also needs to expose the status. So this is a common logic. And here, for example, we start the service and pass service descriptor. So basically we pass configuration here. And then our service will be basically managed by Windows. And there is a lot of logic there. So move file, et cetera. This is a start-up logic where we process configurations. Where it actually starts, there is on-start method. And the on-start method, well, what it happens, we first we retrieve environment variables defined by our configuration. Then we handle file copies. So there is a way in Windows service wrapper to implement data migration. For example, if you want to update the service, if you want to change the configuration, you can define migration in the configuration file. And this file copy is actually a desktop. After that, it does null nodes. So here you can see a funny thing already that a code appears completely differently on Vnext. And basically it's .NET 4.6.1 and also .NET Core. So here we use .NET features. But for all the versions, we don't really have standard methods to download files, et cetera. So we will just do it manually using, again, code-based in the repository. So here we download all files configured by the configuration. And so on and so on. So basically Windows service will implement the hook and do this management. Same for shutdown. When it receives shutdown, it processes the hook from, again, Windows management interface. And here we just stop the service after receiving. Again, we do a lot of configuration processing, et cetera. But basically it's an intelligent version of killing the process with some of it. And here we have a super useful feature in configuration that you can beep on shutdown. And it was implemented long ago, but it's still a part of the system configuration if you need that. Okay, does it answer your question, Boudicca? Yeah, thank you. Okay. So yeah, this is the code. Actually the code is the relatively simple configuration management part. Yes, it's not this simple in all this service descriptor and callbacks, but hopefully we won't need to modify the code too much during this project. But yeah, that's the plan. Let's take a look at how it's used on the Jenkins site. So in Jenkins site, by default, in Windows service wrapper, we publish GitHub releases. We also publish versions to Nougat. But Jenkins uses additional way to retrieve the repositories. So, for example, it's strange. Okay, but here for example, we can go to our repositories and, well, to the Jenkins repositories. And here's the first thing you can discover is maybe packaging for Windows service wrapper. It's a historic artifacts because Jenkins build flows to lie on a Maiden packaging. And here we can repackage artifacts as Maiden and deploy them to Jenkins Maiden repository. So this is the repository which we actually consume within Jenkins. And basically, if you go to our artifactory, here you can see that, yeah, it's just Windows service wrapper. And here you have a number of versions. So the last read this was 2.720. And here you can see that we actually include .NET 4, .NET 4.6.1, and the empty version, which is actually .NET 2. So this is all we include right now. We don't publish .NET Core versions. But yeah, most likely we won't need that if we ever decide to do that. So where is this library consumed? We have three areas. So firstly, it's Jenkins Core itself. So Jenkins CI. Sorry. Crap, Jenkins CI, Jenkins. So in Jenkins repository, we include Windows service wrapper in order to support installing Jenkins as a service after it's started up. So here, for example, you can see that there is Windows service lifecycle. So it's a special logic which allows Jenkins to properly operate as a Windows service. So for example, here it can self-update the Windows service wrapper if needed. And for that, it uses this file migration schema which you have seen in the conflict. Also, for example, it supports restart logic. So you can restart from the web UI if you use Windows service and other such bits. So this logic exists, but the first thing that I'm not really sure that this logic is being used by anyone nowadays. So how we actually recommend installing Jenkins on Windows if you go to our download side. Jenkins is your download. Here, you can see that you can just download Windows and it offers your MSI package. So for example, here, I'm not sure what. Yeah, it offers me to download the Jenkins MSI. This MSI file is defined in packaging. Right now, we are working on a complete replacement packaging for new core-based automation fall. But in principle, it's the same. There is an installer and this installer is based on Wix. It's a Windows installer toolkit and it actually includes the same Windows service wrapper with XML configurations. So here, you can see that there is, for example, exec config file. XML file is being generated somewhere in the scripts and the bundle of Windows service wrapper as well. Any questions? Microphone. Okay. So this is Windows service. Another area when we work, we use Windows service is actually for agents. So there is a Windows agent installer repository and this repository again uses Windows service wrapper in order to install agents on Jenkins. There is some magic there, but if you go to this repository, you can see that there is XML file, which actually is a template. So here you can see that this is a template XML file. We have some code generation where we inject IDs, where we inject Java versions, where we inject arguments and also we inject agents to download URLs because Jenkins supports automatic upgrade of agent jars and of Windows service wrapper and in order to do that, we just co-generate config depending on the configurations. So for HTTPS, we enable that if your instance is exposed via HTTP, no, it's not enabled. Okay. So some updates in these components during the project in order to apply new features to Jenkins. So for example, support configuration with YAML, but it's likely a second or third code phase. So right now we do not have to worry about that. Maybe another component you may want to know about, though we definitely won't be touching that. It's my Windows agents plugin or historically Windows slaves plugin. This plugin actually allows to use Windows service management interface in order to install agents as services remotely using DICOM and other remote communication services. So you can connect to remote Windows instance. You can install Windows agent on there, connect your master and keep using it as an agent. So this tooling works pretty fine. It's widely used, but at the same time I wouldn't recommend it for modern Jenkins setups. Okay. These are four main use cases within the Jenkins project. And I believe if you just Google for Windows service wrapper in the network, you can find more uses. Again, Jenkins is just one of examples where Windows service wrapper is used. Okay. So Dika, do you have any questions you want to clarify anything? I think I have this video. Okay, I have some problems. Okay. Now, after this coordination, I have document documented this what we discussed in here. So I would like to ask you what are the aspects that I have to cover during the documentation? Let's take a look. I think almost all the things that you discussed here, I think also covered in GitHub. Yes. So that's why we have acceptance criteria in this project. So right now we need this codef session. So basically you expect to create a new developer guide, which is relatively short. It would include a review of the project structure. So basically the first thing we've been doing in this video, which would just include a link to this video, that's it. So now you are not expected to document everything we discussed today, just on top level so that it can help potential contributors to start developing the component. That's it. So I have to maintain an external document. I don't have to update it to repository or something like that. I just have to maintain an external document. I don't have to update it to markdown or anything. No, I think it should be in the repository because we prefer to follow the documentation as code approach, but it can be a new file. So for example, how I will see that. So there is top folder. Here, for example, we create a folder for developer documentation. The way you can adjust the file, project structure and view and then link it from contributing guidance. Okay. Thank you. Okay, so anything else to discuss today? I think pretty much there. Is there anything I can ask in additional? Yeah, right. Again, it's just the first overview. We didn't really dive into the configuration. Yeah, there is a lot of information to process, but we can do it incrementally. So I definitely do not expect you to digest all this information right now. We have a lot of time to discuss it. So there is no other topics again. So next time we meet we will be next Tuesday and today we can have a follow-up discussion. So if you have any questions, we can deep dive in particular areas there. I'm happy to do more sessions if you need. Thank you too. So then I will stop the recording and I will try to publish the video by the end of the day. Thank you so much. Happy to help. Thanks for working on this project. So you have already built it locally so now we can try to do some patches and see how it goes. Okay. Thank you and have a nice day.