 Welcome to my presentation on Jenkins machine learning plugin. I'm so excited to present my work I have done for this GSOC. And also I would like to thank my mentors Bruno, Anis, Maki, and Shivai for great feedbacks and the suggestions. And also thankful for our Jenkins org, that means that who makes this event successful. So let me introduce myself. I'm a GSOC student for machine learning plugin Jenkins for 2020. And I'm doing my major in computer science and engineering. So before we move into this technical aspects of the machine learning plugin, I would like to explain a little bit about what is machine learning operations. So if you take any machine learning operations, continuously analyze and continually training is a very essential part to improve this quality of this machine learning products. So as you see in the diagram, data scientists will analyze the data sets and then they build their model and train their model. And finally they will hand it to the DevOps engineers and they will test and deploy and they will do the rest of production works using the application code. So let me explain about how it's possible, how they are achieving using this, using the plugins of the Jenkins in the ML operations. So if you take one machine learning operations, there must be multiple steps like training, preprocessing, and validation. So there's a lot of intermediate steps out there. So for the clustering tools, you can make it possible to use one of those Docker plugin or a Kubernetes plugin. So there's a lots of clustering tools available in Jenkins. And also if you want to manage your source codes, you have a JIT plugin. And if you have a large size, large file size, like large data sets, large models, you have to manage that also. So Jenkins has some advanced plugins like S3 Publishers. You can easily publish your models or data sets to this AWS S3. And if you want to analyze, if you want to submit your analysis and if you want to publish your reports in somewhere, you have some analytic plugins also available in the Jenkins. And the Jupyter notebook is widely used by data scientists to analysis and training models. So in addition, we are giving this Jenkins machine learning plugin to leverage the older plugins to make the ML operations successful. So let me explain what are the features we have implemented during this years of period. So the first one is very data science community friendly configuration about the kernels. So if you want to check what are the kernels you have installed in your system, you can just go to your terminal or your command prompt. You can just type Jupyter kernel spec list. So it will display all the kernels you have installed in your system. So I will explain a little more when we're doing the demo, how to do that. And let me pass to the next slide. And we have this wrapper included in our plugin that will help for converting your Python or converting your notebooks to the Python or JSON. Or if you want to rename this notebook that you can do with this wrapper, you can plug in. And this is our main feature of this plugin. So we have to give the ML task name which will be helpful if you have any artifacts like images, HTML generated by the notebook. It will be saved under the name. Like for example, it will be saved under the training folder. So it's the purpose of this machine learning task name. And the next one is you have to select one of kernels you have configured in the kernel configuration. So for example, I have selected for this short as a JavaScript. And you can give this code or if you can write the code directly or you can give the notebooks name or a script name in the file cluster. So finally, we have a feature like if you have any images or HTML generated in the Jupyter notebook, it will be saved under workspace. So there's action in the left panel. You can see image and HTML. So you can click it and you can view what are the images generated in the Jupyter notebook. So we will see very detailed in the demo. So yeah, I will call the demo directly. So before we go in demo, I have GitHub repository which contains some CSV data sets and some Python enabled notebooks and R enabled notebooks. So for this demo, I'm going to run this Python enabled notebook and R notebook. So you can see here, this is a R notebook. You can see they have used R codes here. So there's a POR code. So they have using a GTP plot to make the graphs in this notebook. And the other one is the, it's using the Python code directly. You can see that. So for graphing, they are using the math plot library. So yeah. So first we're going to create, we are going to configure our kernels here. So you can go to configure system and then you can find kernel configurations here. So we can just add new kernel here. And I will tell you how to find what are the kernels have been installed in your system. You can just type Jupyter kernel spec list. So these are the list of kernels that I have installed in my system. So I just choosing Python 3 and R. So I just configuring Python 3 language. I just type Python. So the launching time mode, I just setting as default 10 seconds. So it takes time to connect. The launching time mode is how long it would take to connect to this kernel from the Jenkins. And the max results is how many lines can be sent back to these Jenkins from the kernel. So you can here have a test connection button. You can test your kernels from here. So I am adding also a new kernel for IR. Like we have seen that there is IR kernel. So for this R, so I just setting this 100 and I am testing this kernel here. So yeah, we are going to go with this kernel configurations. So we are going to create a simple freestyle job for this demo. So for that, I am going to check out my repository here. And for building these notebooks, this is the essential part of this plugin. You have to give if you want, if it's training, you can just add training. And you can select either one of these. So we are going to run here, Python NMB notebook. So I just selecting my Python notebook, the name. So I just giving the name of the Python notebook because it is already exists. It has, it will be checked out from the GDAP to Jenkins Workspace. So you can directly use the name of this notebook. And also there's, I will add another Builder for IR. So I just and then select this IR. So for the file, I just giving the name of our notebooks. Yeah. So yeah, we are going to go with this job configuration. So we just one step left. I will run that. So here you can see this, this is the checkout console. And you can see there's HTML added to the training folder. And there's lots of images have been added to this training. So if you want to check whether it is added to this workspace, you can just check here. And the generated images and the other products are here. So you can even try to view your HTML generated in the notebook. Yeah. So if you want to publish these results to somewhere, you can use the HTML publisher plugin. You know, you can use, you can leverage the other plugins to make the result viewable in somewhere. Yeah. So yeah. So that's the demo part. I will shifting to this presentation. So this is the link to our GDAP repository. And if you feel any issues about our plugin, I forgot to say that we have released our plugin version 1.0.1. You can directly download from the main update center. So if you, if any of you are using, if you feel any issues, if you feel any errors, you can report it here. We are happy to help you. And if you have any questions, you can ask directly in our GDAP channel dedicated 4G sub machine learning project. And this is my demo repository. You can use it if you have, if you want to try the demo I have shown before. And I will show you, I will show you what are the kernels available in the Jupiter kernel project. So you can see if you want to run a Julia code, if you want to run Haskell, Haskell, if you want to run Ruby, you can just install these kernels in your system. And you have to configure that in your Jenkins and you can directly use that. So that's the best part of this machine learning plugin. And yeah. So for future improvements, we are trying to implement, we will try to implement the issue like at this moment, we have to give this the kernel features and we have to type it manually. So if we implemented this automatically adding a label to the agent based on the kernel features, we don't have to type that we can have a dropdown list or something like that, or we can leverage that thing to easily configure our kernels. So that's the thing we are concerning right now. And and we are also struggling when we are testing this code, testing this plugin because everything is in the back, there's lots of Python work. So we are also expecting to write more tests in the plugin. And in ML operations, we have to, we have to concern about the performance of the plugin because performance is one of the essential thing in the ML operations. So that's the thing we are concerning right now. And yeah. So thank you once again, thank you for joining with with this presentation. And thanks for my mentors. And thanks for Martin to hosting this presentation. Thank you, Logi. Do we have comments or questions? Yeah. Not that they see so far in the chat. So if anyone has any questions, please ask using Zoom Q&A, and we will ask presenters. Meanwhile, while we wait for questions, if mentors or other panelists, so would like to give some comments, please be strong. Oleg, if I may comment, this is Yanis Mujachis. I was one of the mentors. And, you know, we have to work now twice last year and this year and finally we succeeded. And I think he has progress quite well. Logi was a very excellent student and he was able to bring together a lot of technologies that are required for this particular plugin, which is a little bit challenging, but, you know, he rose to the challenge. And I think we have a really nice version one. Oh, now that a lot of us really, you know, as users now are excited and waiting to use. So I am really thankful to the GSOC program for making this a reality. And I think we're moving into a great area with new Jenkins applications. Thanks very much. And I mean, it's fine. I also like to speak a couple of words for Logi because, you know, it has been really amazing, you know, working with Logi on this project because, you know, first of all, working on it from scratch, you know, and Logi, from the start, you know, he was always up to the task and always completing all of the task before, you know, the deadlines. And that was, you know, probably the most, you know, the best, you know, work that we saw was, you know, always making, maintaining that consistency and also coming up with, you know, great solutions for solving all the different kind of issues that we run into with the plugin. And they were also instances, you know, where we did take in some, you know, suggestions from external people as well. And those are also implemented. And probably, you know, like one of the best features at the end was to add multi-language support, you know, not just Python, but also the other kind of, you know, data science languages that people will be using. A lot of them we are using are Julia. So having that, you know, integration as well within this plugin makes it so accessible for all different kind of, you know, data science-related workflows, right? So it was, you know, like a treat to be working with Logi and all the other, you know, mentors as well, both ionists, Marky in the starting and, you know, Bruno. So, I mean, in all, you know, Logi made it completely, you know, successful and I'm like so proud of, you know, whatever work he has done for the plugin. Yeah. Yeah. Thank you, Shivay. Thanks for your compliments. Thank you. I would just like to echo what everybody else has said and also just say, Logi, you're an amazing person. You not only put together a very futuristic plugin here, but you did it during a pandemic, which is no, you know, no, that's a pretty big deal. So thank you and thank you to all the other mentors. Great. Thanks for all your great comments, everybody. And let's move on.