 Great welcome to and today's by native seg meeting really happy because we have Santina today Who's joined us to discuss some cloud events work and? Welcome Russia do you want to say hi? Hello everyone. Yes. I'm super happy to be here and super happy to be a mentor again for Google summer of code I've been I was doing mentoring way back when I was working at red hand And I've been missing that for for so long and I think that just this mixed between janking sex and cloud events It's kind of like the perfect thing for for me to come back and help here So whatever I can do to help I will be doing that That is awesome. Thank you so much for being here Vibhav do you want to talk a bit actually because you're here with machine? Would you like to speak a bit about your ideas for the cloud events plug-in and how that's evolved recently in your work? So hey, thank you like joining and we definitely need like a lot of help regarding some cloud events specific Subject matter expertise and thanks for joining. So This the idea for the cloud events plug-in. We just have had like out of the blue and the first time I actually threw this idea against someone was Oleg and he said, okay, this is a great idea. Probably we should do it and then after that we brought it up again in the cloud events sake, sorry the cloud native sake and Yeah, it's good to have you so the idea for the cloud events plug-in was basically just be able to listen on cloud events Be able to trigger Jenkins jobs on cloud events and also You know be able to emit cloud events for Jenkins jobs projects or whatever events that are being created by Jenkins and kind of Like have it in the cloud events format instead of the Format that is there that there is a plug-in Plug-in which you are looking at Initially and this plug-in basically kind of helps like kind of helps with the eventing and stuff and initially we're thinking of like, you know converting those Events which you see in the statistics other plugins to like cloud events plug-in, but I Was doing some reading up yesterday and I figured I was under the impression that discovery and subscription was already Implemented for cloud events. So I don't think that's the case anymore but I was so Do you have any idea like about the discovery and subscription when it might get So started in cloud and sdk and probably like what an idea of what you might have How events should be done in Jenkins Yeah, so that's a very good question So like the subscription and discovery it's kind of like a work that it's starting So I wouldn't rely on that that much for the the scope of this project, right? So in general what comes to my mind when you mention cloud events and Jenkins is exactly what you mentioned So basically it's a plug-in that it's going to emit events whenever you run things inside Jenkins And also an endpoint that it will be exposed for people to be able to submit cloud events to Jenkins itself I think that that's pretty much what like the scope of the work should be and again when I'm saying emitting events That's usually sending a post request to a URL that you can configure As soon as you provide that we can hook that Thing with other tools that are going to be able to deal with forwarding cloud events to our systems and doing all that stuff So it is pretty simple It's pretty well a scope the only problem that you will have and that's the problem that we are also trying to solve There in the CDF is the format of the events and what events do you emit, right? We should try to avoid emitting Jenkins specific events because if you emit Jenkins specific events Then you are like tied to the Jenkins ecosystem But if you emit like a standardized events, then you can interact with other tools And that's basically what we are trying to achieve there in the CDF So my proposal to you would be to start like experimenting with creating a plug-in for Jenkins that basically emits cloud events and Then also exposing an endpoint for consuming cloud events coming from all the systems And then we can focus on the format of the events themselves, right? like you can start with a simple cloud event with Not much data making sure that you can emit these events at the right times when when you know When Jenkins trigger a job or when it finished and then figuring out what Information is available inside the jobs for being able to include that into the events You need to kind of like figure it out What's kind of like the unique identifier for all the events that you're going to use? Related to the jobs right like you will need to correlate all these events that are related to the same kind like pipeline execution and that's kind of like where the You know the details becomes a little bit more tricky that's why I would love to first can like see something like a plug-in that it's emitting an event basically and Consuming an event and then we can just go into the details of the data that we are going to include in each Different event that it's going to be emitted by Jenkins does that sounds like a good plan or it's easy to complicate it Yeah, this is actually the same thing about thinking of like starting off with emitting events like on events such as you know starting the job or Completion of a job like these would be like good places to start so the idea where I got this from was basically tech town has Support for cloud events and it kind of MS cloud events in a in that way so In the design doc you might have seen it just shared with you earlier in that design doc You you must have seen that there are these there is a table of cloud events, which are written and Fresh I'll just share it again Just in case would you like to share your screen? Yeah So this is this is basically like What keep a cable but initially like how these events We cannot see your screen we can do I cannot see your screen. No not right now Okay, right. Do you see that? No, not yet. It just says you've started but it's it's still black. It's probably Yeah, can you try to like stopping and sharing again? Yeah, I'm using another laptop right now. So it's not set up for No, Karah, do you mind helping me out with this? Sure. I'll share So there's a table at the very end, which I want to talk about a little bit. So Yeah, this this I this I this table I made based on the tecton Cloud events table and what the resource looks like the event That happens on that resource and what the event type should look like and this is the event I which would be Emitted like at every event So if if a so if a job is started the event which is emits Which is emitted the event type on that event would be okay The job is started and the data regarding that job would be given so when you say that you know those event the format should be a Bit more generic and not Jenkins specific. I I didn't understand how We could steer away from Like being generic in any good good good So we are defining kind like as part of the city foundation these kind of like generic events for this exact same reason Right, like if you do this you will meet these events tecton It's not going to be able to understand these events right because they have a different set of events So basically what we are creating is this shared language between these tools So they can exchange events and react based on different events I think that I don't want to confuse you with that right now I think that if you get this working moving to emitting all the other generic events will be kind like an easy thing to Do or maybe it can be a parallel plug-in that we can create right like it's going to be pretty similar Just the events format and the event types are going to be a little bit different But it's the same mechanism So as soon as you can meet events for now and listen to events That's something that I can see down there that it's listening for cloud events. I'm not entirely sure how are you going to do that? But as soon as you can do that That's good right like that's definitely like a starting point so It's better to so the point of this plug-in is kind of Interoperability between different things so I think if we start from the beginning to aim at You know supporting gender givens that'd be there'd be something good. So When you say that these events, they are not very they are not a generic type So what would what would a generic one look like do you have like an example that you could guess? Yeah, I can send you a link so to some pull requests that are we are trying to manage right now in the CDF That are basically defining these events, but it's basically the same thing but again for example the event type It's not going to be IO though Jenkins. It's going to be probably something CDF related, right? And probably we are not going to use the the name job for it Which is kind of like a Jenkins concept. It's going to be something like Task or step or something like that or pipeline for example, right? So it's kind of like the terminology the vocabulary and then the event types That's needs to be a little bit more generic But again as you assume my fear right like the content of the cloud events that you're emitting like the way of the Meeting events. It's going to be the same So, yeah, one of the things that I think that we can do as part of this project is also create for example The Java mappings like the Java objects that are going to represent these generic events so So then you can go to the plug-in and add that dependency to that library Which is basically just defining the structure of these events If that makes sense Can you repeat that line? The last the last part. Yeah So for example, right like this is going to be I'm guessing that this is going to be a Java object that it's going to be created inside the Jenkins plugin And then we are going to use the cloud events SDK in Java to emit the events basically, right? so As part of this project what we can do is like instead of creating like a Java object inside the Jenkins plugin itself We can create a separate jar file a library that Includes this kind like events definition and then we can just definitely plug in the generic ones whenever we have them We can also help building those generic events while we are doing this project Yes, yes, definitely most mostly because of that like the interoperability Idea there that we would love to listen for cloud events And we are not we don't want to be restricted to just only listen for these events, right for this event types So I wonder I wonder if you have already like a git repository or something that we can have discussions Or maybe create issues in there. I think that that will be useful. Yeah so we Started On the chat If you can add it to the document, that's good. I mean so I can have that document as a reference And I'll also give edit rights to you so you can Probably expand it based on what you feel like. Can I have your email ID? Yes, I will just put it here in the chat Okay, have you seen have you seen something already for listening? For cloud events inside kind like the plug-in. Is that possible exposing kind of like a rest endpoint? We haven't looked at the technical details, but it should be possible because I think the promiscuous plug-in works in a similar way If I'm not Yeah, there's also the Makes sense Yeah, it's pretty similar to that So just exposing a rest endpoint where we can listen to events and then we need to we will need to think about how to map events To do things inside Jenkins, right? But that's part of the definition I guess of the plane So what happens when I receive a cloud event? What do I do with it and what kind of events do I support and which ones are not supported for example? So when a cloud event comes by you should be able to Kind of put a cloud event trigger on a job Which is listening on a specific cloud event and once that cloud event comes by you should be able to trigger that job like if the if it is listening on something like a tecton Job completion like a task task run completion and this cloud event is The job is listening on this cloud event So the job will be triggered by that So that's good. That is like the initial idea for implementation And the and the main configuration for all this The only the question I have here is Garth maybe you can also help out with this is Where do we do we keep all the? So All the information regarding what events we should listen on like on the sources for the events in the global plug-in configuration or in the jobs this is one doubt I have because Should it be global or should it be like localized to a job? Are you talking about like a sort of a global allow list of Events to filter on Yeah, something like that That's I think if you if you wanted to do like yeah, like a like a global or only interest in these types of events Then yeah global plug-in configuration makes sense there You may also want to do on it on the job trigger as well to say Like I'm actually only interested in this particular type of event So on the job trigger, I was thinking like we could reference back to the Global plug-in configuration and choose from there Why I was thinking of doing in the global plug-in configuration is It it would be this is more of an admin thing, you know, like allowing certain cloud events to get into the system getting certain cloud events and Allowing certain projects to listen on certain cloud events. Otherwise, I Don't know. Maybe I'm overthinking the security aspects of this Of what events could be listened on what you can't cannot listen on So that's how I was thinking if we should keep it like, you know per job basis like we're on a job the user can define the source for the cloud events and Or should we do it on the global? I can see there being a use in the at some point in the future for having the ability to filter out events of a particular type If you've got a block a bad actor in the system that's ending Invalid events or something like that, you know, you may want an admin to filter something out for a period of time That might be useful, but I think that's like a yeah future thing Cara, I think you're gonna say something then Well, okay, these are just ideas But one one thing you could do if you wanted to be able to control which Events were going to be emitted from a certain pipeline. You really could write a Policy for that. I think that is actually a great use case for a policy and I see yes CD pipeline and then in terms of what Who came when you have like, okay So these events are admitted and they're put in what like a cue that people can subscribe to like an information That is that pretty much how the system so then you could you could have permissions, right on who listens to that or is there Is there not a way to filter who has access to what? Yeah, but that should be outside of Jenkins, right? So Jenkins shouldn't have control. So shit Jenkins should just admit it to an endpoint and forget about it Kind of yeah, we can then write some other components to deal with more complex things in there But I wouldn't start with that. That's yeah, that goes into more complex Go on I was just wanting I was asking about the work the event Sega's doing and have you defined Have you been looking at how to define the data formats for the events? Is that already done or is that is not any question that's working progress And I'm really pushing to merge a couple of like I have five pull requests in there Basically defining the vocabulary and there has been a lot of conversations about like the initial vocabulary I'm hoping to be able to get that marriage by Monday and Basically what will follow up from there? It's the definition of that vocabulary in cloud events terminology So at least the metadata of the events should be there and with that we can already create go and create for example like a Java model for those events and I also have like a draft implementation in go for a go library. So that's where I am right now and unfortunately, I Really need to have those pull request merged to then move forward and create the cloud events definition That's what I'm saying. It's not there yet But if we start with just sending like the cloud events that Vita Vita is proposing right now We can just move forward with that then swap the cloud events definitions later Yep, that that does sound like a good place to start and later on we could add a issue or like a story for extending the extending to CDF events Yep, I don't don't get me wrong defining the Jenkins specific events I think that it's also a valid thing to do, right? It is something valid But for interoperatability, I would just definitely go with the CDF event So maybe the plugin is doing both and you can select by configuration which events are emitted That does sound like a good idea like that also gives us like some time to kind of figure out like You know the initial bits without having to wait for the cloud event Stuff to like wait for the events CDF events to work itself out. So oh So you were saying you've implemented a initial Library in go for this one Yeah, so basically what I've created is a small two things this library that basically emits cloud events using go And I have because I've been working on the vocabulary I've already implemented some of these cloud events and then that library basically also provides you like a binary that you can use From the command line to emit the events So that's another possibility So I'm not entirely sure the limitations from like building Jenkins plugin But if you can call a binary from Jenkins plugin, we can definitely use that or we can just create the same library in Java That's that shouldn't be a problem to us. It's not a big thing Just I'm consuming. It's it's basically like probably can use it to like a debug A cloud events listener So if it's a bit emitting events, you could probably use it to Debug like a cloud and so it's not which someone is writing because initially so So I was actually asking me earlier like how should he get started on this and all I suggested that he he first like try to emit events out of the Jenkins plugins and then and Then kind of move ahead from there And I am not sure how the listening part will work because even I have to read up on how the web of trigger plug-in works and stuff So but it could probably be similar Yeah, there is there is one project in in the K native community that basically I think it's called Sock I or something like that that basically allows you to Define that project as a sync for the events and you can graphically see the events that are arriving there So basically So I I think let me find it for you. I will just look for it and base it here in the job Yeah, I've just pasted there so basically yeah, I think that it's just a simple user interface That allows you to receive cloud events and see them kind of like while they are arriving It isn't go So that might not be the best thing but But yeah, I think that if you are planning to see and try to the bug What's happening inside the plugins and see what are the events that are being emitted using something like that might help Yeah, this is this is this would actually be very nice for debugging Yeah, I'm gonna put this down in the Designed also one thing if I can mention something is there any way to get like a Jenkins instance in the cloud Where we can put this flying we can so we can spin off Jenkins probably in a mini cube or something or like While testing so when we are testing the plug-in We usually use MVN HBR and that's what I use for running the plug-in locally with a Jenkins instance and Then Jenkins runs with just a plug-in installed and you can test the plug-in from there Yeah, is Does that answer your question? Yeah, yeah, I was thinking more like on provisioning it somewhere in the cloud I'm not sure if cloud is can provide us that but Yeah, I think that like as soon as we can run it locally I think it's fine for now, but then if we want to connect different tools It might be better to run it like in a cloud provider or something, but that's fine I mean we can definitely leave that for later When we get to the point I Was just gonna say when we get to the point where we we do need that I can ask Okay, yes, that's good. Sorry. I interrupted you about No, like I was just I was just asking for debugging purposes because I was thinking like Like I've never used Jenkins directly from the cloud or like debug max I've done is like my Jenkins would run in an OpenShift cluster and we would and to test it against the OpenShift API, we would have to Load the we had to load the plug-in into the volume and then restart the Jenkins Container and then you know go ahead from there and test it. Yeah, it's it's a one-minute It's a one-minute But that one minute is long because because Jenkins takes a long time to restart so so I Recently I figured out that working locally is much easier in that way because Jenkins takes a Lot less time to start and you can you can easily iterate on So though we have enough for getting it started with that Yeah, I think so Yeah, I think this has been great I really wanted you two to meet and and to start great project. So this has been really successful. Thank you Yeah, so now you have my email feel free to write me like to my private email and Send me the links for github if you add that to the Repository if you want to have like a weekly catch up that will be good just to see how it's that going But no pressure right like you you have your time whenever you make some progress. Let me know I Love you a lot because cloud events is something that is That is very cool to see that such kind of Standardization is happening and it's it's really nice to see like different tools coming together And it's just it's great. So yes Yeah, awesome feel free to again feel free to bring me any time and I will definitely keep you updated about the Generic events progress if I make substantial progress, I will let you know and I will Try to Push you into that direction whenever those things are ready not before Pleasure thank you Kara for the introduction. Yep. Great. Thank you for being here great meeting Excellent any other outstanding questions before we go or issues topics to discuss So currently I was dealing with cloud events on how to send it through Htt server to catch up it on the another local host port so It is maybe a coding related stuff. So maybe I will Should I ask now or like it's just a coding related stuff like yeah Can I share a screen just to show that like it just a snippet of code please yeah, I think you should be able to Okay, so are you able to see my screen? Yes So like here, I made a HTTPS server. So started a server on local host 8080 and I read the doc over Java SDK So here it's making our cloud events and then attach and here it's making the HTTPL URL collection, but I don't know exactly what here this line of code doing actually and I read the doc also So I'm unable to understand completely like how this is writing what it is actually doing if you know, like Yeah, I think that's encoding the payload of the cloud event into a binary format So you don't really need to worry about that like so for example there like with data. You are including a JSON payload in there But those that's basically Codified into like a binary format when it's sent through the wire. So that's something that you need to do To avoid issues within coding Basically, so you can send the same kind like a string to a different platform If you're sending from Linux to Mac or to Windows They should all be able to get that binary thing that it's compressed and decompress that and then just get the same text That's why you're doing that Okay, if that's kind of like what the documentation said, you just need to do it and it should work like that. That's fine Okay, okay, and this is just then using that same encoding to read that is it? Yes, that's correct Okay, okay, you are not sending. Are you sending there like the the message? No, like currently just like it's from the Example basic STTPS from cloud events. So it's just like catching the local So in order to send it through STTPS connection, should I need to Serialize it with JSON and then I need to send it and then converting it back into cloud events. Is it I? Think that that's correct. Right like if you want to send it via HTTP, you need to Create so basically the cloud events SDK. It's going to give you some helpers to Create like the HTTP representation for that because some of these attributes for example with source or the idea of the cloud event It's going to go into the HTTP headers and the data It's going to be into the body of the request So as far as I remember the cloud events Java SDK They will give you these helpers to create an HTTP request that then basically you just need to send using a library Okay, so is it like so it's like kind of I'm using the Jackson that they are Showing me to deserialize and then With their helper classes Yes They were they are going to probably use Jackson for doing that. So And and just then doing a post request and just catching it on the another maybe local host 1990 from 8080 and just to then Test it to sending a post and right Yeah, that's correct Okay, okay, and then where this is using like after creating it into binary like some encoding is done into this object, right? Yes, that's correct So probably for for Jason if you're sending Jason data, you don't really need to do the right to binary But let's check the documentation. Let me check that Because it really depends on on the format that you want to use to send the payload of the cloud event If it's binary of it's just Jason Okay There it is. Um, so Here is the basic as ttps example. They are showing sending this is cloud events. Um, yeah Yeah, so if you yeah a standard client, let's just take a look at that like HTTP client with HTTP URL connection At the bottom of the page. Yeah One of these links Oh, okay, that's the same one. Okay, perfect. Yeah So the first one is basically just encoding it and just showing that you can just read Send and read the same thing, right? And if you if you scroll down, there is nothing else in there. No, right? No, that's just the the helpers Yeah, let me see if I can find it I know that I've seen some examples that just shows more advanced usage So for example this one Uh, is there any like I'm not entirely sure about Jenkins, but I'm pretty sure you cannot use like a spring boot in there, right? Or the spring library I also don't Know that there's yeah, there's there's some quite. Yeah Like restrictions on the libraries that can be used inside Jenkins for sending this but there will be something there to do it Okay In conference, they also said you can't I don't know if I heard wrong, but I'm just confirming They said you can't subscribe to a cloud events in basic studio, but you can do on a spring using spring No No, I think that you can code that definitely, right like you can definitely subscribe to not subscribe you can listen to cloud events, right? Okay That that's again, that's just exposing a normal rest endpoint It's like it's nothing specific to the cloud event because at the end of the day You're going to get an HTTP request that have a cloud event Inside it, right? So the only thing that you need to do from the HTTP request you need to parse it And put it kind of like inside the cloud event class Yeah, okay, go ahead. I I think that this for example this example here. Let me put it in the chat It's something that you might want to look into it's there as well and Basically, it's just sending the the the request to another server Using HTTP exchange And that's basically what I was mentioning before. It's just basically getting the cloud event And creating kind of like creating an hd request for it But yeah, definitely the examples are not the best to be honest Hmm And this is because it's very kind of like low-level Java. It's like Java with no libraries at all Yeah, that's why they look more complicated. Yeah, they are using like sun libraries like Okay, so go to like now I will just make a HTTP request Serialize and just trying to parse and just make a demo of it That would be great. Yes Yeah, thanks And um, you can take the um, um sharing access. I'm just figuring out how to Okay overriding. Yeah, that's fine I See are you part of the cloud event sake gitter channel by any chance The cloud event. Sorry what The cloud the cloud native sake gitter channel by any chance for Jenkins Probably not I was thinking Yeah, it'll be it'll be nice if you could join because I have I have feeling we are going to get a lot more questions Oh Yeah, yeah, that sounds good And is there is any slack channel or group community for the cloud events as we get who are actually using it in their project like integrating Um, that's a good question. So yes So there is like a channel for cloud events themselves Mm-hmm, and let me figure out. Where was that? That's That's definitely in the cncf slack Uh organization Uh, it's very specific channel for cloud uh, uh, java cloud events sdk I think that no, I think that cloud events sdk. It's the group. It's the main channel where you can ask questions And like the maintainers for the cloud events sdk's are are quite like at least the java ones They are quite responsive. So if you have any problems with the sdk itself Uh, we can definitely send a pull request and an engaging conversation there Okay That's awesome. Oh, I can see you saga there. Yeah, you are there Yeah, is that you yeah Yeah Yeah, yeah saga saga you posted some questions yesterday, you know Uh in the cloud event sdk Yeah Yeah, it's just like regarding Yeah, that's I think that you're not getting any any any answers there because the question itself. It's a little bit tricky You don't really need to do anything besides exposing A rest endpoint or opening a message queue or whatever you need to do to listen that it's not cloud event specific, right? so You can listen to cloud events in any way you want and then you need to transform whatever you're getting into a cloud event If that makes sense, hopefully that makes sense, but I totally understand if it doesn't at the beginning Yeah, it's just like passing Again, uh cloud events is a format It's not the thing like you can take data and like put in a cloud events format and pass it around It's like a river of cloud events data where you're just like pouring your juice into so it's just going as a cloud event so Yeah It it can it can be tricky to understand that's a standard standardization, but initially I think I have seen one Jason format like that's what exactly what you are saying like that's Just some new attributes to specify Folks, I know that you're in Gitter and I would join the Gitter channel at some point But I'm definitely on a slack. So if you don't see me there just ping me there in a slack I'm definitely in the cncf and in the cdf slack channels Uh, that might be the fastest way to contact me or via email, right? Like if you send me an email I'm I'm going to answer there for sure I have too many chats already But I would try definitely to join Gitter. There's also the cdf um g-soc channel That's always I mean if you want to ask your questions more publicly like that that can be helpful to other people That's a good one. Yeah, or I guess the cdf events channel Can you invite me to that channel car care? invite me to the You should I think it's open if you're in the cdf Um Yeah, I will try to join out. Okay. Are you in this? Yeah, you're in the cdf one. Okay. I am. Okay awesome Yeah, I'm there good stuff also meeting we're basically at time. Thank you all so much for being here today This was really interesting and informative So thank you Thank you very much. Thank you everyone. Let's keep talking You