 All right, and now it's online um so I'm uh sorry I'm a little late today um I had a busy weekend uh which we can talk about later on a not recorded channel um So today is the uh for better or worse it is the theoretical end of milestone one and as as you both know we're not there yet um not exactly not exactly so let me let me share my screen uh google hangouts would let me share my screen that would be great it doesn't look like it wants to let me do I can take over I think if needed just tell me yeah why don't you share your screen opening uh do you want to open as you post the the board and the uh the github repo that's what I had opened okay so I can share the whole screen just not the window so normally you should see my screen now yeah you can see that so and I had also opened that um so we go to the evergreen repository yeah and a PR or something no just uh down in the read me I had an outline of what milestone one was and if you're right in hands now I guess now I get what you wanted to show yeah we zoom into that um perfect it's huge all right um so I sense uh or I wrote a blog post last Friday for in my time and I don't know if the t-store um probably you had seen it I'm actually given you know given how much work we've been doing and how we had anticipated having one more person working on this a person yet to be hired at clubbies um I'm actually quite pleased with the progress that we've made because we've got we've got sort of the basics package we don't have them all connected and working correctly um we've got the design of the error telemetry um on the client side and based on your comments in in Gitter Batiste the server side stuff is what you're working on now so uh yeah I guess I'll give status just after or you want me to talk a bit more in there that's fine no we'll get a diss status a little bit um we're really the the pre-baking of configuration that's that's there in some basic level um and then the updates and the version stuff that's working in progress and I think that's I think it's moving along well um we've got a I think a decent design out there I think the client is completely ready for um for those updates to to start being applied from the the snapshotting system but I think we're very close that's what I'm getting at I think it's um tying up some loose ends um I guess we have the 80% of work done which means it's that last 20% that's going to take us exactly why are you right why are things together how can how can how hard can it be um um so I guess uh if you want to bounce over to the uh to the JIRA board we might as well go through some of some of the stuff that's that's going on do you want to start out Ro maybe I'm going to show the closed ones Ro yeah so for my part you know I've been working on the developer workflow oh uh I'm Jenkins 50540 and Jenkins 5061 the P the P request is already there uh just waiting for March this but I can't put that on on review there is one P request that adds the runway J another one that runs the PCT uh just the runs don't use get custom word package and get 305 there are another ticket for that when I can do that with um this is all happening within the get plugin repository right now correct yeah okay no sorry go ahead um I guess one of the the questions that I wanted to ask uh while you pull something up um is this tooling far enough along so where Baptiste could pull it into another plugin like the build trigger batch plugin for example or is it not ready for that yet well it's ready to get the plugin but it's not to get the complete workflow we want I mean this this is not getting the full latest released version of essentials and using the custom word packages to generate a custom word file with all the latest versions in a deterministic way to run the P and the PCT so there is a previous step to what is already there which is being able to be a custom word with all the decided versions in another deterministic way because now it's uh one in the 88 within the update yeah right um on the custom word packages side we have uh tasks uh to integrate with GEP 305 I mean deployment of incrementals I hope to handle it within the next two weeks so that we will have tooling for both deployment and for using uh this releases and custom word packages but yeah the main thing is we depend on that um all plugins used in essentials need to be updated to the latest parent form so the changes created by jesse get applied without that it doesn't really make sense to even try a custom word packages now because it just doesn't pick the versions you would use is that is that a requirement for all of the top level dependencies or all of the transitive dependencies and plugins as well all transitive dependencies as well so yeah if you want to ship all latest you need to modify a parent form for everything so I just suggest you did the new release of plugin form on Friday and it needs to be in every plugin well there are other changes it's not just the parent form but well I mean it depends on what you're actually trying to accomplish so if you switch metadata to incrementals format that means you're able to do publish binaries from individual commits but whether those are actually used by a particular test suite depends on how that test suite is configured um so I didn't see anything get plugin pr for example that would directly use that let me ask this question a slightly different way if I wanted to take the latest for what is arguably called Jenkins pipeline that suite of plugins right um what work do we need to do to get to where um we could consume that latest well if you just a need the latest uh from the master branch you or from whatever other branch you can do it now in custom work packaging you just define this latest branch as a source the problem that this latest versions won't be tracked and won't be published to incremental releases so that you won't have a traceability so what you actually test well like that's the the custom work package or stuff is not what I'm getting at it's really that like for me it's some the the latest has to be in the incrementals repository in order for us to ship it um yeah and so I'm just I'm trying to get a sense of what are the what's the outstanding worker like what are the tickets to where pipeline for example we could say is fully published in the incrementals repository for consumption and testing of course well in that case I mean all of those plugin repositories would need to be switched to the new metadata format um there's also the uh there's also a problem that currently we're only able to publish binaries from pull requests not from right um which is not critical I think since you'd have to at least prepare that work in a pull request to even see if it isn't broken um but then we would need to so if you wanted it for example if you wanted the yaml file in the evergreen repository to pin particular versions of those plugins then you could do so as soon as all of those plugins had their metadata appropriately edited you know when you refer to the metadata are you referring to the repo permissions or uh that's part of it but then in the plugin repository itself it needs it needs a new parent palm and then it needs a few edits to the palm and some associated files so there's the little developer's guide created as part of the incrementals work that gives the steps you have to take to switch it over okay um but we don't currently have a tool for I guess looking up the expected version number um the expected version number so well so if you wanted a tool to say attach the yaml file in the evergreen repository to be the currently latest commit and master for all of those plugins we don't yet have a tool to look that up are you referring to the to the email I sent on Friday to the dev list uh no I haven't read that yet well it sounds like it sounds like we might be on a similar wavelength okay well I'll read that and that might be a task you'd have to put into the essential into the um incrementals epic yeah but uh once uh maybe have an update center for incrementals then we can pull it using the existing tooling I don't know I have no plans on pointing the update center generation towards that incremental repository okay go ahead um and just just for your background only part of the reason that I'm fairly strongly against that is we had significant I would say load challenges that were difficult to diagnose in Artifactory because of how heavily how heavily um uh the the churn was in the releases repository and Artifactory's indexing was taking uh quite a long time we uncovered issues in Artifactory that fortunately JFrog had fixed um but it took us probably four weeks to get that done because it's sort of a black box if something goes wrong yeah I remember the story I mean so we don't want to repeat it in the incrementals yes yeah I mean for um yeah for incrementals I don't see any reason to use the Artifactory index anyway I mean part of the point is to get away from using the Artifactory index right well so Jesse maybe after this um just to keep us moving along we can follow up I just want to make sure I've got the appropriate scope of if I were to start take like the the test automation work is going on the get plugin and that's moving along well um if I wanted to start taking incrementals and of blue ocean and pipeline for example those I think are going to have the biggest impact for for the teams working on those if we can start to consume those more rapidly um just what's the delta in work to where we could reasonably um pull those in that's all I want to get scoped out so I have an idea of what I can maybe beg Andrew to work oh yeah yeah maybe I should sync up with Andrew about that um I mean there's um there's also a conversation we'll have with the pipeline team about what jinking baselines are appropriate but that's a little bit of orthogonal I think because you can do incrementals even when you're not picking up an incremental version of core even when you're using old LTS but what I'm thinking is if um if Raul and Oleg have some there's some connections to be made between what Raul's working on and the customer package or stuff that uh that Oleg's been working on well that's going on maybe you um Andrew and I can bridge the gap for pipeline to sort of get pipeline ready to be consumed and tested um and that may I think that'll reduce some of the wall clock time for getting this work done yeah what I would like to see very roughly is is some simple tool you could you could run where you have a if you have a checkout of evergreen then you could just say run this tool and it would inspect all of the repositories that go into it and propose updates for all of the plugins that are listed there um updates you mean github like parent pom etc no that it would say okay um evergreen is currently bundling you know such and such uh but I think master branch now has such and such commit you're you know applied yes yeah you should read the email I sent on Friday and then we can talk about that because we're 100% on the same we've like that never happened is it an infrastructure and they developed our mailing list it's on the developer mailing list it was just a follow-up to the bill of materials thread that Carlos had started okay oh okay just failed to read it for sure um I I sent it Friday afternoon my time so oh was it just a response to the thread from Carlos yes oh that one I read okay um so we got Raul um Baptiste you want to go tell us what's up yeah so basically last week we we there was many PR many the JEPs ongoing and many things going on evergreen but on my side I mainly worked on the client side I think it was mergerally last week the JEP for the client side of pushing the error logging and since then but I've been out a bit I've been working on the server side barring others other issues on my on my side learning not js I most most I would say don niche because now I think by ish I mean that it kind of works now but I know I need to make it works for real with with checks and so and I think because right now I can push whatever I want so I think overall and you know over time I think we should you know begin to be more picky about what is acceptable on the server side thing pushes oh by the way I didn't write I don't start writing the any kind of JEP for the server side I guess we should do that or do you think for the server side it's maybe more an IEP no no no I definitely read a JEP the IEP stuff is for you know stuff that you would be writing in my puppet or terraform or in any case it's not in the wrong order in any case because I still need to prototype and so on to have some matter for for filing any kind of JEP so anyway it's the right direction um and I need probably some think sync with you Tyler indeed on the uh how do we make it testable and hackable on uh locally on a developer machine on the server side yeah exactly and kind of related related to what you said in your introduction today we need to start wiring things together uh so that it actually uses things locally even when developing a nevergreen which now is mostly mocked so well yeah that's it um with the with the server side of that error telemetry api have you given much thought to um how developers should consume these errors or how we should be where we should be sending them next after we receive them uh not lot not enough I think I yeah but I was thinking for instance should we have different endpoints or only one for error logging you know and use a different let's say type field to push I don't know things from Jenkins things like the evergreen client things like I don't know the system log uh well there's not really system logs in the docker container but anything new for instance if we introduce in the future any kind of reverse proxy for instance we could push logs from from it error logs from from from there uh or you know many endpoints as I said so I'm not sure yet and then we would probably process those in a different way because they wouldn't they wouldn't be pushing the same exact same JSON or something but yeah not enough yet to to dig on okay um the reason I bring that up is uh early early on when I was prototyping some of uh evergreen out I was planning on just shooting uh errors straight from the client to sentry uh sentry.io and one of the things that I've been debating whether we should do is we still want to own that endpoint like we want error logs to come into into the back and services um but for ease of use it might make sense for us to then relay those to a service like sentry. Yeah so for your information you've written that in in uh in some of the tickets saying we should end on the endpoint indeed and possibly later or now I don't know yet send things outside for instance uh what I think we discussed some time ago maybe also the GC logging for instance uh we could push that to some external service where some experts for instance and they would analyze and you'll be be able to trigger some analysis and so on so yeah that's can be interesting to discuss um will you do me a favor and will you um maybe set up some time to talk with Olivier about some of the error logging work that he had done originally when I he started working on Jenkins infrastructure we looked at some log aggregation for infrastructure services um and just talk to him about what we have right now in our kubernetes infrastructure for uh handling logs from services we might have something we can take advantage of there right um okay and I guess I'm I'm last I think yeah um so the stuff that uh most of the stuff that I was working on last week as as I hope you all saw was some of the update level um processes and how are you gonna incorporate updates to the actual instances um some of that stuff I I have um wired on the server side so the the update levels the data is all being shuffled around correctly what I started to get into last Friday which is what prompted the email to the bill of materials thread was how I'm gonna actually like there's two integration points there's Essentials YAML has to turn into updates in the in the database and then the client has to consume those in some reasonable way and so while I while I wait to figure out some of the versioning or some of the um tooling stuff as as we were alluding to earlier Jesse I'm just gonna wrap up the client side of um storing the update level and making sure that we have uh the right information there but I think honestly I think assuming that the tooling isn't going to be overly complex and doesn't take more than a day to get something basic working um I think at least on somebody's laptop we can be shipping updates this week because I think all the pieces are there famous last word sizzling this week will be the week for sure yeah yeah actually regarding tooling I'll follow up on the thread because I think we can uh get benefit from some stories and maybe work around uh the need to update each plugin so I will follow up with Jesse to understand the incremental release more we have a story for deployment to incremental releases from custom work packages and actually if it works we may close the gap at least for the hub for the story without updating plugins but no warranty that it can really work and to understand it I have to talk to you with Jesse right um so I don't think that there's anything that I'm working on that I'm I'm blocked on except for the discussion around the tooling with with Jesse Baptiste is there anything that I'm blocking you on not really but I think you know much more than me on the uh Node.js and uh feather.js so yeah if you if you can possibly either show me how you're doing it already or how to wear things you know locally to be able to test things and maybe write you know as I said in on the chat kind of end users uh kind of ask acceptance testing I would say um from you know triggering the actual instance uh in Jenkins uh in the container like we do now and check that uh the for instance the error logging service do or service does receive the the warning the warnings and so on for instance we um will you add a ticket into this milestone about the integration testing and just incorporating some basic level of integration testing yep yep not noted um yeah and in in the like in the next hour or two I have to I have to go to an appointment so we could jump on a hangout later today or early tomorrow morning my time later today it works for me okay roller are there any things that um we're blocking one or I can help with one is it seems there are no windows nodes and shell code can't control that I mean my you're supposed to read this second block I can't really understand what you're saying you say that there are no more windows agents on ti.txt.io is it not provisioning them again I don't know what is happening I only know you're supposed to be plugging this stale since yesterday I mean it's trying to find a windows node honestly there is no windows node yeah this looks like a this is a this looks like a familiar problem in our infrastructure so we'll take a look at this after after the meeting other than that uh person does just a general comment if we are going to bring the flow to more plugins and it will be great a way to test the engine files on the pure request because at this moment I'm just appearing the pure request to my own plugin where I am a trusted source so maintainer can see working but this doesn't escape so it's not blocking it's not very important at this moment but maybe it's something to think about um yeah so tomorrow uh Raul if you ping uh Batiste he's the maintain the reason that I mentioned the build trigger badge is he's the maintainer of it and it's a very small and simple plugin to where if you wanted to test some of the the PCT or ATH work that you're doing with that it might be a much smaller surface area than the git plugin um if you wanted to compare and contrast no I have I have my own plugin for oh right yeah you've got the status notify plugin yeah that's true so but maybe a bit strange for some maintainers to see you know they would like to see but that is not blocking anyone so that's something for the future yeah I have created some demo pipeline unit things for gington's cintro pipeline library when I was working on my conference talks in December but I have not even created full requests for that so maybe at some point yeah yeah can you send me the link to that or yeah I can try to find it it's somewhere in my branches okay don't worry I'll be granting the two of you access to merge to the pipeline library um I'm fine with sharing maintenance of that um and then you two should be able to do some topic branches for more testing on cijang and sayo okay perfect and now you know who will be fixing color stuff that's nice now I'm going to be able to break another master right all right with that I don't know if there's anything else that we need to discuss for today okay I didn't want to to put you the thread but it's the same in France tomorrow really another holiday Francis killing me public holiday yeah so you're the only ones who work tomorrow I'm working half days today and tomorrow Europeans plus east gasters let's see it's the chivalrous labor day we don't work go figure okay well I guess I'll uh I'll be very lonely tomorrow and I'll look forward to to sing you all on Wednesday see you later today for me we can think about what we said okay sounds good I'll see you later bye