 Bonjour tout le monde, bienvenue à la meeting d'infrastructure week-learn de Jean-Kinz. Aujourd'hui, nous sommes le 21 mars 2023. Sur le table, nous avons moi-même Damien du Portal, nous n'avons pas Hervé. Marc Waite, Stéphane Merle, Bruno Barton et Kevin Martins sont là. Annoncements, nous allons commencer avec aujourd'hui. Je n'ai pas encore checké. La bague a été placée environ une heure auparavant. Le changement final n'a pas été posté, mais Kevin a créé les updates sur le draft. Pour le faire préparer pour le final. Je ne sais pas quand ça va être préparé, parce que la dernière fois que j'ai checké ci.jankinz.io, c'était encore en train de faire un update sur la sécurité. Ok. C'est en revanche. Oh, merci. C'est très bien. Ok, très bien. La délice du conseil de sécurité. Ça devrait être bien, pour les prochaines heures. Je n'ai pas checké les portes de baguette, mais j'assume que ce sera rapidement là. Il n'y a pas d'issue dans le processus de relance. Je n'ai pas checké les loges et je n'ai pas accès à la VPN. Il n'y a pas de mal. On peut croire que Alex Brandes et les autres ne le prennent pas. Ok. Oh, quelqu'un est regardé. C'est ouvert. C'est bien. On ne le prend pas. C'est bien, c'est bien. Il y a un point de vue. On a l'air de voir un point de vue. Il n'y a pas de mal, mais c'est bien. C'est bien, c'est bien. On a l'air de voir un point de vue. Il n'y a pas de mal. Il n'y a pas de mal. Ok, on est à l'adviserie de l'Adviserie de Jean-Kinz, qui a été élevé aujourd'hui. Security advisory today, today on plugins. So as we said CIGEN Kinsaiyo is back. The advisory has been published on Gen Kinsaiyo a few minutes ago. So everything looks good, I haven't seen any error. So please update your instances and your plugins. Upcoming calendar next week will be, the next weekly will be 28 of March as usual. I don't remember the version, do I've lost track? 2.378. Or you say the upcoming LTS. 387. Oh, next weekly is 397. 97. Ok, right. Next LTS will be 387.2. And that will be on April 5. April, ok. And worth noting there the release candidate may already be available or soon to arrive. Chris Stern is release lead. So we had security release today. Next major events. The scale is finished. Mark is back home safely after visiting Los Angeles. So next really is probably CD con in May. Oh no, DevOps France. DevOps France April 12 through 14. And then CD con in May. Ok, parfait. Perfect, perfect. Is there any other announcement calendar events that you folks want to talk about? No. Ok, let's proceed with the work that we were able to finish during the past milestone. So first of all, a public apology. Stefan and I have let the poor airway alone last week because we were in holidays. And we absolutely gave no reminder to anyone. So you might have been a bit alone last week and I'm sincerely sorry for that airway. Next time. So self-improvement for the person going in holidays in particular them at the portal. Please really appreciate it. So at least raise the topic one week before doing the infra weekly meeting. I usually brag about being on holidays every day a month before. So usually everybody knows. That's like my birthday. I remind people all the time. So next week. We were able to update the VPN. So we were able to update the VPN. So next week. We were able to update the VPN certificate revocation list. It's deployed and it's okay. We received the calendar alert. The new calendar alert for the next version in six months has been placed. The issue is closed and everything is working as expected. No issue. That's a usual task. The credential for third.ci Allowing it to spin virtual machine as Azure agents. Were expired and we had to finish the work related to that topic. The last miles were being able to properly define a full terraform as code automation. So we now have a credential with an explicit expiration date in the terraform code, which is publicly available. So we should be able to monitor that. I don't know if you will have time for that. But at least it's visible publicly. Also on that issue. I've mentioned a topic that has been blocked by numerous people including Harvey and Tim Yakom. I've created a new issue for that. The goal is to use workload identity management inside Azure in order to avoid having to manage credential. So the issue that has been created with no milestone, we won't work on that right now. We'll just remove that password with the whole expiration and rotation and calendar event, etc. And instead that will be automatically managed by the cloud. That will be a nice improvement, but that wasn't the case on the issue. The issue was about renewing the certificates and the improvement was putting it as code. So we can track it. We have one year for that now. I've closed the issue related to Maven 17 because the initial issue was closed. One of the major changes we did two weeks ago was to increase the capacity on CI Jenkins. There were the hypotheses of splitting the workload on different node pools for the BOM builds and the other builds. Alas, right now, since today until the end of the month, we have removed a lot of AWS kind of agents because we went clearly above the billing limits from the AWS account. And adding two different BOM node pools is not right now something we should work on because we don't have the capacity. We had to decrease the capacity. So what we did is to ensure that the Maven 17 agents might be waiting, but they should not be hanging or not working for hours. Just a note, given the decreasing capacity and the risk here, we might need to do a drastic reduction on the way BOM, the BOM builds are done. We might need to add a lock system to ensure that only one BOM build is done at a time, whether it's on the main branch or on pull requests. So that has to be checked and studied during the upcoming weeks. Right now, we'll see the shift, the workload shift was done on Digital Assign where we have credits. So that's why we closed the issue. Unless someone object, we can at any moment reopen the issue, but if we reopen it, that will be a isolated topic. Yes. That's not the same problem to solve. That was an idea and that will be an improvement, but no need to keep the issue open. Agreed. And if we need to use lockable resources or something like that to say that we want only one BOM build running at a time, I think that's perfectly fine also. It certainly is, when a flurry of pull requests arrive from Dependabot for the BOM, it can certainly be overwhelming to see a thousand plus jobs in the queue and 200 machines allocated on ci.jankins.io. Next step. So that one, Stefan, under a new, so same problem as third CI, but in that case, it's for our Packer job. We need Packer to be able to spin Azure Virtual Machine to build our agent templates. So you need a credential for that. Now that credential has been defined as code with the same pattern as third CI. So next time it will expire, same problem. It has been renewed for one year, calendar, etc. And we could study in the future using workload identity management for the same reason. The main difference between both is the set of permissions required for both. There are subtly different. In the case of Packer, the main challenge is that we use Packer with the default Azure Mod, which was creating Azure Resource Group each time it was building. So it creates an ephemeral resource group, puts everything on that build inside that one and remove it at the end of the build. Now, we have restricted the credential because being able to create and delete resource groups at the whole subscription level was risky. So now we have one resource group where Packer put everything within. It's a bit less practical for the Packer process itself. However, that ensures that Packer only have writing access inside that resource group. So Packer is no longer able to reach the rest of the infrastructure on Azure. And we add to fine tune Packer and the credential for that. PK Geology and SSL Certificates. So in the long series of consequence of my mistakes by updating Python 3 on that machine, that one is another one on the list that has been solved. Thanks Mark for you certificate monitoring. We fixed by adding the missing PIP package. Yeah, but without your monitoring we would have waited for the exploration that would have been worse. Clearly, we have a room for improvement on the infrastructure team monitoring. We didn't have time or maybe the priority on that's both our reasons. But now it has been expired and it work and it was. The positive consequences that a bunch of SSL certificate were updated all over the platform showing that the Let's Encrypt system is still working as expected. What do we have? We have one issue that I closed after generating the notes for that. I forgot to post the message. So one of the plugin developer had one issue when releasing their plugin they retried a few hours later and it worked. The reason has been identified the repository permission updated job that run on trusted CI run every three hours and the TTL applications used to publish plugins is five hours. So every time the build run successfully at the end it regenerates the tokens. So if you have one failure then you will have one hour during while you cannot release. So the problem of the user is solved and we have tracked it. So I took on me on closing the issue and opening a pull request on the repository permission of data which aims to retry the steps at least one time. Because these failures are most of the time, at least on the path months, due to a GitHub API error. That's a five or four or five or three server side error. Alex is ok on that. Daniel suggested that we might improve the native Java or Groovy code of the repository permission of data jobs instead of the pipeline. But no objection on both. I understand one can be better but I need help on the second one. I'm not feeling at ease writing Java code for doing so. And I don't want to spend time on that as much. So we have a pull request with the request for enhancement. The usual problem is ok. So issue closed. Unless someone object, of course. No question. Issues that has been closed with no work on it or at least only analysis. Sounds like there was an issue on the docker up description of the Jenkins in BodeAgent image that Alex cooked. But someone fixed it or maybe mixed up. We don't really know but we checked earlier today and it sounds like it's gone. So not sure. I wasn't there every day so someone already might have fixed that and we missed it. But Alex closed the issue saying it's ok now. We had 1, 2, 3, 4 account issues most of the time as people doesn't answer. Thanks survey for managing all of them. No answer 1, 2 or 3 weeks afterwards. So my proposal is we close them and the user can feel free to reopen them if they need. 1 of the 4-1 was flagged as a sensitive a few weeks ago because it's someone asking for the Ops Genie plugin takeover. So we asked them to contact internally in Atlasian. They said they will and never answer back. So maybe it takes time or it's not a priority. Or it wasn't a legit request that can happen as well. Now work in progress. I'm taking them on the order of the list here. Could not create account. Someone had issues with the account. So we tried reseting the person. The person says they've been blocked by the anti-spam system. But the anti-spam system always throw a java stack in the logs. Neither airway or high were able to find this stack so it's worth. But we have more and more of this user. Not sure if there is is there something word running with account app or these people are trying takeover. Not sure. Maybe it's chat GPT for experiments. I don't know. Anyway, in that case, try to reset the password and the person will be guided to send us an email to the new email. We will check by asking an answer by email check the full back and forth exchange to see if that person is a human and is not trying to takeover. If it's not the case, then we can reset the email in the account directly. Main issue today we worked on EC2 that started as a lot of EC2 agent on CI agent, Kinsayo marked as broken states on the left column. It looks like there has been numerous issues with the EC2 plugin. Starting with idle termination time, the time where the garbage collector for these agents it has an amount of time before passing and checking again. It was a 30 minutes by default when GCask wasn't specifying any value which we didn't because we used to have something as soon as it was used on the job finish we used to have agent being deleted immediately. I don't know when it started but it started to be 30 minutes. So once it's finished you have to wait 30 minutes before something pass and worst case check if the agent gets deleted or not. That leads to a lot of broken states mostly because spot instances but not only which is still something we can't explain here. But the consequence is that the AWS account went clearly above the budgets. So we did a drastic action by removing any kind of EC2 agent and decreasing the availability. We still have to switch fully infrascii from AWS virtual machine to Azure virtual machine that one will be definitive once done. So we should be able to leverage the amount of billing for this month and we should be able back to normal next month or maybe not. That will be discussed in April. But right now we absolutely need to avoid crossing the 16K top limit. Any question on this? So likewise the previous one they will move automatically to the new milestone because that means we still have work to do on this one. Ma event central package is not found on BuildAge I close that one too quickly I see you reopened it. Thanks for taking care of that. What is the status of this one? Sounds like you're muted. Is it on me? Sorry. Not muted. My problem maybe. Too many lines maybe. I don't know how to resolve it. I'm still in the same as 2 weeks ago. I need your help on this. Sorry, I didn't hear you need I need help on this. Ok, can spend some time on this after the meeting if you're ok. So we keep continue working on this one so that sounds good for you. Thanks. With that issue next issue is from James Nord about agent instabilities on CIG and Kinsayo. I can't grasp how to efficiently use Datadog. None of us were able to find matrix, but the matrix should be collected. So maybe they have been deleted after time window we need to fine tune. But a lot of information on that issue to say one of the builds failed with an error message. Clearly it's not OOM kill because we didn't see any 137 code and none of the usual, these kind of events are shown on the AWS or Azure console. In that case it was on AWS and no alert about pod or container being OOM killed. So that mean that the pod there was still an error. Assumption is that the pod was killed. So the machine could have been removed or maybe the spot instance was deleted and removed and there might be an issue in the ability to retry or to catch the issue. This morning we checked this is a plugin and the build plugin function use retry, the retry pipeline function and it should receive from Kubernetes when there is an event about the pod being deleted and not being drained or something exhausted in terms of resources. We didn't see anything and any retry kind on that job so that sounds like an edge case. Jams build was finally rebuilt two or three times and the pull request was merged. So we don't know what happened and we were unable to observe. Thing is the timeline was during the 2000 builds of the bomb that was on that gray area so gut feeling and only gut feeling I have no formal proof and I don't know but I don't have any way to prove where does it come from. So yeah not sure what to do with this one but any idea welcome there I don't know how to go further here because a lot of hypothesis and assumption but nothing factual or miserable here right. So my proposal is I asked Jams if he was able to to see that problem on another build or another plugin and the last thing for us to do is to check the spot history inside the AWS console. We have a history so we can see if some spot instances were reclaimed on that timeline that will explain the problem. So if it's okay for you we'll move that one in the next in the next milestone and if we don't have any answer back from Jams next week then we'll just close the issue explaining way. Unless someone else want to keep it all hot details. No objection. Okay. We have as your credential for CI Jenkins.io was due 2 or 3 weeks ago CI Jenkins.io wasn't able to spin as your virtual machines and ACI container. Same expired credential. A manual credential has been generated on short term that is valid until June. Now we have to put that one as code. The tricky part on this one might be around the ACI. We will need not only permission on the virtual machine resource group but also the ACI resource group for Windows container but the rest is the same as cert.ca. And also same improvement on long term that should be a candidate for workload identity management in the future. So yep that one move automatically to the upcoming milestone. PGP keys expire on March 13th Mark I took this one was related to the DG certs that we had to generate the PGP key with DG certs. But that might not be the case we should be able to operate this one quickly at least for generating the credential. Right. As far as I know now I hope that 3 years ago when we did this we left ourselves some facility that will allow us to do this without requiring that every user install the new key. But I don't know that for sure we've got to do the research to figure it out and we've got nine days to do it. So we absolutely need to start working on this one. Okay. Is it worth it to go to ask Olivier for help on this one? I think the first that Olivier would want is let's read the materials that were documented from 3 years ago go through them and be sure that we understood what they say and then if we have a question we talk to him but I'm confident given the experience with the DG cert thing where Marc said hey don't have any documentation Olivier then said yes we do it's right here look at this page read this page and it was all right there so I suspect the same thing will apply here is that we've got documentation that was assembled 3 years ago when we lasted this transition and we just need to read it first to be sure if we do need help from Olivier Okay we need to check the impacts that's the most important part do we need to can we just renew the key can we just extend its availability do we have to create a new one I'm not sure Right and that workflow is the key question is the crucial question not a new key and also of course what if we have to focus on depending on the workflow do we need to trigger a new LTS and core weekly release for signing again a new version with the new key if we need a new key and that I think we won't need to at last the last time we didn't what we did was we announced a we did a blog post the last time which said if you are changing please install the new signing key now I don't know if that kind of message will be needed again or if we found a way in our last exercise to allow that to be avoided again so avoided this time sorry it was three years ago I don't remember No problem so that means we need to check with the security team if there are any impact no security impact in that sense because this is all provider authentication provider provider validation information not anything to do with with vulnerability so last time we didn't involve the security team I think at all and they were just fine with it okay my question to them will be more about the impact of renewing the expiration date of an existing GPG key versus creating a new GPG key what we know the the consequences will be interested in the difference in terms of security pattern I see okay and in that case they're experts good thing to ask them that makes sense thanks out of space on the bomb builds survey do you mind giving us summary of what has been done on this one we noticed that the bomb build were using MTD volume to what they have to do and the MTD were using the node space available space the cluster node space which were 20 gigabytes and the build the bomb need more than that so they were filling we increased the cluster nodes disk size to 200 gigabytes and increase their IOPS and type we now need to maybe decrease size to 90 gigabytes it should be enough since there are at most three builds on each node at the same time and the bomb build after the bomb build we can see that 23 gigabytes are used that's a nice data point that we have we know how much space we need on worst case for running bomb builds we also need to check some volumes as they are limited as overlay as docker overlay and not as MTD and so they are they are not efficient as a matter of fact a good rule of thumb will be enabling the read only flag for any container whether it's Kubernetes container or docker container and then solve of the denying right issues by defining docker volume or Kubernetes MTD or TMPFS because these volumes are allowed to be written and the performances and the overlay as survey said are particularly poor so in the case of our builds for instance Kubernetes plugin on Jenkins as proven by chassis create an empty here for the workspace where the build happens which is good however on a default build when you run Maven for instance it creates and writes data in the home directory slash dot m2 repository by default that thing is written inside the container layering file system so if you need to read or write a lot of data the performances will be really bad there so we might want to mount explicitly the home Jenkins as an empty here now the question is we know how it behave with docker docker mounts the directory and copy the expected data on the new directory out of the overlay which is a few seconds more but in the case of Kubernetes we don't know also the slash TMP directory is not a TMPFS and is not an empty here so on to define that as a TMPFS a RAM disk that will take a bit of memory but we have margin on these machines worst case we use 24 GB of the 32 so we have a few GB left so mounting this slash TMP to something like 500 MB will be enough if it's full that means something is going very wrong because you even a build a Maven build shouldn't write that amount of data on a TMP here particularly because Jenkins provide a workspace underscore TMP environment variable which is a temp directory to write files within so these are improvements Erveille what do you prefer so do you want to continue working on that desk first question I will not but I need the help to check where these volumes are ok no problem do you feel we should keep the issue open and work this optimization as part of the issue or the definition of done or do you want to create separated issue as improvement now the problem is solved what do you prefer ok because since I did differently on other issues that way I am asking and both are fine it looks ok for you so just a note we don't have any more disk full issues for the bomb Mark can you confirm that you didn't see any more issues since Erveille's operation on that confirmed we cleared the backlog of 20 pull requests over the weekend and released a new version of the bomb early Monday so or Monday sometime 20 pull requests of the bomb during the weekend yes so it took quite a bit well it was a lovely weekend exercise to try to get that thing cleared and I'm glad that we did it because that meant that there were 36 dependency changes in the most recent release of the bomb 36 is probably 3 times larger than our typical bomb size release so it was time to catch up thanks very much for doing it ok so that means 6000 pods were were executed well one ok I admit one of the one of the things to catch up on those 20 pull requests actually combined six pull requests into one ok so so I did some I did some attempt to machine minimize but it was still at least 10 cycles of the bill of materials therefore you can approximate how many machines were built up yes it was a lot so that mean we that explain partly some of the increase of the AWS bill though but digitalized also it's part of the workload so yeah yes my apologies it probably was I tried to keep the cost low but you're absolutely right it was not free to run that to catch up on those pull requests ok so I will move that issue on next milestone buttons 10 à paradigm les 1 je m'en ai tout à fait rifles Jams 9 50 four non Est-ce que vous vous rappelez que le scope des changements nécessaires pour celui-là? C'est le shooting down release, c'est le CIO de Jenkins.io, pour voir mes questions. Ok, donc le CIO de release doit être le dernier mile avant de serrer, est-ce que c'est correct? Ou est-ce que nous avons d'autres services identifiés pour s'entraîner à ce cluster? C'est le dernier service. Nous avons besoin de bouger de ce cluster pour celui-là privé. C'est le suivi des services publics. Ok. Comment est-ce que l'opération doit être approximative pour vous? Je l'ai estimé jusqu'à aujourd'hui. Je ne pense pas qu'on a besoin de ces deux jours. Mais oui, pour être sûr, je l'ai estimé. Ok. Qu'est-ce que ça fait le plan d'entraînement à la fin de la semaine? Qu'est-ce que c'est le meilleur détail pour vous? Merci pour cette semaine. Ok. Est-ce que ça peut vous envoyer pour tout le monde? Est-ce que vous pensez que nous devons commencer cette partie de la migration? Si le plan est prêt. Pourquoi pas? Ok. Vous devez décider le temps d'un jour, mettre ça sur l'issue, et puis communiquer. Nous devons aussi préparer la communication. Nous devons aussi préparer la communication. Nous devons penser à l'accès de la VPN, parce que ça va requiser les clients de la relance de la CIO pour apporter les accounts avec notre aide, avec notre recommandation, pour qu'ils puissent utiliser les nouvelles instances de VPN privées. Oui. Est-ce que c'est le partenariat? Est-ce que c'est ok pour vous? Ou vous avez besoin d'un peu plus de temps sur cette partie? Non, ce n'est pas seulement le partenariat. C'est pas un problème. C'est-à-dire que vous devez... Est-ce que c'est ok pour vous de prioritiser le plan? Préparer le message, pour que les gens puissent savoir ce qu'ils doivent faire. Si pour les gens qui vérifient comme Christian, qui vérifient la relance de la CIO pour la semaine, qu'est-ce qu'ils doivent faire jusqu'à la semaine prochaine, pour vérifier et monitorer la relance de la semaine prochaine? Alors, on va définir le temps d'adaptation dans l'issue et la communication. Vous avez dit que l'application de VPN, c'est-à-dire l'accès à l'adaptation privée. Alors, est-ce que vous avez pu vérifier l'availabilité de l'application de Windows? Est-ce qu'il est capable d'attendre l'agent et tout? Vous avez dit que nous n'avons qu'à migrer la relance de la CIO. Nous avons besoin d'un set de tests de dummy avec un pipeline qui ressemble à la relance de l'application, mais ce n'est rien d'exception. C'est le matin. Exception d'agents? Je vais discuter avec ce acteur. Je ne sais pas. Je vais vérifier le plan d'availabilité de l'application. Si vous voulez. Oui. Oui, mais ce n'est pas sur comment, c'est sur quoi. Nous avons besoin avant de planir la migration, d'être sûrs que nous pouvons couper les portes de Windows et l'agent Linux avec la même définition de l'application que l'availabilité de l'application. Depuis que ce n'était pas le point que je disais, je sais que si nous pouvons vérifier le plan que j'ai fait, parce qu'il n'y a pas d'adaptation. Depuis que je n'ai pas d'adaptation, il y a déjà beaucoup d'adaptations. Non, ce n'est pas beaucoup. Nous avons besoin de vérifier le plan d'availabilité de l'application. Nous pouvons prendre ce temps comme objectif. Le temps d'availabilité de l'availabilité de l'application sera décidé seulement si nous sommesOK sur le plan. Est-ce que c'est OK pour vous? Oui, c'est parfait. Juste la dernière, parce que c'est juste ma pensée. Nous avons besoin de vérifier le point Azure pour l'availabilité d'availabilité de l'application. Juste pensez à ce point, nous allons parler de ça, mais c'est important. Réguis-le. Oui, des mesures importantes pour vérifier. Nous avons VPN, agent allocation, et nous avons credential Azure Vaults et points. OK. La prochaine tâche est appliquée pour la coopération de Morgenkind Manager et Jennifer Leaders, et cette est pas la question, mais je dois t'en ramassé un email pour les gens et on verra la réponse. Para celles que j'aielescère, c'était déjà le cas. Donc, ce sera mon erreur. Mais je rassure que ce soit собирé. Je ne m'en souviens pas de prendre ce que je vous ai dé Kolk. Je les ai envoyés la dernière fois. unless someone wants to take the subject, and I don't mind either. If they say no for Jenkins CI Infra, the alternative would be to use GHCR as suggested by RV, that could ease the access control and that could scope the images per repository since we really have a kind of airbag permission system inside GitHub. Oh, GHCR is the one in GitHub. OK. Yes. If not possible, we could move to GHCR, GitHub registry. Sure. Ah, you need to write down. Rose better airbag already managed per repository. Risk, in that case, outside the fact that we only have to change the name and create GitHub tokens, that should be OK. That would be pipe and library change. Instead of using the pre-built Docker Hub credential, we use it with IAT, the GitHub application, one-hour value token generated on each call. So we should just wrap a risk credential with the GitHub app and that should be OK. The risk is more rate limit costs in GHCR. I don't know the limitation, but maybe it's free for us since we are sponsored, but that remain to be checked. I mean, GitHub action have limits on GHCR as well. For Jenkins forever, we'll see, we'll discuss. Is there any objection if I send an e-mail to ask the developer what do we still need that one and would GHCR be a solution for this one? Because that one produces unsafe artifacts, so I would rather having that one removed from Docker Hub and not used anymore. But maybe I'm missing something and let's ask the contributors. We have a service that should be sensitive and removed, Robo Butler. That one should be an easy task. It's a set of puppets. I propose to move it outside the next milestone given the work, but that one is uneasy to take if someone is bored and have some time to spend. Is that OK for you? Let's move to backlog. I am not sure what is the Gatsby plugin Jenkins but it shows about, can you, do you remember? Yes, it's about an issue which has been done and you were not sure if we should create a separate GitHub action just for the semantic release action. For me, it's not needed, but I'm not sure. A GitHub action? There is a GitHub action to do the semantic release on this repository and it's the same I've reused the GitHub action from the Jenkins IO components repository and you were not sure about that and you mentioned which might need to create another GitHub action for that. Mike, Oh, a GitHub application, do you mean? OK, GitHub action is a CI system. OK, sorry, that's why I was confused. You said GitHub action, that's not a GitHub action here, right? Or did I miss? Sorry, I don't understand. It's a specific process for the release on this repository and it's using a GitHub app. OK. And you were not sure if we should have used the same GitHub app for these two repositories. OK, I see. Yeah. The risk here is if one application tries to write or perform a release on another repository that use the same GitHub app, then you could have cross repository writings which should not be possible by default. So that's why ends my proposal of creating multiple GitHub app one per repository, but that could be one per area. The question is more, is it acceptable if, in our case, Jenkins IU components that you see on the screen, is the risk of Gatsby plug-in Jenkins layer being able to write on Jenkins IU components? Is this risk acceptable or not? Otherwise, we could increase the scope of GitHub app to, let's say, an area of particular technology or services that have a link between each other. That's my concern. Pour the Jenkins infrastructure that looks like the same area, I will put that under the umbrella of whatever front-end web stuff. The thing is managing manuals, all those GitHub apps start to be problematic and as you underlined correctly, or the last time, now I remember, the amount of manual management and credentials to updates could be a counter-effect of trying to separate properly elements. Unless we have a GitHub app management as code in an automated way, your argument makes some survey and I propose that you can close this issue. It's okay that we don't separate GitHub app because that will be another world topic on that one. Does everyone agree on this one or do you think we should? If I understood correctly, we lower the workload by having some kind of risk but lower level instead of separating everything and having to handle all the credentials. If it's okay for all of you, I will close the issue with a message about the trade-off here. Does it make sense for all of you? Thanks, sir. That means the issue is fixed technically or functionally speaking and we only have to write on the trade-off. Thanks for handling that. To be closed, trade-off secure in term of security and permissions. We have another issue to work on grant access to some security folk to release CI. These people need to be added on both the VPN the new VPN eventually I'm crossing my fingers for this one but they need a VPN account in any case and then in the release CI airbag system. Also we need to check the airbag configuration on release CI because right now anyone with the VPN access and authenticated has too much permissions. So we need to create an airbag matrix inside Jenkins. So that task must be done quite quickly. Not sure what's gonna make it I will move that to the upcoming milestone by default I will take it unless someone want to help me or work on it. I will let you decide if someone has some time to spend on this one. For me, we won't have time next milestone but I want to put it just in case. Update center job is failing. That one the root cause is fixed Ivan closed it yet there are some puppet management things to ensure that we have the correct Blobix for install everywhere and we have to perform a Blobix for upgrade migration that could have consequences on a lot of scripts but I think we will have to do this one quickly however that was the point last time but I saw that was able to solve some issues on other unrelated repository but that also use Blobix for and I don't remember if the the work that you did or team did on this area were able to use Azure CLI instead of Blobix for Ivan stuff OK so if it's OK for everyone the definition of done for that issue will be the second item here meaning putting on the infras code a way to ensure that Blobix for is installed that should be explicit on puppets because it was done manually before once it's done we can close the issue that will be the improvement to avoid reproducing the issue the Blobix for upgrade campaign should be a subsequent issues if it's OK for everyone it won't be priority that could be superseded by the migration to Azure CLI instead that's why I want to make it another issue is that OK for everyone then code signing certificate renewal process that one is quick we are waiting for lawyers of the Linux foundation to discuss with lawyers of digital and lawyers of CDF and once then the CDF Fatih from CDF will come back to mark with a new digital a new certificate to sign the MSI of Jenkins on the jar the impact shouldn't be that big but that should be better if we could have that renewed otherwise anyone trying to run the Jenkins.msi installer will be congratulated with hey no don't I'm not really sure of all the impact though from our point of view in the infrastructure that mean we will take the new private certificate and put it on the Azure Vault which is used by ReleaseCI through the Vault endpoints and my remark earlier ReleaseCI use it to sign the jar and the windows package and that needs to be encrypted inside the Azure Vault password system so that should be only a Vault update so we have to unseal the Vault put the new version seal it and wait for the next release to trigger let's try a replay with a dummy command that say hey do we have the five first character of the key inside just to validate it's mounted correctly so nothing for us there that will move automatically to the next that one is outside our hands any question on this one ok the rest is artifact caching proxy can you confirm that I did everything you needed for merging on javadoc the javadoc project yes I haven't checked is it working as expected or did you had time to check if it was working because it's once a week or I haven't checked on just ok it was running as expected cool so that mean it uses ACP right it's using ACP on CI and it should not entrusted so that should continue working as before is that correct nice job is there any other ACP topic outside the issue you were dealing with are there other projects or can we say that one is almost being able to be closed isn't that lovely that's a big one sorry and that's a big one I mean ACP it's huge but I didn't hear what Serge was saying oh ok so once we are sure that javadoc is ok the last mile for us will be to check can I ask you to drive that topic so you should be able to close the issue the goal is to check the measure from Gfrog to see we have our top IPs and if they are eating Gfrog less than it was in December at least Mark might be able to help and Basil as well and if you see we have decreased and we are not on the top whatever that mean we can proceed and close that issue sounds good to you that will be your end of quarter victory on the real line sorry that HA work in progress I wasn't able to work on this one I need to focus on how to have a replicated LDAP especially inside the Kubernetes the last one computer and BMC plugin removal is just for information in our case Danielle created it, if it's ok we will remove any milestone it's just to track the work of there has been a copyright infringement we received the notification by the Github whatever system used and the Jenkins project at free day to comply so it has been treated by the Jenkins board yesterday and all of the plugin related to compare and BMC has been removed because of the copyright thing the issue there has been an email sent to compare war people by the board Mark also copied the content of the email on the issue there is no infrastructure action right now it's only to track this and Danielle did that as part of the LDAC process any question so I know that's a lot taking a lot of time but we need to check if we have new issues that hasn't been triage right now we have this one I want to create a new account update social link on Josh organization monitored ACPD excusez-vous c'est déjà fait c'est fait ok c'est la question d'aujourd'hui pas de problème si c'est ok pour vous, je vais prendre les issues pour vous remercier quelque chose ok c'est le même c'est le même c'est le même non c'est moi je ne sais pas qui vous référez c'est le même mais c'est le même vous n'avez pas filteré non je ne sais pas je suis juste filtrant avec ma A3 ok donc nous avons 2 tâches d'account et le même c'est plus facile ok je vais les mettre et ce sera fermé parfait d'autres topics ou des issues ou des choses que j'aurais mises ? non ok donc, laissez-moi arrêter la recordation pour tout le monde qui s'occupe, à la prochaine week alors arrêtez de partager, arrêtez la recordation