 Okay, I realize. Welcome to the regular Jenkins configuration as code sub-project meeting. Today is June 5th, 2019. We have several people on the call. It's Joseph Patterson in Jackamp and Ramon Lyon. We will have several topics in the agenda today. Probably I'll just share my screen. Okay, do you see the screen? Yes. I hope this time I don't mess it up because yesterday we had a knowledge transfer and finally the screen share wasn't recorded properly. Okay, so yay. I just followed the agenda template which we had before, so I really wasn't able to make the meeting today, but I believe she will join next meetings. So yeah, let's talk about news. There are also some updates in compatibility for JCASC. There are some ongoing discussions about JCASC test automation in Jenkins Core and also test framework enhancements. And if you have any other topics you would like to add to the agenda, let's discuss that. So does anyone want to discuss anything else apart from this list? No. Yeah, one of the topics we could discuss, we have one open pull request to the Jenkins Core. Well, not just one, but one seems to be stalled about default properties. So maybe we could also add them to the agenda. Sure. Okay. So yeah, last meeting was almost one month ago. Since that we had a bunch of Jenkins configurations called releases. Actually, we had quite a number of releases since that time. I believe for last release we were discussing was 1.14. So maybe you Joseph would like to summarize what changed since that time because you were driving the integrations. Yes. So we fixed a bunch of the export issues we had. But that also meant while fixing those we moved all the stuff out to the decided or the respective plugins. So we no longer have any support plugins inside the latest release. So that's a good achievement I would say that we now have everyone in every plugin has their own configurator that they used to be hosted inside the support plugin. So people can go ahead and uninstall support plugin. And we mostly did bug fixes. I guess we had a stable converter issue because of a core update that was luckily caught in time before it became an issue, I would say. So yeah, and other small bug fixes. We made some regressions though from 17, 18, they're probably a bit off going straight to 19. Yeah, right. Yeah, I tested the last version and it looks perfectly fine in terms of experts. Yes. So yeah, thanks a lot for fixing it quickly. One comment about the configuration as code support. Yeah, it's cool that we got rid of that. But yeah, there is a way to actually mark it as deprecated on Jenkins plugin page because for example here it's not marked as deprecated. And it's also not marked as deprecated in update centers. Don't think it has a biggie page. That's the issue, I guess. Yeah, maybe it's also something to consider just to have a kind of Tom stone there. So yeah, basically deprecating a plugin. So there is a process which is pretty straightforward actually. So you can use that in order to just get the notice on a plugin page and in update centers. I'll go through that. Sure. So yeah, it's something to keep in mind. Do we have anything else about new releases? I think we covered most of it. Yeah, a few bug fixes. That's not too noteworthy to mention. Oh, there's actually one important. So there was a bug where if you were using Docker instances or any environment variables, the config file that you could point to was being, or the environment variable wasn't the preferred thing. So if it was previously loaded the configuration, then it would always use the one that was configured inside the source, I guess, or the configuration file over preferring the environment variable. So that's one improvement, I would say. Yeah, it's important. Yeah. Then a previous configuration. Yeah. Okay. Okay. Thank you. So what else do we have? So regarding new maintainers. Yeah, now I believe it's official. So jacamp and sorry, there is some background noise. It's too. Yeah, it's a background noise from London. It's totally justified. Yeah. So yeah, team and me. There was a discussion yesterday and now we officially joined the maintainer team. So we have all permissions including releases if needed. So just in case we have summer breaks and other things we have mobile and wife in the plugin in terms of maintenance. So if you take a look at current repository permission on data. By the way, team, you have a merge permissions, right? Yep. Yep. So I didn't give out support plugin because it was too, because we finished with that. So Joseph needs to do one more release of that potentially. No, the final release for support plugin was the last one. So actually, if we need to deprecate it in the update center though, we need to do one more. Wow. Okay. So this is our current configuration. So program is released both used by Evelina. Then we have Joseph for me team. And yeah, Evelina stays security contact. So yeah, this is the current configuration for the configuration squad plugin. And obviously there is a lot of things happening around in the project. So yeah, maybe maintenance will be more and more simple. Okay. Back to the agenda. Yeah. We also got a number of integrations. So one of the topics already referenced by Joseph that configuration as code support plugin is deprecated. So all integrations have been moved to the respective plugins. There is also a number of ongoing work for making other plugins compatible. Yeah. I think we can just switch to the next topic about ability abuse. There were some changes. So firstly, over the past few weeks, I spent some time to scrap all the existing PGA configuration issues in Jenkins Jira. So yeah, before that we had some issues in GitHub issues. We had some issues in Jenkins Jira. Some issues were not categorized correctly. So I just spent some time in order to process all the reports and put them in Jira. So we use the dashboard we originally created almost one year ago. And now here you can see all the references and those resolutions. So yeah, there are still some gaps, which I will clean up. But basically it can be a good source of information for whomever wants to understand what is not supported. I think in this dashboard that we don't have a list of what is supported because we assume that this is our current frameworks. Many things actually supported out of the box. So we didn't test everything. But yeah, this dashboard tracks newly discovered issues at least. Okay. So regarding this dashboard, maybe it's a question to you, Tim and Joseph. What we do to duplicate the efforts? Because my understanding is that we still have a kind of overlap between this dashboard and this plugin compatibility tracking. So we should add a link to the dashboards in that comment, I think. There's two dashboards. There's one that shows all the issues and then there's one that just shows open issues. I think the open issues one is in the readme now. Okay. The idea for the plugin tracker on GitHub was more also for pull requests because it's a lot easier to track what is ongoing for pull requests on GitHub at least. So that was the idea for that. Yeah, which totally makes sense because you can plug in maintain ourselves back to somehow facilitating integration reviews. Exactly. It's our joint interest. GitHub dashboard is rather for pull requests. So what I was doing, I actually agree with Joseph that it's rather for pull requests. When I was scrubbing to the dashboard, I just went through the list of issues here and I hope I created the G-re issues for all plugins. So you may see links here. I haven't renamed all pull requests, but I believe that I went through this list. Yeah, so we can still keep using this for tracking pull requests and then somebody takes a look and just creates a G-re mirror issue so that we get to this dashboard because one of the advantages here that we have released as field, which kind of helps. But that could probably be a nice integration to G-re, whether plug-in or pull request oversight or something. We had such integration, but at some point we disabled it because it was bringing down our G-re instance. Now there is an ongoing project for migrating G-re instance from self-hosted instance to Atlas in Cloud. Once we move to Atlas in Cloud, maybe we will have less performance issues and we will be able to reconsider that. But yeah, right now performance was the main concern and the root cause why we disabled integration with GitHub. I think that Atlas in Cloud also has a better integration for GitHub, so that should help. Also a fair point. It's just you need to be on the new version of G-re, unless in Cloud it should be the same. Yeah. Yeah, wait, I think for creation to... So yeah, you can also make the meeting notes, by the way. Okay. Another thing by Joseph is actually we now have a new query for Jenkins core issues because in addition to plug-in we also discovered the number of issues related to Jenkins core. So here we use the same Jcast compatibility table as for plug-ins, but it just filters on all core issues. And here you might see that, well, actually it's quite a number of issues. Some issues are other pull requests like the only configurations, which is, well, it's probably big, it's really an epic. It's not just a single feature. But yeah, there are also some relatively small issues we are trying to fix, like listing view management, default properties into them, which we will discuss later, and yeah, other such things. And hopefully we will be able to learn some requests soon. Okay. Anything else about this topic? Okay. One issue which still concerns me a lot is about this one, job loading and Jcast chicken and egg. So yeah, it's a pretty old issue we have in Jcast plug-in that basically job loading is not really reliable. So maybe it's something we could think about maybe at the next meeting because it's a big chunk of work and we really need to coordinate what we need to fix that. Does anybody hear this issue except me? No. I've heard it several times with some plugins at least as well that expect configuration to be alive before it's there. Yeah. I don't have it in my setup. Okay. So yeah, probably this issue will end up in jab. But yeah, this is what we need to discuss because currently initialization graph for Jenkins isn't really configuration is called friendly in any means. So yeah, it's likely a massive patch for Jenkins core. There is also the plugin tool management tool that we could probably discuss introducing them at the same time. So they probably need one before us, I would say. Yeah. So regarding plugin management tool. So basically this is a project which happens on the umbrella of a platform special interest group. So just a second. I'll open this page. Oh, no. So yeah, we have this project plugin installation manager manager to sell it to. Yeah. So this is a project, which is currently in progress by Natasha stop by. She's one of JSOC students. You can see some familiar names in the mental list. And basically this project is going forward to have a plugin manager and CLI library or library. And once it's done, we could consider integrating this library into the core and maybe managing plugins by configuration as cool. The biggest problem with this plugin with this story, I agree with you, Joseph, that it's related. But the problem is that it still goes not in the configuration startup. So yeah, plugin management would need to happen here. Not here. Yeah. So it's another chicken and egg problem because you kind of really manage plugins from a plugin. And basically what it means for me that it shouldn't be a plugin. It should be rather a library, which we inject in the Jenkins word file on startup and which does some management. And there are some ways to do that. One of the ways is to use custom word packages as we discussed before and inject this library. Or maybe we could just make it a part of Jenkins core functionality, which is also possible. But it will require discussion and also probably a job. But I think it might make sense to make it a part of Jenkins core. Yeah. Okay. Yeah, so it's also a kind of bigger story. I just do nothing that we will get to it right now because yeah, this project is in progress. So likely we will have something by August fingers crossed. But yeah. Yeah. So for now, it's safer to say that jcasc just doesn't manage Jenkins plugins. And yeah, likely we'll have jcasc support in the core or in this tool. But yeah, you might have support of reading the same configuration files if you want. But yeah, it's a story yet to be discussed. I would say. Okay. Anything else about core issues? Yeah. By the way, I should probably get plugin management to this list because now we don't really have it. Okay. So if there is no top XK, we can just move on to testing things. We basically have two stories. One is about automating jcasc test in the Jenkins core. So it comes actually from these stories because yeah, we know about several bugs in Jenkins core. And these issues are not the first defects we fixed in Jenkins core to support the Jenkins configuration score. So this query now doesn't support historical issues, which have been fixed, for example, last summer by Nicola Belov and probably we'll need to add them to the dashboard later. But yeah, anyway, at least some issues represented here. And when we fix something, it would be nice to have some test automations so that we can cover that. And we had a discussion about it yesterday at this in the pull request. And the proposal was to actually have jcasc integration tests in the Jenkins core. There are multiple ways to do that. So one of the ways which was pretty straightforward is just adding everything to Jenkins core test suite. But it looks like this approach gets negative feedback from Jesse and Alex L. Because yeah, they are concerned that jcasc already has a lot of Sorry, this test suite already has a lot of tests for different plugins. Jenkins. Yeah, so basically this is a test suite. And the concern that it's already too heavy. So we could resolve it by multiple approaches. So one of the approaches that we put to just put another test suite here. So for example, we already have from test suite for jcasc, which includes tests which are not relevant on Java 11. So we could also put a test jcasc whatever test suite here as one of the solutions to address Jesse's concerns. But yeah, it might be a bit lame. And actually what Jesse proposed is to rather have plugin compatibility tester. So plugin compatibility tester hasn't been discussed at this special interest group meetings yet. Right? Do you want to have a quick introduction for this tool? I would. Okay. So what we have here, there is a plugin compatibility tester framework. Actually, this framework is a kind of cornerstone of all new automation we are doing. This framework has been heavily used when we were working on JEP 200, then when we were working on Java 11 support in Jenkins. So this framework has a lot of enhancements, lots of tooling documentation now. What it basically does, it runs tests from a plugin against a particular Jenkins core or word package. So it means that if you use custom word packages or other tools to package your own Jenkins word file, you still can use this framework. And then, well, you can run it in Docker, you can run it in local thing, but what it takes, it actually takes artifact ID and it takes Jenkins core if you want. So you can, for example, pass custom Jenkins word file, you can pass custom sources from a repository or from a local directory. And then a plugin compatibility tester which will build combination of Jenkins word file and the plugin you passed. So basically, it's a really complicated version of this command. So, yeah, what we do in Jenkins sometimes to fight against the new Jenkins core. Defy, minus D, the Jenkins version equal, let's say, to what 164. So, plugin compatibility tester is an advanced version of it. And it also produces some fancy test reports and compatibility matrixes. I'm not sure I have any on this machine because it's a new one. Let me take a look. Jenkins. Demo. Okay. So, let's, oh, I have some report. Yeah, plugin, PCT report. Positive and working. Yeah, it should work. Okay, it doesn't work. So, yeah, the demo failed. Documents Jenkins. Yeah, I'll probably apply some tweaks for that once I'm on a VSL to the zero. But yeah, for now, yeah, I'll probably just open it in such way out. PCT report HTML. So, I have no idea what the reports there. Oh, yeah, so this is a combination matrix for some last Java 11 test. So, you may see that actually we launched plugin compatibility test against a number of plugins and against multiple Jenkins configurations. So, it was weekly release. It was LTS. And here you can see some test results. So, basically, if you go there, just, well, you just execute Jenkins build to be some updates. So, it's not just this maybe clean-verify-manus Jenkins version because we apply a lot of fixes to plugin form and to form in order to make it compatible with newer version. We automatically resolve enforcer issues, we resolve issues with detached plugins. So, when just a plugin form doesn't work, a plugin path tester can work and this is the main advantage of it. Now, it leads to a lot of complexities and as PCT maintainers, we also have a lot of fun keeping the consensus stable. So, what we could do, we could just run a plugin compatibility tester against configuration code plugin as a part of a Jenkins pipeline. So, we could run take the latest release of jcask plugin or maybe the development version and we could take the underdevelopment version of Jenkins core and run within compatibility tester in order to verify compatibility as a part of Jenkins pipeline. And yeah, pretty much the same, we could also use PCT in order to do some integration testing and set a jcask plugin if you want. Okay, any questions so far? So, yeah maybe somebody has already tried PCT for Jenkins configuration as code? Yes. Oh, you already did that? Yes. It fails. It fails because the first thing PCT does is process classes and it fails not funding one library Jamel the Jamel library we use if you do the same in the plugin compatibility in the configuration as code plugin if you do the same it fails as well. So, maybe we need to do a hook in the PCT or adjust the life cycle of configuration as code plugin. Yeah, so it's speaking of advanced peaks, you may see that there is a number of hooks here because, yeah, these hooks I actually needed in order to work around some issues we know about. For example, low channel hook, what it does it just it properly retrieves where how the plugin is organized where it's POM data located and here there is some, just some location logic which allows to understand whether the plugin actually belongs to the ocean. So, yeah, there is a lot of such hooks which you make the repositories working properly and if you want to use PCT it's something we will need to fix. So, yeah, it would be great if we have the issue for that later so that we could take a look together maybe at the next meeting so, let's see. Okay. So, yeah, Joseph team, what do you think about using PCT for the repository and maybe for Jenkins Core once it runs properly? So, we would move our integrations to PCT rather than the actual repo, is that how it would work? Well, you can keep integrations as this so, all existing tests can be retained as we have but we can use PCT in order to invoke this test against different Jenkins Core versions which are not compatible by default or maybe we could use PCT in other versions to test against jcask. Does it make sense? Yeah, it definitely seems worthwhile to be able to test on more recent Core versions we're already running against roughly the latest LTS I think. Yeah, so you're running recent LTS you're running the current version in the baseline but if you want we can add more. The build already takes quite a long time and it's very easy to hit infrastructure issues that's why Windows isn't being run there anymore. Is it important from perspective? Windows not so much but it would be nice to get the fixed agents running out of disk space that's an issue we'll hit a lot because we have a lot of people trying to do pull request. Yeah, right. Regular disk space issues. Yeah, we'll be talking about it with Olivier yesterday at the meeting actually so maybe we will need to work a bit on improving the situation. Well, we will definitely need to do something but it's complicated now. One of the things which we actually hit a few days ago that if you want to import configuration it should have no spaces in the past. No, it was fixed. It was failure in the test because the path I was setting in the input text in the UI was wrong in Windows and Linux was right so I fixed that and now it can have spaces. Okay, so basically it's not relevant. Yeah, sorry. No worries. Yeah, but basically having at least some Windows test automation definitely wouldn't hurt. Okay. Yeah, so probably yeah, I like to create a low up task. We'll just keep it in mind so that we don't miss it. Okay, thanks for the fixes. Okay. Other related stories about Jhask test framework enhancements. It's a pull request by Ramon and maybe he could explain context. Yes. We would like to add Jhask support for several plugins so to go faster on the integration I added an abstract class to make it easier for plugin authors to make a test to check the complete roundtrip we said about configuring an instance sporting the configuration, validating that configuration against the UI and importing the configuration. So far the sporting file cannot be imported so I did a workaround in the PR to just use the specific settings for the plugin. It concerns a team as he wrote because the goal of the test is not achieved because of the failure in the export. I think that maybe what we can do until the export file can be imported successfully is just to crop the specific part of the plugin under test from the exported file and just import that part. With this changing the PR I mean the plugin author is going to be sure that the configuration is successfully exported and imported. So how much is missing to get it working? Is it only about this cool request? Yes, it's a changing the PR. Also I was trying to get the sported file, the sported configuration against the schema but I didn't manage to do that because I didn't find a right tool to do that. I was trying to do the JSON schema is very broken. Functionality just doesn't work. Do we have tasks for that? There's an issue with it with some comments on it. I spent an hour or two on it but it just doesn't work. There's lots of issues with it. I fixed a couple of them but it needs a good amount of work or rewrite. I think the best thing to do for the schema is to rewrite it. Use a library instead or something that it feels wrong that it's not like it's handwritten with some what it's called. It's stapler magic or jelly magic. I would prefer the Java code instead of someone handwritten jelly code. I have a question for Tim. He would be happy if I do that change in the PR and export the file and just ignore it from this file, the specific part of the plugin and import that part to be sure that the configure and export configuration of the plugin is safely imported. That should be fine. I've got to hit it. Okay. Here we agreed that import configuration files does work around. One of the stories about this particular pull request while we are around, there is a pull request from the Jenkins core and I'm kind of body blocking for that. I wanted to understand what we do next. I approved it. The situation with this pull request is that we want to get rid of these default properties but there are two ways to do that. One way is what is currently implemented by Joseph just by restricting method in the Jenkins API. We did some verification and this method is not used anywhere where we have access code too so it includes open source plugins and for example some private source plugins I can access. So this method is not used and we could probably just lend this patch as a workaround. The problem is that even if you lend this patch, it won't be in the LCS baseline because we cannot backport this change where we duplicate the API. Well, probably we could but it would require serious discussion with Oliver Gorn as a LCS officer. One of the workarounds I was proposing for this particular change is actually having a blacklist on the jcask side so yeah there was a kind of discussion here. I basically proposed that instead of doing this patch or maybe in parallel with doing this patch, we teach jcask to somehow ignore arbitrary methods we pass and for example we can use this engine in order to just skip this method so that it doesn't get exported in jcask anymore. So my question to everybody on this call is what do we do about it? Do we want to press this code on the patch or do we want to have a patch on the jcask side? I think a patch in the jcask side is fine because I already did one patch actually for blacklisting something in credential plugin. So you already have an engine for... No, because that was specific for credential plugin. It was exporting in a very specific place. It was reporting the wrong category but it was weird. It shouldn't be there, it's gone now. I think we should definitely create an engine for introducing blacklisted in other places. I think it's something which would be important for us because my problem is not with this puller issue specifically but with the fact that it's rather an architectural issue because you likely have a lot of such methods and plugins. By having a way just blacklisted them on jcask side or maybe like we did for JEP I forgot the JEP number. Yeah, JEP 200. We actually have blacklist on the plugin side JEP 200. Yeah, so what we did in this implementation is that we actually support passing a resource inside the plugin directly which blacklist particular configurations from expert and I think that you can pass this resource without modifying APIs without depending on new versions so it was pretty straightforward for us to just whitelist particular things via resource file and probably we could consider doing similar approach for jcask as well if we hit these issues a lot. For Jenkins Core we can just hard code it or have an engine inside the jcask plugin and then we can extend by reducing some of the code from JEP 200 if needed. Sounds like a good plan. Can this file be set by the plugins itself to be able to deploy in south or to write his part? Yeah This was a whole idea because we were patching something like dozens more than 100 plugins for JEP 200 and we didn't want to bump dependencies to the recent versions like we would have needed so here you basically there is a remote in class filter which you put directly to the repository for example here and here for JEP 200 you whitelisted two class civilizations inside pipeline support plugin and you just put the resource file and it starts working. I was asking to do the same for jcask. The plugins on it can do that not only in the jcask plugin itself I would totally agree Yeah That would be the way for it. I don't want to Yeah So in the plugins instead of jcask So yeah this is what we can do so we can just create a file called something like let's say it uses Hudson Remoting and we call it IO Configuration as code expert at Blacklist I like the name meaningful Yeah with some a list of methods we ignore Yeah it might be a bit more tricky to implement it because let's say we can have inheritance so we will need to somehow handle that Question could it not also be annotations instead perhaps or both Yeah we can do both but so the problem with annotations is that annotations may require code bump so for example if we just deliver annotations as a part of existing configurations code plugin then yeah it will likely require a new code what we could have is for example having a new model in jcask like something annotations which would for example target java 7 or java 8 with Jenkins course and then plugins can include this annotations library so pretty much like we do for example for symbol annotations or for find bugs annotations well that kind of leads into another thing that me and Tim have been discussing in private is that we should probably consider splitting configurator API and the core plugin or the jcask plugin because that would be now we get back to why it's beneficial to split it basically it's because we need to bump the core and some plugins aren't ready and yeah yeah so we already approached this topic earlier I believe yeah for me I do have strong yeah so that is by the way it relates the job from Nicola about declarative data binding which also references the APIs regarding splitting so we had an old issue for me I still quarterly believe that having a split is really important because yeah split the jcask API plugin and jcask API so why it's important because we have a pull request like target Jenkins 164.1 so once this pull request is ready whatever annotations may include a plugin using this annotation would have to bump to these versions which is not acceptable for many maintainers and another thing that for example if we resolve chicken load issues in Jenkins core again we will have to bump Jenkins core and then all dependencies will have to bump as well so that's why I believe that this ticket is important and probably we could revise that the other thing is that we were looking at the numbers I think we saw that configuration as code most of them were 90% was above 250 LTS so yeah I think for the future of jcask it would make sense because we need new features okay so I'll just reopen I'll just reopen it for discussion yeah I basically told already had a discussion with Nikola and he's finally revisited that so yeah so yeah I reopened that and yeah I'll have well we need to investigate that but I believe of course sustainability of the project it would be something really important yeah sadly my original public request didn't get merged and now just needs to be reward completely because there is no way we resolve merge conflicts in it yeah I think actually I know Tim already started working on it he did the most minimal viable products so he already has a branch and everything working it could he has some working prototype at least okay so who does have it Tim okay okay yeah if we could do so it would be really nice so yeah if Tim creates a pull request yeah I'll just CC team so yeah I believe he knows what to do okay yeah because if we get it landed it would be really helpful for the project I think so too yeah it would be nice okay yeah that's cool okay so we have seven minutes left yeah so this engine is something we will discuss with Ramon because yeah since we depend on this story somewhat maybe we will be able to deliver it yet to be discussed and yeah since you guys work on that already you can take a look on that okay there is one other thing we probably could talk about is I know that the test framework the Jenkins test framework there's a plan to bump the version that would also be very nice because that would mean we could introduce JK's stuff we have a lot of test method can I show you something because yeah in another Google summer of code project we are already working on benchmarks for Jenkins test harness and one of the things we did for benchmarking so there was a demo last week this demo has a video but what we have here is just a second there is a lot of code but yeah there is a configuration as code in each benchmark so what it means it's a framework which allows to actually configure pre-configure benchmark instance using Jenkins configuration as code and we already use it in a role strategy plugin so in a role strategy plugin we configure benchmarks using the JKask so what it includes here it basically includes dependencies on configuration as code which is optional but yeah I suspect that something like that it could be a good foundation for other test framework improvements we have a few functions inside JKask that could be benefit instead of us distributing it it would be nice if the test harness could distribute them yeah so I believe it can be discussed in the same thread because since we get some pushback about including the JKask test in the core Jenkins test harness is also on the question but we can try discussing that with cases yeah so I'm still using the new keyboard and the MSApp thing so in JKask test logic for benchmarking basically I think that we should discuss it in the same way increased as for Jenkins code because yeah I believe this story is interrelated yes thank you so my expectation is that as long as the dependency is optional I already lost it it's fine yeah so as long as we have dependency as optional in Jenkins test harness we are fine but it's something which needs feedback and now we are also waiting for feedback about this pull request okay what I will do so yeah I think yeah JKask plugin is getting started on production so if we get it included in the default test framework personally I see no issue with it but yeah let's discuss this wider community okay anything else to discuss today no nope okay and we have meeting notes so whoever is interested you can just take a look at the meetings it's nice to see so fast progress in the plugin and yeah now with all this test automation and such we kind of work towards making even more production ready and of course story is like but export and maybe even JSON schema is something what is really important to us okay yeah so thanks all and I hope to see you in two weeks we'll do okay thank you bye