 So welcome everyone to the second day of DEF CONF and thank you for joining on Saturday morning. And I'm Tania and Viktor and I will be moderating this session. And let's get started with our first workshop and hear from Miroslav Frantzišek and Petr about the TMT tool. Guys, take it away. Sure. So, hey everyone. I hope you had a good night's sleep. And let's get into the TMT workshop. So first things first, probably a bit of interaction would be nice. So who are we actually? So my name is Frantzišek Nečas. I'm currently on the Pekki team and I enjoy automating just everything and anything mainly with Python. Petr, could you please go next? Yeah, so I'm a member of the operating system CI and my focus is on improving testing tools and the testing process. So TMT is one of my favorite topics in this area. And Miroslav. Hey everybody, I'm Miroslav Frantzišek. I'm from the testing farm team and we are providing the testing system as a service which serves as a backend for Federal, CRL, CN, CentroStream CI. Let it be there. Okay, thanks for the interactions and let's now get into it. So what can you actually expect from this talk or workshop? So first we are going to briefly introduce TMT so that we are on the same page but then we would like to get a bit more hands-on so that you can actually experiment and get something, some real result from this workshop that would be great. So feel free to pick one of your packages or components that you would like to try TMT with if you don't have any. That's also fine. Petr will have some suggestions once the time comes. Also it would be nice to have your Federal account ready but it's not completely necessary. It's only for one step I think, nice. And also even though this is a virtual workshop it will be quite a bit more difficult to help you but we will try as much as possible. So if you have any problems, questions, ask anytime, we will try to monitor the chat. So that's hopefully also, we'll go fine. We'll also briefly cover what's new since Asdefcon since there are quite a lot of new features and also some plans for the near future. Okay, so let's now get into it. Also I forgot to mention that there are some links in the presentation for example to our guide or to the TMT cheat sheet if you want to refer to that. They are also available from the chat event so maybe even after the workshop but it's not necessary doing the workshop completely. Okay, so what actually is TMT or test management tool and why was it created? The name suggests a lot I'd say but the motivation was to have a user-friendly way to work with tests. How do we achieve that? It is by having a human readable clear and concise configuration which is supposed to be consistent across various environments. And we use the FMF format which sends a flexible metadata format which is based on YAML so that's human readable but it has some features for example inheritance but those are not very important for this workshop. And we make use of three levels of metadata. So first there are tests which define for example the maximum duration for which the test can run. Then there are plans which basically group tests and define how the test should be executed. So for example that they should be executed in a virtual machine or in a container and then there are stories which define for example the Y of some features etc. But those are not very important for this workshop. Okay and there are also some more links to the guides or to examples which you can refer to later if you are interested. Okay so let's go next and the workshop's title was quite catchy I'd say. It was TMT is equal to freedom for tests and comfort for users. So I think we should elaborate a bit more on what we actually mean by this. So by freedom for tests we mean the fact that the tests are not reliant on some kind of internal infrastructure they can be anywhere basically and also they are not tied to a particular test case management system. The tests are stored directly in the github repository which makes it convenient to open source them and share them for example between upstream and downstream. And also it makes it easier to integrate components and write integration tests thanks to this. And then the second power was comfort for users. So the fact that the metadata is stored in the github repository is also quite convenient for this because everything is in one place which makes it convenient to track and to orientate yourself in it. It is human readable and should be quite simple and short. And also the configuration is consistent whether you want to run your tests in Peket inside github or github or in federal CI on your CentOS stream. Also we can quite easily modify on how it actually is supposed to be run using the command line interface which is supposed to be quite flexible so you can easily specify that you want to run your tests in a container or even locally or perhaps on a physical server that you have. And it also makes cooperation between developing easier. Okay so let's now get into some first practical steps where we are going to show you the differences between metadata. So the first step obviously is installing TMT if you don't have it already. There is a federal package called TMT which you can install directly. This is the base package. There are then some sub-packages for example for running tests inside containers or inside virtual machines. If you want to install them all then there is the TMT package for you which you can install. I will mention the exact name of the packages a bit later. And also if you prefer containers then there is the container image on Quay.io under the user testing farm which you can use as well for experimenting during this workshop. You will prefer this. Okay so you can install this well and give you a bit more examples. I hope maybe the other guys can paste something in the chat to the commands which you can follow but let's continue for now. The first step in working with TMT if you want to start from scratch is initializing your repository. So there is a command TMT in it which is basically equivalent of git init. It prepares the repository. You can make use of some examples that we have example templates. So for example if you use a template mini it is going to create some supplement data so that we can get a bit more acquainted with TMT. Also just a side note from me. Basically every command in TMT has quite detailed help message. So if you wonder about some flags or maybe some options of the command or how does it actually work then refer to help. I think it's usually quite helpful. You can maybe show you yes and you can go into the commands and use TMT run minus minus help etc. So and there's quite a lot of flags we wanted to give as much options as possible so it's flexible. Okay so let's take a look at what it actually created. The TMT in it minus minus sent by mini command. Can you go next, Peter? Yeah, it's time to switch the window. That's a bit too much here. Sorry. No problem. So it created a simple plan. So that's L2 metadata which defines the script which should be executed during the test. So that's TMT minus minus help and now that we want to run the plan we can use TMT run command which by default uses the virtual machine for safety reasons. So if you want to run this then you need the TMT provision virtual sub package. You can install it also using bnf but if you don't have this you can also run it locally and that's by adding the provision minus minus how local to the command but also you need to add minus minus all to TMT run because otherwise we just run the provision step which is not what we want. We want to run all the steps also execute them and also if you want to for example specify the image which you want to run the tests in then it's possible by adding an argument to the virtual step. Okay but however this way of specifying tests wouldn't be very convenient because each time that you would use TMT run it would run only one test because this script is defined in the plan. That would be very inconvenient so this is where test metadata or L1 comes in. So if you want to define a new test metadata we can use a TMT test create command. There are currently two templates which you can make use of. The first one is pure shell and then there is beakerlib which is a shell library basically and if we use this command that they could look at the result is quite similar not too much more difficult it just defines a path to the test that should be executed but now that we have this we also need to somehow link these two paths that we created in the last two steps. So this is where the discover step comes in. The discover step basically defines how the tests to be executed should be found. So we need to specify that we want to execute tests that are in the local FMF3 so we use discover how FMF and that should be all. If we now use TMT run it's going to run the plan that we defined previously and during the discover step the test that we created will be found and then execute it during the execute step and that would be avoided for the interaction so I'm going to work to better with some more practical examples. Okay thanks Fanto for the interaction and for the examples. So now I would suggest you all open your terminals and let's start experimenting with the TMT directly. So what I propose we start with some exploration so let's have an example project let's do some experiments with the FMF so if you are a packageer and have the FED package to install the FED package clone FMF is all what you need otherwise you can do git clone and check the project directly using this. So that part I have already done I have this repository here let's see what's there there's just just a couple of files in the repo and we will do some exploration so if you run TMT just the TMT command you would see what is included what's included in the repository so here we see okay so we have we have three plans there's some smoke TMT I don't know what is it plans upstream so what can you do there is the plan sub command so you can you can do some sub commands some actions with the plans for example listing the plans or if you would like to see some more information some more details you can do TMT plan a show so maybe in the chat if I think it would be very good to to get some feedback for example any any one of you guys were able to install TMT did you have any did you have any problems with installing the package any any dependencies broken or any problems am I able to run container test from the portman TMT execution so it means running container uncontainer mirror I guess there are some possibilities to do that right they are this is actually a lot testing for an unseat but it doesn't work from inside TMT so if you have if you're running TMT without container and inside you will run another container that we do not support it but I think it's doable to do but I think the first container needs to be privileged so currently this is how actually testing farm runs TMT it runs inside the container but for TMT and so you can read from locals currently this is not possible and EGI I think you can file an issue maybe we can make it working for the for for the local execution for the execution in the automation I'm not entirely sure but because that would be three potmans actually there so that's quite quite a lot of nesting there so I would file an issue and we can take a look at it what we can do at least for the local execution I think that should work mm-hmm yeah I'll see where do I get it okay very well any other was there any anybody successful with installing TMT any any other other problems or questions try to do that if yes please please feel feel free to use the chat to ask any questions so so I suppose all you all have who is experimenting install TMT and you are able to access the repository you check the stuff here so let's see what we have so for example there's some plan smoke minimal test smoke it does some preparation it installs FMF and the execution parties uh it just verifies that the help switch works fine so let's let's see how actually the plan looks like so this is this is the minimal smoke test the summary is optional so just these you just need these two lines to actually enable on this this simple thing let's look about the others so for example the plans upstream this is interesting stuff Fernanda already commented that one of the one of the freedoms for tests is that you can store them on in a kit and then you can reference it from another repositories or from another projects you don't have to duplicate a test code and in this way it's it's possible to very very easily share the test code so for example the plans upstream runs upstream FMF tests so this means that there's a bunch of tests in the on the GitHub project and instead of copying them we just can reference them and during building building the Federa package the same set of tests can be can be run okay so that's that's the upstream thing and as we already mentioned in the chat TMT and FMF are very big very good friends and it might happen that change in one could break the other so we are running we are running integration tests here this is integration with TMT in the discover step we are referencing the the TMT repository and because TMT has a quite a lot of tests we are selecting only some which which makes sense just to confirm that with the new FMF TMT would still be would still be working let's let's see how how the plans look like so for example the upstream I would say very very straightforward you just say discover tests using FMF from this repository and we are selecting a specific F from the GitHub also that it's it selects the the latest bits which represent what's what's currently in Federa and finally let's see what's in the plans TMT very similar stuff you're checking the GitHub repository with this reference and we are applying the filter select just tier one tier two tests some additional prepare step which says we want to install TMT all so that the guest is prepared TMT is installed there and the test can be done good so that's about like some some first exploration let's jump to the next one let's do some real stuff okay so imagine you come to a good report and there is something you can see there are some plans but you would like to know so what would run if I if I execute the tests so this is for this as I said discover step is used to like check check the tests maybe fetch them from another repository or discover them from from the local repo so if you if you want to see what would be run you would say please run but run only the discover step this means TMT and discover okay so here we have the plan smoke just one one test from the TMT integration about 60 tests and some a test from the upstream repository if you wonder what tests are included there you can use the minus V switch for to enable more verbosity so that's about that's about it and you can you can apply multiple switches here there is no additional information but if you sometimes are interested about what happens under the hood you can enable debugging and the debugging will will tell you some some more information about what's happening in the under the under the hood so let's let's maybe we can discover just the plan name I will select the name upstream and discover with minus D you would see that we are calling some repositories and there are some checkouts and if you are adding the debug options there is some more more detail so if anything goes wrong this is this is the way how to investigate where the problem could be so this is about the discover part now let's actually run tests one more way how to do a dry run is using the dry switch so I'm here I would say TMT run but just let me let me show what would happen so here you see we have the we have the three plans each of them would do the discover part then the provision would be done as Ferranta mentioned virtual is the default which provides like the full the full functionality full guest with with all the capabilities but you run the tests in a safe environment some defaults for the guest then some preparation of the of the guests for example installing the the package FMF or TMT package then executing the tests once the execution is done a report would be the reports that would say inform you about about the results and the finish the finish step is using for cleaning up so that's that's the that's the example so we could do maybe we just take this smoke test and we we try to run it so let's see let's see what will happen Tim Tirana plan Naima smoke and I like at least a little bit more so so virtual machine takes some time it must boot then we need to install the packages through the preparation so this is probably not not one of the fastest ways how how to check testing one one tip one hand for you if you do TMT run last means like the last run follow it would show you what's what's happening under the hood so here we see some synchronization was done but now during the preparation I think the NF is now fetching the metadata so this will take some time and we will we will see the updates updates here it seems the metadata are very slow today maybe the network I don't know see yeah yeah we see so further 35 so it's it's fetching stuff and it will it will take some oh this brings us very quickly to the next option so we have FMF it doesn't need the full virtualization so what about running the running the plan in in a container that will be faster so yeah that's this definitely is an option let's see what's what's what's here oh okay so we have we have finished everything so here we see the preparation was done execution one test past and that's the that's the summary of it of it all okay so let's let's try to be a little bit faster so we will say run all steps about the provisioning step but the provisioning step be done using container provision how container and that's it and that's it so instead of provisioning the virtual machine we will we will we see the container was was much faster and here if we connect if we connect to the run we will see that again we are very slow on the on the fetching that you have metadata because of installing the package so okay let's let's make it faster one one more tip for you I'm when using the containers frequently I'm I'm doing a cache like I actually prepare a container with fresh federal metadata so that so that this installation and that stuff can work much faster so let's see if we do this same but check the federal fresh image let's see so preparation and we still do not see what's what's under the hood so let's I would say let's let's turn on some more debugging outputs so that we see what's what's actually happening so it prepares the preparation much faster it installs the package okay and once the installation is complete test is executed and we see the results execute some preparations here's the output and finishing I'm not sure what portman why portman container stop takes so long just stopping the container but fine so that was an example about about using a container and now finally if you feel safe if you have some tests you you know what they are exactly doing you can you can do them you can run the tests directly on your on your laptop the provision method is called local so that would mean here I would say instead of provision how container I would say provision how low so that was that was it very very fast way out to help around the best so I was using double verbose here so here you would see direct link sorry direct link to the to the log so if you want to get it you can you can put check like that okay so this was fine or you could do minus v you would see the output of the test directly so so this is this is a very easy way how to how to quickly execute tests you just come to the repository you just do the drama you select the right provision method which is which is good for you and that's it so I would like maybe ask anybody anybody was able to execute the tests from the git repository or do you do did you run into any did you run into any problems does it work for you maybe if you if you can just if you were able to do the tmt discovery please just paste in the chat tmt discover done so that we know that actually things work it doesn't make sense to continue unless the very basic stuff doesn't work for you oh boon to some experimenting okay nice yeah containers yeah okay no issues great thanks for the feedback so I will jump to the next slide so we thought it would be good actually to try some really tangible stuff we think that tmt makes it very easy to enable tests for your component so we now propose to take your project your favorite project it can be the package you maintain it can be another package or if you do not have any any any package so we propose you can you can try did which is which is quite a simple tool used for reporting what what happened for example how many issues you were able to fix on github or in bugzilla or how many how many poor requests you created on github and such stuff so did is a simple project we which you could which you could experiment with so let's let's maybe paste the paste the link into the chat so this is this is the does the work does the link work actually I believe okay so that's that's the project you can clone it and start experimenting with it we will be doing the examples on it so first thing you can use git clone or fat package clone did and and change directory to that component and let's let's start with checking out some some example a branch for for testing so I will be accessing my the repository here and let's let's do the git get out branch one new test for example uh switch the branch and then what was franticical reshowing the tmt init command can be used for quickly get started so tmt init minus template mini minimal tmt init minus template mini minimal and here you see uh there was a plans example created let's see what's there plans example uh so this is just a populated example with the with the with the script minus minus help of the component okay so we are we are testing did and we would like to make a simple simple smoke test for data so I just I just change change the command and maybe I don't like the the example I am so I move it to plan smoke fmf now if I check what's around okay there's the plan smoke uh fine um plan smoke very good uh so what do we do uh use the minimal template and that's done we can rename it we can adjust it and now the next step is to create to create the pull request actually so if you if you if you are uh following us please follow the link to the rpms did and here uh there would be a button which says fork you click on it and it will create your fork and now from your fork you would just access access you would use this this link for ssh and you would add this as a new remote to get so if you are if you are following us please make the fork copy paste the URL from here and the step for adding the remote is kit remote add fork and this uh and this uh and this URL so for for me this this would look like like this but I already have it there so get the fork and and that's the that's the URL is there so once uh once you have this done uh we can and continue we can continue I'm looking if we have any this project has no been for because that's not the original do we have any successful forks there okay I see we have radic money successfully forked uh fine okay so uh so some new files uh and uh I would like to I would like to create a pull request uh which would enable enable them enable the test so uh I would add the dot fms this is important because it uh it marks the root of the metadata 3 similar to dot gate which which marks the root of the grid repository and I get plans get add plans and okay so look about it for the fmf version and plan smoke mate uh enable a simple test push uh fork the second fork which is the same upstream and the name would be one new test okay so let's wait until Peggy responds with the URL for creating the pull request open the URL enable a simple smoke test create pull request and now let's uh now we can just just wait until until uh there are some results uh I actually did it from from after which is some of some old reference so we will see what will happen actually but uh we can maybe until until we have some some results in the pull requests we can we can continue with the examples so uh okay fine one thing which was mentioned by by front I think very very good point usually uh you do not want to write all tests in a single file but when you have a component which is a reasonable number of tests you would rather create uh separate tests so maybe separate directories for each test with the this metadata and with with the scripts which they are implementing so uh this is this is the way how to go uh would be to create a plan on the base template um and then create individual tests so uh I will check out the the row height the right range actually because uh this this is the this is the latest what we have plans upstream uh is the actual content of the latest row height so um what do I do now is I create the sorry just just let's let's see what's there so this is this is running uh some upstream this tests uh yeah so this is what you already know but we are now going to create a new plan tmt plan create and we check the template base uh missing name okay plans and it would be it would be test for individual plugins plugins so maybe just just to give you more tangible stuff what did uh what did command does get uh for example github so it would it would check what what happened during this week in github it would show you the issues which were closed pull requests created uh very very very useful tool for generating your reports uh this very very uh stuff which most people do not do not like to do it and it can take care of this so um and it has uh individual plugins for access in github bugzy logitlub and other other places so so you can do this at the end of the week and it will print over what happened during the week for example so um I create create the the plan the plan is there it is called plugins and it says uh okay we will be testing uh test individual plugins it would discover tests from the local git repository using fmf and it would execute once this is done I would create uh I would create a new test tmt test create um it would be called let's let's create a test for bugzilla uh so tmt test create tests bugzilla you can choose it whether it should be a shell simple shell test or big clip like big clip logging and some other functions features which are which are useful so I choose this one and then change the directory of the test bugzilla and here you see what's what's created so um there's some summary verify reporting bugzilla reporting and this test it would use big clip okay uh so that's about it uh the test looks like this this is some skeleton used by default uh it provides some setup test and a cleanup what we will be doing is just checking um checking the config um I will do maybe something very fast here um for example instead of doing uh okay so um let's do something very fast right now so it has this feature this option test which uh it's with each does some checking um and for for the for the changes which are on on github we don't need to create a directory everything from from this actually is not needed this is from some more complex tests I would be just very very fast now with this this example um um big clip test is there um now let's see how things would work together tmp around discover minus v see that the plan plugin the new plan would do the discovery here would be the test bugzilla uh and uh it would be it would execute the stuff so um maybe um I'm just checking the time we have about 20 minutes uh until the end so I guess I will I will not uh deep dive more here because we have some other other stuff still to uh still to go through so maybe uh okay so that's that's the example how to uh how to create multiple tests in the in the repository and in the same way you would do for example tnt test create a bugzilla uh it would be test for forget how tnt test create you know template um big clip uh let's top and and in this way uh you would um test uh for each individual plugin uh and then in the same way as we did the first smoke test we could enable we could enable the uh yeah test for the individual plugins okay so that's um that's the example of the more uh real life scenario when you have like multiple steps in the in the repository maybe let's see if anything is happening in the pull request okay good scratch build is done and this test is running so let's see what will happen and now we have here one more slide uh which is uh for middle I'd like you to reproducing issues thanks very much better so with reproducibility we have a vision that failures and errors should be easily reproducible from our local host right this is very important for the good experience so for that we will be uh showing the snippet in testing farmer results which you can paste in your local host and easily rerun the test in the same environment as they run in ci we have a first mvp version prepared for defconk where we are showing the snippet and it's available on fedora ci and cento stream ci because like the empty test are available in via multiple environments right so here for fedora ci and cento stream ci we have the snippet available and with that snippet basically you get the same code that the tests run for the same context just need to check that the screen is too small for me so you get the same uh the empty context environment variables image and the same test code which runs in testing farm uh yeah better is here showing the reproducer currently is quite long the this is an mvp and we hope to get a habit a lot shorter ideally with one tmd command right currently there are certain steps which tmd doesn't do for example cloning the test repository and also preparing a little bit the environment uh so so that's why there's there are multiple steps here but we hope that we will get to one command uh later on here you can see as as as the snippet provides basically how it will provision a fedora search for machine so this was actually uh this was a test from packet uh from the hello world i think yes there is the the github packet hello world uh at the cloning at the beginning yeah on the next example i think we have an installability test right installability test this is from fedora ci so installability test and all the generic tests in fedora ci are written in tmd so currently the mt provides there a very easy way how how actually users can extend generic tests on top of on top of fedora ci and here basically the test we run this exactly the same code uh that uh that the installability test is written in and uh it will are testing on fedora roll height so it's very similar experience the third one is special because it will not run in a virtual machine but it will run in a container the rpm inspect test is a generic test which has how container inside the the test called the fmf file so they always see i here is actually maintaining the rpm and rpm inspect image which which they provide and they handle and they run inside some scripts which do the job basically all the information about which test is being run is under that tmd environment rpm maybe if you can open that thing with the curl better that is the that is the file which the testing for the service generates and this is what actually fedora ci passes to testing farm so they know exactly which which component is being tested by by rpm aspect so this is the way how how how basically the user who is calling the testing service can uh can configure parameterize the generic test so it does what they need right so this is something which we should which was one of the dreams like uh when there is a fail in the ci we would like to provide users with a super easy and comfortable way to reproduce the issue so this is one one next steps step into this direction so thanks a lot middle for working on this and making this working we have somebody joined so sure if he wants to jump in maybe there are some questions or maybe he's just an expander right okay uh just a quick check uh when one shall be finished at uh 10 50 or 55 how much time do we need uh do we have uh you can finish uh right at 11 because there is a half an hour break till the next session so it's fine if you end like right at 11 no problem okay good very well um okay uh so we have still a couple of slides uh to show some some useful stuff uh for example one very common scenario is that you have um as we as we yeah as we saw with the virtual machine sometime that the setup can take some time so for example you really have a very simple test but it takes really too long to boot the machine fetch the dm at eight times without the packages and that that stuff so one of the common scenarios you want to we want to support and it's it's already working although we we are working also on improving the user experience is that you you can select the steps which which would be run for example run everything until execute so i will i would try to do it for example what do we have here so plans uh plans upstream let's let's say upstream tm tm uh and uh until until execute uh and i select the plan by name upstream so let's let's see what will happen here uh so a virtual machine would be booted uh the package would be installed there uh we will see what will actually actually happen and this takes some time uh then uh um for example you execute the test and then you find out oh okay so there was something there was something wrong and i want to execute the test again but the machine is still running the guest is prepared everything is there so you just you can do just oh okay so let's let's just call the execution once more and even if it is finished please force it like re-execute the the step once again uh because um maybe there was a net for glitch or for example there is a package missing on the on the box and i would like to uh i would like to prepare it so maybe let's see what's what's happening under the root then we can run last below so we are just yeah we'll see this will be installing click on the guest which will take some time um so let's let's give it a few uh seconds hopefully uh so as i said i can say please run on the last uh on the last uh on the last run which was which was executed before this is this is the minor minus minus last uh do the execution once again or one of the supported sub commands is login this is very useful and this allows you to look into the guest and maybe do some adjustments uh it's possible to log in actually during okay we see installation is underway so hopefully in a in a short time there will be something on the guest and running uh okay the test is uh the test is running some upstream two tests two tests failed okay and maybe we could see what was the failure oh did command was not found oh so this means that on the on the guest there is no did install we we forgot to add it to we forgot to add it to the plan so now let's see if this will run that was a test back yesterday so let's see if it will work today tm tiran last login okay uh we'll get to the machine okay there is no did okay that's fine so let's install the package it will install it there uh go outside and now tm t run last and execute again if i would just execute it would say two tests failed i have already executed this step so i no more point to execute it again but we know we want to execute it again because there was some change on guest and here we see the progress one test second test and the result is oh nice two tests passed very good so in this way uh you can you can do uh you can do this do this stuff there uh sometimes um you or very frequently actually you want to change the test code and then rerun the tests again uh which would be which could be done uh using tm t uh run uh discover so maybe just i will just now uh not not show so show this stuff but if you apply the test code change you would do um force the discover step once once again so this like discover test again so use the fresh test code and execute again would allow you to do this change very very quickly and save a lot of time so that's one of the scenarios we support and maybe let's let's go further to maybe show a couple of more couple of more examples uh so um one of the motivations to create tm t was to get rid of the old beaker uh make files uh for those who know them they are really really ugly stuff um and we wanted to provide something very very much more readable uh so for those who do not know it's it's possible to take take the old make file and just use tm t test import to convert it so just quick uh i can quickly show we'll get tm t we have there are some examples for for the conversion let's see here's the make file it looks like this and i do tm t test import and let's make it faster without nitrate so it was before and this is after you just a very very human readable gamma with all the information uh the nitrate integration allows to import some additional uh okay so i don't have i don't have uh i don't have initialized let's see now tm t test import and it should contact migrate and it would bring all the details from the nitrate test case so and there are some some more more stuff there so you would see that there are some links related bugs adjust rules some additional information fetched and now this is this can be stored in git and you're done we would like also to help people who who used sundar test interface tests for for enabling tests and on the documentation side we have a couple of we have a couple of uh examples how to take the old uh standard test interface test for example like this playbook if you want to run just a simple smoke that was the original form for standard test interface these two lines are now for tm t much more easier much much more much more faster required packages uh yes and previously it would look like this so here here you can get some some inspiration some examples for inspiration how to achieve what what was in uh in the sti how to do this in tm t so these are some examples which can help with this um and so that was it would be about the migration of some old tests and maybe just very quickly mention some additional comments uh commands which we have uh which haven't been mentioned uh still so tmp status and gives you some information about the runs uh were previously run and what what is their status so this is uh this is checking all the runs which were there before you can choose to tidy them up tmp status minus minus v some more information like maybe you could be running multiple jobs on your on your guest testing more things so in this way you can see what what is already done and what's what's in progress uh tmp clean uh it's a quick way to do the cleanup tmp clean the runs for example uh which would uh which would remove the runs but there are maybe uh some abandoned machines right status abandoned abandon abandoned there is some abandoned machines i was actually running the virtual machine which which is which is still running so maybe we could clean up the guest uh before tmp clean guests and let's let's be very gross now so it's it's stopping some guests in some older plan and maybe this will be something which already is not running let's see so that's that's about the cleanup uh tmp link uh can be used for uh making sure that this metadata are okay and the tmp stories can be used for tracking implementation test and documentation coverage which could be useful which could be useful uh and we don't have unfortunately enough time to show uh all of that and now we have a couple of news what happened since last year uh maybe we can you can show this slide or yeah of course so like the last year we were able to finish the zoom integration so for zoom is used as a uh ci system on top of poor request or merge request in federacy and also zento streamline so that is available there and if you drop tmp tests into these kits enable zoom then zoom will automatically run tmp tests nicely and provide it in a similar fashion as other checks uh also what happened in terms of the features so we have now support for reboot during the test execution so it is supported that the test can be rebooted and it is actually made in a way that it's compatible with restraint looks very important for redhead folks which rely on this feature uh yeah almost compatible we have a back there but yeah soon it will be of course um so we have also a way how to parameterize tmp plans from environment variables but can be useful uh from ci and also not to have hardcoded there's some data so this can be parameterized nicely thanks uh we have integration with uh workflow tomorrow what's again another legacy internal tools uh sorry the slides are the way yeah so we have we have a way how basically you can export them uh tmp test uh and and uh and we then fmp by the end the workflow tomorrow can steer on it this is important for some fluent transition from the old way how we run tests in cyra also there is possibility to discover test from sources this is great if you that you can bundle the tests together with sources as you are releasing your package and uh actually tmp can discover the test from those sources so you can lock them down the the test code can come together with the uh with the our source rpm can be bundled inside uh and and and look it down like that uh then we have tmp clean tmp linked uh implemented to clean up the the resources tmp created and linking uh linking uh the the tmp fmp files uh we have support for exiting after first failure so if you don't want to continue to run uh when first test fails tmp can exit if you ask for it and there is a nice progress bar implemented also there is a there is there is a lot of documentation improvements for example there is the second chapter of the guide under the hood and we document it how you can easily migrate for the xti format yeah so what we are what we are planning very shortly because we had just one minute so uh we will be extending and the documentation that's an ever-ending story well important thing that is happening now that we are implementing multi-host support so we can write multi-host tests using tmp and very big feature will be that you can be able to specify very rigorous hardware requirements for your tests and uh the underlying ci will support this also we will be completely revamping how ci and testing farm the service insurance tmp test works with tmp so it's all aligned and works consistently so you will get this nice experience for reproducing so you'll be able to run the same thing as ci runs on the from your local host uh that comes with improved debugging at usability so for the common use cases we would like the workflows to be clean when we are debugging developing tests and implement a lot of nice ideas that we have out there and fix a lot of bugs that always come with software so if you have some ideas and or found some bugs definitely use our github issue tracker to file them or if you want to contribute there is a good first issue tracker label so you can use that and you can find us on the tmp libera chat or internal redhead irc contact us if you want so thanks very much for joining today all right thank you very much guys uh thank you Miroslav Frantisek and better and thank you everyone for um attending the talk um please remember that you can go to the work adventure and continue discussion uh i suggest you to use session room number five which is in top left part of the map um so feel free to continue discussion and uh don't forget about the coffee tasting which is starting right now at the stage thank you very much have a good day have a good day