 Yeah, so yeah, my name is Cisco for Lee I work for the document foundation and Yeah, I don't I didn't know how to title this Talk so I just say that's coverage in liberal office. I'm gonna split So few years ago I Decided well like I found that we had Sometimes fixes in liberal office that didn't have a unit test Along with the fix and I found after being involved in QA for some years that Yeah, that we had regressions that we fixed and we even have a Regression fix and then the same regression happening again. So Yeah It the pandemic came and I had a lot of time and I Yeah, I decided okay, I think it's it's time to Learn how to because I already knew how to write some some test but for instance CPP unit test I didn't much Didn't know much about that. So I decided okay It's it's time to to learn it and try to Improve the test coverage of liberal office. So Yeah The problem I found is that well we have After the project was fork, then we had a long list of comics and It was for me difficult to find which which test which Fixes had a fix Sorry, which fixes had a unit test and which was which fixes didn't have any Any coverage or any way to to be tested automatically So then I started to think about that and I thought okay It's hard to For instance, we have some commits in liberal office where the Author of the commit of the patch just gives a Short explanation of the fix, but if there is no bug ID normally when you Comet a patch then you normally if there is a Baxilla ticket related to that you normally mention it in the in the commit message message You say TDF and then the bug ID of the ticket in Baxilla so yeah, I found that for For a third person like me trying to add unit test It was easier to find Those issues in in in Baxilla because normally you have the documents to reproduce the issue You have the steps to reproduce the issue So then I started to think about that and I came with the idea of creating a script to list All these fixes that didn't have Unit test assigned to them so for that Well, I just used the the lock of of Git So I decided okay. I'm gonna start a random date, but yeah after the fork of liberal office So I decided okay. I'm gonna take the log since the beginning of 2012 and then I'm gonna do it in reverse order why in reverse because sometimes we have Comets fixing an issue and after that Sometimes we have the unit test or the test covering the fix and the fix itself in the same commit but sometimes it's not the case and we have Maybe one after the other we have the fix and then we have the unit test or we may have the fix and after two weeks or one month or sometime we have the unit test so the way to to for me to test it was to know if Yeah, there was a unit test or not was to do it in in reverse order and then Then we look for the pattern and well the the script looks for the pattern So normally nowadays we use that the pattern the TDF pattern But in the past we also have this this FDO or bug or LO and then for each Comet we check the Modified files in that commit so LibreOffice Structure in folders it's in a way that Every module we have for instance SWU for writer or SC for calc inside these Folders we have a sub folder called QA That's common pattern in all the folders where all these Unit test or yeah with the well all the tests are in there. So I thought okay I'm gonna look for in every commit for for the modified files and then if there is QA in one of the Pathies of one of the modified files, then I'll assume that there is a Unit test for for for that fix and if not then we can Yeah, we can assume that there is not a unit test for this fix and after that We go to Baxila and we check with the rest API Some parameters like okay, let's check if the bug is Close or fix because sometimes There is already a fix in the in the log in in in in git But maybe the developer will I add the unit test after after that and in this particular case because testing performance Performance issues is kind of difficult. I decided to avoid these Issues that we have in Baxila which have the keyword perform perf with means performance and then once we have the list of of fixes that Doesn't have a unit test with them Then we just the output we use the media markup for that and then we Just put it in or in my case I just put it in this website which I'm going to show you and The reason I say it is good for history. It's a well and let me show you So, yeah, I'm hoping it many times. So basically. Oh Now I don't see it here Yeah, so basically this is the Wiki page that the script creates So, yeah, basically it says the there's the this report is created by this script with which is in in the LibreOffice Core then VIN Slash check missing unit test and then when this report was created the number of comics that you read it there Yeah, so the number of comics that were analyzed since 2012 Then the branching master and the the the last committee that was analyzed. So basically Well, what I do is to group the the different missing unit test by parts of the of the code, but now you we have a list of Missing unit test so and then you have the date where the commit was done the Priority of the of the bug and then you have the description of the bug and also you can have a link to this ticket so then if you want to Try to add the unit test for this. You just come here to Baxila. You read the steps. You have the the files there so, yeah Basically, the idea was to have a place where we can list of the all the missing test that we don't have and try to you know, like shorter the the list because for instance if we go to Yeah, like right there. We have for instance forum do we have 36, but then like we have many It goes really long Yeah, so Just to be clear It's not about all the the code in LibreOffice I'm just trying to pick some parts like SWU and then Sorting them by the path of the of the Files that were modified in the in the fix to group them But they are still missing parts here, which are not because sometimes it's really hard to to have a test for I don't know and and I'm also excluding some parts on purpose because there are some fixes like okay I want to change the label of Dialogue then I don't think that makes sense to test. It's just a label and it can be changed any way any time So yeah back to the presentation You see it now, yeah So Yeah, then now that we have this list Well, if you want to write a test for Specific fix then some ideas then you can check Because you know, you you you have the fix of the issue and then You have the fix of the issue so you know which Files were modified to to to fix that issue then you can check the log for each of these files and see if other fixes Containing a unit test are already in the in the log so you can already know. Okay. I know this this fix In a similar in in the same file containing this unit test then Probably the unit test can be similar. So you already know you have a Starting point then one question to to ask yourself is okay Do I need to write a UI test or CPP unit test? Well, we have other ways to test also in liberal office But right now these are the main ones and for a starter then it's Well, you I test is Basically when you are doing something with the UI then You should write a UI test. The problem is that when it's running Jenkins. It's only Executed on Linux and then we don't have it executed in on Mac or Windows and They are slower than CPP unit test, but then in other cases other than UI then you should normally use a CPP unit test and we already have a bunch of test is test as examples, so then it's a kind of That's not that difficult to to write your first test. Yeah Can you explain in a sentence or to? how tests which Consider the rendering output like what appears on the user on the user's Application screen can be written when it's not, you know Numeric output in the file or something that you can easily compare in code Yeah, so Basically when you let's say open a dialogue in liberal office You are using a Juno command. So with a UI test framework You can just call this a unit command and then it will open the the dialogue and then you can just get the top windows as a As a variable and then you can Examine it like okay. I have this widget like I have a text text box So then you can get that text box and see the properties of of it So that's the way. Let's say you expect to have a text box say in table one So then you just get the properties of this specific widget and you assert that it's a The text in the in the in the widget is table one Does it? Okay, and I can explain you and then in well, I continue in in in many cases. Yeah, so ideally when a test needs to be written then the Yeah in a way to test that it's a Properly implemented Well, the the the fix should be reverted So if you write a test Then it passes and then you revert the fix it should not pass The problem is that we are checking fixes since 2012 and revert in a patch from like five six six years ago. Sometimes it's tricky and difficult but That's the the ideal Solution there Basically, I don't know nothing about test strategies like you explain now, but wouldn't it be wise to start Backwards in creating those unit tests or otherwise is it is it really a good idea to create? Something to test the bug fix which is let's say eight years old and has no regression since then So or can we just Check it and okay, it works and we don't need we don't need to create a test for that Yeah, you're right, and that's a good question and I forgot to mention before in the website here when we have the The list of issues Let's say I updated once every month Then I can come here to the history where the bear is to real see history and then If I see the differences between changes I Can see that here in blue These were fixes added in the last month without a unit test so now, okay because for me, it's easier to revert this fix than eight years old fix so that's kind of the way to try to Catch up. Yeah, exactly. So But anyway, I like it like I decided, okay some fixes from the past are also Easy to revert some of them like minor this is Fixes so I decided, okay, I'm gonna like show all the log But yeah, we could change like let's say We could analyze only the last three years or four years or one year would make sense It would the the list would be shorter, but yeah, that's that's a matter of taste, but yeah, you're right and Yeah, so in some cases the the issues are specific to Specific documents so in that case when we create a test for that The idea the ideal test would have a minimized document because we don't want to well if we start actually when when you create a patch, there is a Maximum size of per file of half a megabyte but yeah, there's the the smaller the document the better and then just If it contains the minimum information inside the document, it's it's better and then once we create a commit Then don't forget to to add the TDF and the then the ID of the of the issue So then the next time The list is updated. It will be gone from the from the list and Yeah, finally I have Some data here. This is about SWU source. So inside SWU source. There are few folders There is one for UI which I excluded here because as I explained Yeah, in in many cases those are cosmetic changes and well, I ignore ignore them. So since 2012 2300 more than 2300 bags were fixed and Yeah, we have a coverage of 61% and yeah, the idea would be to increase it and ideally Both both charts should go parallel. Yeah together. Yeah, exactly Yeah, yeah, so here that the blue one was going like steeper and now kind of the the orange one in the last year is going a bit steeper Then for SC we have a 57% of coverage Yeah, then SC SD sorry, it's around 50% and Then we have other modules which were the coverage. It's a really good 85% and I think Yeah The philosophy here was always to have a unit test and maybe it's because It's easier. I don't know like But it's really good here in this module or in this folder Same for writer filter. We have 92% coverage and Then yeah, like this one. It's Getting better since 2018 but yeah, still 41% and Yeah, those were just some examples. I didn't want to like Choose like all the folders in in in LibreOffice because We could be here for four years, but yeah, those were just some examples. I wanted to share. So, yeah, thank you very much And do you have any question? I don't have a question I wanted just to Give a personal thank you as a developer because having the unit tests that catch up helps us helps my work and helped me to prevent regressions and to and Answering a bit of the previous question well, of course please your use your time wisely, but if you have time to create unit tests for something from 2012 Please do that Any other question? I have one I have a small one you you mentioned that you should add the Kind of hashtag of the of the bug when when you create a unit test. This means that You're supposed to update on Garrett the Sorry on Baxilla No, yes on Baxilla when you only create the unit test you mentioned you you have to Do to hashtag the TDF the number of the bug where? Well, you have to do it in the git commit Yeah, so then once it the test is pushed to master then the Baxilla ticket will get a comment with a notification that there is a A new commit related to The ticket and also for the script the next time is executed then the the buckets gone from the list Thank you, I don't have a question either But I have a comment that we are also using this missing unit test page for mentoring So this is a task for people who have done Some beginner easy hacks and now are doing some medium level easy hacks and Cisco is also Helping as a consultant In that phase Yeah, I wanted to add that last year. I think or two years ago. We had a Google summer of code student helping with this Will to reduce this list, but unfortunately Disappeared in the middle of the project so but yeah There is what the elmarie say that it helps newcomers to Get involved as well with more Yeah, interesting task and thank you. Yeah