 Thanks for joining. I think you're going to speak next, right? So I think this is the first session in the DevOps track today. So I am trying to speak something related to a technical practice that's been ignored for quite some time. When someone speaks about code coverage, teams always related to unit testing and more from a developer standpoint. So what I am proposing is, why can't we use code coverage as a system testing level? For the last 15 plus years, I've seen it for the last 15 years that when someone says testing you sit before the screen and start testing it, but there is no scientific method to help you decide whether my test suite is comprehensive, what kind of technical debt I'm creating in the system. There is no traceability. When people speak about traceability, requirement traceability matrix, power, and backward, one key piece missing from a system testing standpoint is the relationship between the code and the test cases. You have your requirements and your test cases traceability, but how does that relate to the code? We don't have a strong relationship with that. Most of the times when someone wants to do a regression test suite, we go by the gut. This is what we have fixed it, and we'll try it looking at it. So there is no way you relate it to the code and say, yeah, this is where we have made changes. These are the possible scenarios that need to be tested, and how are they related to the test suite? So what we are trying to propose is a first step process, you instrument the code the same way as you do it for unit testing. Enable the system to capture the test case coverage at the system test level. Merge the, there are multiple testers that you might have. So we will have different ACR files. We will merge the files and generate the report, and that report will give you a better perspective of do we have dead code in the system, are we missing any test cases, and also to decide what kind of test cases do you need to select for your future regression test suite. Almost every code coverage tool, if you are using code coverage tools at any platform, they have an instrument and process. So there is a sample like when you want to use through and script for Covertura. So you have a process to instrument the build, and then to generate the instrument and classes. So you can use those, almost every code coverage tool gives you the set of instructions that you need to follow. Follow those instructions, you will be able to create an instrumented build and let the teams test on those instrumented build. So once you have tested it, every system level you will have specific ACR files, and teams might be testing it a particular module, not the across the entire application. So you will merge all of them and you will create a final dot ACR file, which is an script for that, like you can decide where do you want to store it and what are the different files that you have, and then you'll be able to merge that. Then you create a report. So if you're using Java then the palm dot XML in the build section and in the report section, you have specific instructions that need to be updated. Once you upload that, you'll be able to create the reports in both HTML and XML format. And then you integrate it to your dashboard. So your Jenkins report will showcase how is that report looking like and you can have your, like whether it's red, amber, green, basis on the number of lines or number of branches that you want to cover and that can give you a representation of the code quality. So this is what the report that I was speaking about. My experience, I've been doing this for the last 10 years. I've seen that like first time when teams use it, they didn't have a clue that this is how the report looked like. They had no, like after the test study, they feel like they have enough test suite. But when you look at the report, it gives a clear instruction that we are not there yet. So then once the team use these, then they went back. Obviously 70% of the teams don't know how to script, so they rely still on manual testing. So when we enabled test coverage at manual testing level, then the team started looking at it. Then obviously they took the help from the development team to understand which of these functions or packages need to be covered. And then they went back and there's a better collaboration in place because obviously they didn't have, they were looking at it as a white box. So now we are trying to mix the concept of white box testing with block box testing so that you'll have a better representation of code quality across the entire application. So we are looking at if we can make it as a practice across teams to create something that will help you devise a mechanism to looking at code quality scientifically instead of going by the gut. So that's the practice I have, that's what I wanted to propose. So it's a small talk, it has more to give a perspective of how do you want to look at it and I'm open for questions. How do you do your test suit execution in your teams? Do you see a need for code coverage? Test driven? It's a very small talk, it's done in 10 minutes. It's pretty simple like I'm not going to, I still have another 15 minutes so I can do the same stuff again. So when someone says code coverage, what does it come to your mind? Yeah, so generally when someone says code coverage people always relate it to the unit testing level or mostly from the development standpoint. Why not at a system testing level? At a system testing level, why not testers use code coverage? The general aspect is we always feel that there's a tremendous progress in the way we are developing things but if you look at the way we are testing it, it's still old school. There's too many, it is advanced of the community, they wanted to do a test driven development, behavioral development, but we still don't have a scientific method to say that okay, I have a comprehensive suit in place. The test suit, how do I know that I have enough test cases in place? How do I select my regression test suit? How do I know that there's a dead code in my system? Until and unless I have tested it and see that this part of my code is not been covered, then when I do an analysis, I'll get to see that okay, this code is not covered because I don't have test cases for it or this code is not covered because this doesn't do any functional but it was written wrong time back, it's part of the legacy code and we are not using it. So the system testing, when I'm doing it, if I can enable my system to track code coverage, obviously I'll be able to address those problems. So when you're doing instrumentation, you'll not just do instrumentation that will only track your unit test cases, you will instrument the code in such a fashion that your manual test execution will also be covered. Once you do that, the testers will go ahead and test it and then there is a merge process because you have 12 or 10 testers or three, four testers who are doing it in different systems. So every system will generate a ACR file, a serialization file. So you collect all those serialization files and using a palm dot XML, you integrate all of them and create a final dot ACR file which will create a report. So that was showing on the screen. If you have a code coverage tool already in place, then it works like Java has it, .NET has it, Python has it. Like anyone who's working in these stuff, for example, any team that's using code coverage for unit testing level, they can implement these practice. Yeah, palm dot XML is basically the build one, that's for Java, but Covertura is the one you'll speak about if you're speaking about Java, EnCover if you're .NET stuff. So depending on the technology you're using in, there are multiple code coverage tools, right? So the whole point is if we can make something scientific, do you have a, I have a, I can use this board, right? See, I'm saying that imagine that you have a matrix. You have functions, one, test case, one, test case, two, test case, three. If you're able to create, like you, once you have this test suite in place and you're able to do that stuff, you have this matrix in place, and you will know which test case is covering which functionality of that stuff. Now you are in a situation where you are doing a regression test suite. How would you select test cases for that? You know that when you are fixing a defect, this is the one that I have worked upon. And I have worked upon this, I have worked upon this. Now once I have this matrix in place, I know exactly where these two functions are executed by which test cases. So you will be able to select test cases for execution, which will hit upon those functionality. And the other stuff is you also know the in and out of it, like what is providing input to it and what is providing output to it. The other functionality is right. So once you have that entire view in place, you'll be able to create your regression test suite accordingly. It can be for anything. Yeah, it can be. You can use it at unit testing level, you can use it at system testing level, integration, at any phase. At any phase. Yeah, yeah. You can create these patterns. Yeah, obviously the teams are using it more predominantly today at unit testing level. Exactly. So but can we take it one step ahead and make sure that we create this kind of stuff at system testing level as well. So does that answer your question? If it's a one-one talk, then we can always have it offset as well. I can give you some more additional details for the same. Right, and most of the test covers tools are available. It's only a matter of convincing the organization that it's important to look at the test suite, comprehensiveness, then experimenting with it on the gut feel itself. And the buy-in is very easy. It's not costly as well. It's not, like most of them are freeware. You can actually use them, see if it's working for you, and then upgrade it to their enterprise version if you want it. Like the Clover can be something that you will buy it, but Covertura is available free, so you can actually use it if you're in the Java stuff. So there are so many tools which are already available freewares. You don't have, you're not investment. It's only from a company policy standpoint, from a security and approval standpoint that will allow you to use that. But if you can create a business case why it is useful, and you pilot it and you see the results, then you can look at an organization-wide implementation. There was something that Raymond spoke to me yesterday. It was like the firmware and other stuff. So obviously when it's a firmware, we still don't have that kind of capabilities. So it's not useful everywhere, but primarily in the technology space where you already have Covertura tools in place, we can start from there. And obviously I'm delaying on OpenTech. When they see there's an opportunity for this, they'll obviously create more solutions around this, then that will be helpful. Thanks guys. Thanks for your time. Thank you very much. Thank you.