 This is the 19th lecture series of this part 2 or unit 2, here we study about the white box testing techniques. In the previous session we have understood about some of the basics of dynamic testing in terms of white box and black box. We need to have a strategy, test case selection methods and coverage criteria, these are very important aspects for doing the white box and black box testing. We also studied about the differences between the white box and the black box testing. Basically black box is a functional or data driven, it is purely based on the requirements and the different techniques are also based on the functional behavior of the system without any knowledge of the internal or implementation of the system and the test. Similarly in the white box testing the implementation of the code will be studied and partially the tester will have knowledge on the entire system but his focus will be fully on the white box design techniques in terms of the structure of the code. Also there is a intermediate layer called a gray box, it can have a mixture of both white box and black box where the coverage is not possible with the white box and black box as well. So we can have a great of both of them that is called gray box. In code coverage we know total number of executed code that is tested divided by the total code that is there in the system. So this is called as code coverage in percentage, similarly requirements coverage we have. This talks about the total number of tested requirements versus the total number of requirements in percentage. Also we studied about the black box testing and white box testing differences and their advantages and disadvantages as well. In continuation of that today we study about the coverage, the white box testing what are the types of white box testing. So the broad categories are statement coverage where all the executable statements in the program we need to test it because they will be executed. So we need to see that they are going to be tested and whether we are going to test it. So we need to have the test cases selected as per the same. Next one is a decision or a branch for it, here test cases are chosen because every branch it is called a branch also the decision. So the decision can lead into a true or false path so both have to be executed at least once so that is why it is called a decision coverage or a branch coverage. Similarly we have a condition coverage, so in this type of broad level coverage we have all test cases we need to force each condition in a decision to take on all possible logic values. Suppose we have a medical conditions available underneath the test so we need to exercise all the possible paths that the condition takes. If it could be a logic expression as well so we will study about that condition coverage in detail in the coming slides. So all this together it is called as a structural coverage, so these are white box testing techniques on total it is called a structural coverage which has a statement, condition, and there are different types that are underneath the structural coverage. In the end of the structural coverage we do a technique called structural coverage. So which is a method which does a coverage and how much coverage is done and how much coverage is done so the end goal is to reach 100% so that is what the structural coverage is called all together it is called as a structural coverage. You can see a table this typically used in industry as part of the coverage is confirmed so you can see different coverages on the statement coverage, decision coverage, condition coverage, condition, decision coverage, MCDC, multiple conditions coverage and multiple conditions coverage. So what are these and how these are covered in different types of coverage criteria so every point of entry and exit in the program have been removed at least once that means we have several program units or the function units so that function will get entered and the function will get executed after the function is executed. So whatever the possible ways of entering into that function are of course there and whatever the possible ways of exiting out of that function has to be involved at least by that program that sort of a testing is done with the help of decision coverage, condition coverage, condition, decision coverage, MCDC and multiple condition coverage. These are responsible because the function could have conditions, decisions, modified conditions etc. So make sure that all the entry and exit criteria have been taken care, okay. Next one is the program will have statements, all the statements need to be touched once all the statements need to be removed with the help of that testing method so this testing method is called a statement coverage. The next one is every decision in the program has taken all possible outcomes at least once that means we have a diamond blocks, decision box, we have a decision box which will be having a certain decision we can take a rule or it can take as this path can take as yes or no, yes, true, no, false. So likewise all the decisions in terms of covering this, this path, this path so these decisions have to be involved, these decisions have to be covered so those type of testing are covered in the decision coverage, condition, decision coverage, MCDC, multiple condition coverage. The next one is every condition in the decision in the program has taken all possible outcomes at least once that means the conditions that has come out in the decision should have taken whatever the possible outcomes that it needs. So that will be taken in the condition coverage explicitly along with the condition decision because we know that decision is possible in all. So the condition coverage, every condition in the decision in the program has taken all possible outcomes at least once it has to be accessed. Next one is every condition in the decision has been shown when independently affect that decision outcome that means that whatever the execution path that we have should be independently, that means the pre-conditions that is required should be independent. So that will be done with the help of MCDC, this MCDC otherwise it is called as a modified conditions decision coverage, decision condition technique is mostly used in aerospace and space industries with the help of a standards called the view on some of the, I will explain that in detail in the, which is called as the view on some of the standards basically. And this one along with the multiple condition coverage, we have this decision coverage criteria is met. Last one is every combination of condition outcomes with the decision has been known at least once. This is with the multiple condition coverage where the different several conditions because a program or the function could have similar or different sort of a diamond boxes and processing segments all that parts have to be executed with a several combinations of them, so that is done with the help of multiple condition. So this table talks about the coverage criteria, what are the types of the coverage we have. We will study each of them in detail, okay. So white box testing has the below types of testing methods, first one is being statement testing and the next one is branch decision testing, the other one is data flow testing and branch condition testing, we have branch condition combination testing, we have modified condition testing and the last one is linear code sequence and jump testing, okay. So decision testing, the idea with the statement coverage is to create enough test cases so that every statement in the source code have been executed at least once. And in branch decision testing, the idea with decision coverage, the idea with decision coverage is to execute every single decision code at least twice, it means we know that decision can be leading to a dual path through or false, both possible outcomes of the decision should be executed in order to reach full decision coverage, that means the decision coverage is being taken for branch or decision testing. Next one is the data flow testing, here what we do is the test cases are designed based on variables that are used within the code, basically purely this is based on the data, so data how it is represented with the help of variables, so all those variables have to be considered in terms of their usage and the flow, what are the values it takes, so that is taken care with the help of data flow testing. The next one is branch condition testing, a test case, this is a test case design program in which test cases are designed to execute branch condition outcomes. The next one is a test case design technique in which test cases are designed to execute combination of branch condition outcomes, that means we have different combinations of the branches, that is done with the help of branch condition combination testing. The next one is the modified condition law testing, here in this design technique test cases are designed to execute branch conditions outcomes that independently affect the decision outcome, that means the independency of that decision will be taken or considered in this modified condition testing. The last one is linear codes, sequence and jump technique, in this select test cases based on jump tree sequence of code, that means some codes like interrupts and those things are independent, so this consists of about 3 type of things, executable statements, the linear sequence end of that it will be there and the target line to which control flow is transferred at the end of the linear sequence, that means we have the start of the sequence and end of the sequence and the target line to which the control flow is transferred at the end of the line sequence. So we will study in detail about this type of things and the last one is one more technique also we will study, that is specifically followed in the other case with the help of D1 assembly standard, that is MCDC, so that also we can consider, basically that is having this modified condition decision coverage, that is nothing but the branch condition combination and modified function both together it is called, so we will study that in detail. Okay, statement coverage, so execute every statement in the code at least once during the first case application, that means it is basically a fundamental technique that every testing does not matter embedded, non embedded application, Java, whatever you call, all this type of testing they use this statement coverage, in this statement coverage test cases will be created, so that every statement in the source code has been approved at least once, so that is the aim of this statement coverage, so basically they use tools basically such as white box statement coverage testing tools, LDRA, Vittercast, loader and RTRT, a lot of tools are there, specific to the need of that particular industry, they use this and the coverage is as I said in earlier slide, earlier session, the coverage is taken into credit with the help of executed the number of statements versus total number of statements, that means suppose with the help of this statement testing coverage how we will arrive it, suppose we have executed total number of lines 18, 9 and total number of statements that are there, statements are nothing but the lines of code, the executable lines of code, this all the commands and everything, so in C if you consider every semicolon is a executable lines of code, also it is called as COC, so basically this total number of statements need to be executed in white box statement coverage, but it may not be possible to do 100%, so there are other methods that we will have to do it, like whatever that methods that we have seen like decision coverage, medical condition coverage and all that, because every path needs to be executed by its own technique, and the coverage is done with the help of percentage, here it is between 9%, between 9% is the statement coverage, so the 11% is there, the 11% we need to take care of, so the work flow when using the statement coverage, first I will give all existing black box test cases that has been created while monitoring the execution, that means if we have done the black box requirements testing, we can do that if possible with the help of white box and monitor what are the statements that have been executed, then this monitoring is in all but the simplest test cases performed with the tool support basically as I said, so when all black box test cases have been executed, the tool can report which parts of the code that remain untested, it means we know that 11% type untested, so now we need to construct new test cases, that will cover the remaining statements as much as possible, so we will start with the part of the code that should be reached, backward in the code to determine the values of the input variables required to reach the desired part of the code, then accordingly we will provide the inputs, whatever the value specific values that are required for that lines of code that are not covered, so how we can feed the inputs with the help of information, whatever it is being spoken about functionality or the variables if it is possible and the expected results also can be drawn out of the specification, so this new test cases also will be executed on the monitor and one thing we should not take care is we should not expect the result from the code itself, we should go by the specification and functionality, so of course the code should complement and not from code, so this is very important aspect, we cannot have because code can have a local variable or uncalculated value, so intermediate values, so we may get biases or it may be half cookable, so we have to go with the functionality for that piece of program, what is expected and accordingly we need to cook the expected results and try to see whether it passes so that is about statement coverage and as I said all the statements have to be monitored and executed on the track, so what will happen is with this monitoring technique or what is that called observation we want to have on the executable statements, it is not possible with the help of manual, if it is say sometimes we can do it but there are big, big embedded programs having 50,000 lines of code, what will you do, it is not impossible to have a manual instruction of that, so what we need to have there are test hooks that can be used something like a print up, we can add to see that every statement is executed, a print up will print suppose print up statement something like this statement number and every lines of code we can have this statement number implemented accordingly, we know that what is that being printed and if there is a missing thing, so we know that how many statements have been missed but the problem is here we cannot have a print up done by ourselves every time, so that is why we have a hands on to cover this we have a tool called instrumentation, this is called this technique is called instrumentation, what we do is we will provide the monitoring ways of observing how many statements have been executed or what are the statements that are missed, so that report will be done with the instrumentation, where the existing lines of code or the code the program will be instrumented, so the tested part will be reported, so for this instrumentation there are a lot of tools, we will study about the tools, the tools so what it does is basically we will create all the source files and it will add the instrumentation code into that and execute on the target of the host depending on the nature of the embedded system. There are other challenges that I will speak about that where the add instrumentation and it will add a lot of lines of code and the total lines of code will increase because the TKLOC we have and the instrumentation is suppose 120K it will become 70K and which may not fit into the target system, so what we can do in that case is we can do partial instrumentation and take the credit by overlapping all of them, that is how the instrumentation of statement testing is done, okay software instrumentation, software on the measurement methods are all based on some form of execution logging that means logging of the execution or the monitoring of the execution is required to do the statement coverage. The implication is that after the black box, after the block is entered every statement in the block is executed by placing a simple trace statement for the printer at the beginning of every basic log you can crack when the block and by implication all the statements in the block are executed is what I explained. So if it is the RTOS, the RTOS itself will provide a logging service with the help of we can do it but not all embedded systems will have an RTOS or facility with RTOS to trace it in terms of the logging calls or memory in the target system, so we will use the instrumentation with the help of the tools, external tools. So let us introduce you form of execution logging model called low intuition printer because it does not introduce much into the existing program, a simple memory write is used in the place of printer. At each basic block entry point, the logging function marks a unique spot in the excess data memory, after the tests are complete the external software correlates these marks to the appropriate section of the code so that the code whether it has used or not will come to know based on the memory aspects. Alternatively the same kind of logging call can write to a single memory cell and the logic analyzer is another tool as I said for instrumentation and statement coverage they use it, so it could be a hardware interface such as logic analyzer which will capture the data from the memory and analyzed. So that is how the software instrumentation is done and the challenging part of the system is that if the system being tested is ROM based and the ROM capacity is close to the limo the instrumented code image might not fit. As I said if there is a 50,000 lines of code is there and instrumentation after we are done with that it could come to something like 70, 75K and whereas the embedded system has a memory of 75K definitely it is in a border or it may creep more than that. So what we can do is we can improve your statement coverage by using two or more rigorous coverage techniques sorry we can improve that by the help of overlapping technique and statements we can tell you partially and the further testing or the statement coverage we can use with the help of decision coverage and we can say you see modified condition decision coverage. So that is how the software instrumentation is done there are various tools we will touch base some of the tools as an example at the end of this session or in the next session. In going in detail of the statement coverage you can see there is an example to achieve a statement coverage every executable statement in the program is removed it is once during the software testing we know that achieving statement coverage shows that all code statements are reachable in the context of the B1 standard B reachable BN zone test cases from the requirement. Here we use either a design or the requirements to have that reachability aspects that is what is spoken in the B1 standard B process standard. So for example if X is that one one and Y equal to 0 then Z will become Z by X and if so this is one statement so by choosing X equal to 2 Y equal to 0 and Z equal to 4 as input to this code segment every statement is executed at least one we know that there are two statements these two statements how we are going to execute is by providing such values. So it will enter into that then portion only when FBX is greater than 1 here we have chosen X as 2 definitely it will go inside this but there is one more condition called and condition so this logic expression into satisfy by putting Y at the way as 0 so when so Y becomes 0 and X is greater than 1 that is 2 so Z will become Z by X and Z is provided as 4 so Z will become 4 by 2 this 2 however if an R is coded by mistake in the first statement instead of an AND the test case will not detect the problem that means we will not be able to get the issue that is here because the implementation is wrong so this some sort of a summary goes purely by the test case selection on code understanding the code is wrong then it may be difficult so as per embedded software expert mayors statement coverage criteria is proved that the general generally considered as useless so at this statement coverage should be considered a minimal requirement that means a sort of statement coverage we need to do to have a confidence that the maximum statements have been reached or statements have been in work so in that way basically we can use this statement coverage statement and rest of them we use the other techniques further elaboration of the statement coverage you can see there are about 5 statements in this program here is a program block within the flower bracket so we have multiple blocks in terms of small flower brackets first two statements are executed first then the condition if the condition is fine then statement 3 and 4 are executed the condition is not good then statement 5 is executed there are total 5 statements that are to be exercised in this program how are we going to do it so because the statement coverage we need to make sure that 5 statements are removed so we need to make sure that the condition we are going to provide such a way that the condition can take both no as well as yes with those values so by statement coverage going plainly we definitely we are going to cover first two statements and the second path it will based on the result of this two statements statement one or two it could be it will go either as yes path or no path so either case we will execute only three statements or four statements one of this will be exercised so with this statement coverage approach at best we can achieve either 3 with the no path or 4 with yes statements are exercised so statement coverage can do this much because the conditions it won't care so that is how this has been done with the help of 5 statements with the conditions that are underneath this when creating test cases statement coverage we can move to the control flow graph so control flow graph is something like how the flow is going to happen the statement coverage requires statements in the code to be executed we know that the boxes and the diamonds represent all the statements in the code so that is also part of the statements so that that also will be reached with the help of statement coverage by following the paths to the code we cover all the diamonds and all the boxes are covered and thus we have statement coverage according to the relation with Maccabay it is a Maccabay is a personal name but he found out he discovered these standard measure the complexity of the program based on all these diamonds and processing statement so according to the relation with Maccabay measure it is called there should be three or less test cases and in this case two are enough so to have two paths what I said is we need to have two test cases the first test case will take care of statement one two and five the second test case will take care of statement one two and three and four so two test cases are enough with this measure how much of the total executable statements have been covered by the various test case scenarios is what is important in terms of statement coverage for a decision statement both true and false condition should be tested even if the decision statement is of implicit form if it is though it is internal that means it depends on the some of the internal not the external one that means to be exchanged still in the given piece of code block if condition is false then three out of four statements are said to be covered so we know that statement one statement two statement three sorry statement five have been excised there is a false condition or no condition exists you achieve 100% decision coverage both true and false conditions have to be achieved into different testing areas even though the statement is of implicit form so the second form is we need to have another statement coverage with the path taking as S or true is the help of that statement one two and three four have been excised in the first form we have excised in the first form and in the second one we have excised two parts or no whatever we want to call so that is how the statement coverage in terms of total number of executable statements will be okay so that is what is about statement coverage all the executable statements have to be reached next type of box testing is on the branch testing or the decision coverage or decision testing it is called so how are we going to do okay so here we create test cases so that each decision in the code executes with both true and false outcomes so basically two test cases so we take that simple example only so we have excised the statement coverage in two techniques similarly for decision or branch coverage testing what we do is what true false conditions we will have it so we use the tools as well for this it is called instrument decision to where decisions have been taken care and here also the coverage aspects are done with the help of a percentage whereas total number of executed decision outcomes have been considered that means total number of decisions how many are there against total number of executed decisions that means suppose we have covered suppose two diamonds and each diamond will have two outcomes true false true false then there are four because two into total number of decisions because each decision will have a two outcomes either or false so one diamond if you have excised or one two diamonds with one outcome we have excised it means the path is true so the coverage is 50% so this also can be done with the help of a tool so both are called same basically branch coverage or decision coverage so it is a technique similar to a statement coverage so the idea with the decision coverage is to execute every decision in the code that is twice it is not once that you have to remember every decision in the code have to be exercised twice in statement coverage only once is enough here twice because the decision will take two parts that is why we need to have this criteria very important so both possible outcomes of the decision should be executed in order to reach the decision coverage very first glance statement and decision coverage seem to be exactly the same procedure but executing every decision with both the true and false outcomes will result in all segments to be executed so in order to execute all the segments all outcomes of every decision needs to be executed that is true and false so but there is one case in which statement coverage come to which without having full decision coverage and that is with an if statement without an else class so in that case we will not have false conditions but that is not a good programming practice to have if something do some condition statement and there is no else so in this case only one way it is possible but it is not a good practice to have one knife without else else have to be there else do some statement some other execution so definitely we are going to have a multiple parts for true as well as false so obviously we still need a second case with the false outcome to reach the decision coverage so coverage is measured by dividing the total number of executed decision outcomes will be more decision times so and the tools that are used in this also same tools for monitoring the coverage where a decision coverage is taken care as well as the statement criteria has been met so that is with that basically and further decision coverage requires a two test cases one for two outcome and for false outcome for simple decisions that is decisions with a single condition decision coverage ensures complete testing and control response for the decision A or B test cases true false and false false will toggle the decision outcomes between two and false so we have reached that however the effect of B is not tested right that is those test cases cannot distinguish between the decision A or B and the decision A so the analysis should confirm the degree of structural coverage appropriate to the software level whatever the level of software that is depending on the analysis should confirm basically whether A as an effect or not for a decision statement both true and false condition should be tested even if the decision statement is of increasing if condition is true then one of two decisions have been covered to achieve 100% decision coverage both true and false conditions have to be achieved in two different scenarios even the statement is equivalent okay so here the same example you see here statement one two the condition yes will take into statement three and four and the condition no will take into statement five so two conditions are to be covered true and false or yes or no decision coverage is with the percentage executed decision outcomes only one decision outcome is divided by total number of decisions that is one into two two so there are two in the denominator and two in the denominator that will make 100% decision coverage so that is about the decision coverage next one we will see the other technique called data flow testing here also the data flow testing uses the control flow graph control flow graph is you know that it is a basic control of the program how it is going to be good with the structure of the program so data flow testing uses the control flow graph to explore the unreasonable things that can happen to data that means basically we use the data in terms of controlling the flow consideration of data flow anomaly through test path selection strategies that fill the gaps between complete path testing and the branch or checking testing that means we test using the control flow all the time but sometimes what will happen is purely the flow is based on the data for certain cases so there are gaps by doing that selection criteria with the help of path testing and the branch or checking test code so in that case data flow in testing is useful data flow testing is the name given to a family of test strategies based on selecting paths through the programs control flow in order to explore sequences of events related to the status of data objects you know that data flow testing is the only problem of data flow objects where data is the key here data could be a variable example pick enough paths to assure that every data object has been initialized prior to its use all defined objects have been used so how data flow testing example is that the variables or the data that are used should be initialized because there is a requirement that the initialization function will take care of the global variables system gets initialized upon order so what will happen is the system gets initialized so all the variables from the data that are part of the initialization function unit function will get initialized so how do we do is with the help of the data flow whether that initialization is done for each of the intended data it could be a variable or anything so what will happen is that unit function whatever the where 1 we have initialized to 0 so initialized it does not mean that it could be 0 it could be any value that is used as a unit value and throughout the entire program so this is very important because the data flow if instead of 1 if you are put as 0 it could fail because where 3 is being initialized and that initial value in static increment in the other program may go so that the increment will happen if it is initialized with 1 as 2, 3, 4 if it is initialized 0 the increment will go wrong similarly if the variable 1 and 2 are initialized to another value than 0 the program can collapse there is a very important to have data flow testing along with other type of testing techniques. The next one is this also very important whatever the data or the objects that are defined should be used so very important because we cannot have a data what is that it is called a data object which is not used also called as data object so this is very important we cannot have because unnecessarily it will occupy the memory without the need and sometimes it may so happen that there are other functions that could use the same name as the one that is there in the other program which is unused will result in some sort of an anomaly in the program so very to avoid this type of things and this will be done with the help of data flow testing and further the data type objects are defined in the data flow testing so type D type of data or the objects or the types having defined created initialized types K type is killed undefined or released once mostly this is used in application embedded applications or C++ applications U type of data object is used type so used in the calculation or no particular and object that is a variable is defined so when it appears in a data declaration or present a new value file that has been opened is dynamically allocated etc so all those are called as an object so there is a definitely a requirement for this and against that objective of that particular variable or the object as for the definition we need to test it so an object is used is called as used when it is part of a computation or a predicate so you can see C or P type P is for calculation P is for predicate the variable is used for a computation when it appears on the right hand side that means the variable is used when it is used for comparison or assignment etc some sort of a computation that is called as C type used the object data object sometimes even the left hand side in case of array indices also will be used of an assignment state understanding RHS so accordingly the variables are computed so variable is used in a predicate when it appears directly in that predicate that means variable can be used as a predicate by itself that means the variable can be compared directly something like if where A is something then take some conditions or decisions and that where A is getting updated with some sort of signals or whatever so those are called predicate variables where there is no assignment or RHS or LHS type of a usage is all done with the help of that variables so next one is examples of data processing data object types we can see definition C is P is as I said definition is defined created initialized the other one is C use and P use for calculations we use C type of objects for predicate we use P type of objects here is a small example program we will solve z equals x plus 2 if there is a condition so so so so this is a print of segment and how are we going to aggregate the different types of data objects here x and y are defined with the help of read statement read function rather z is also a defined one w is another one y is another one C use a computation use because x is used for computation also we have z also we have y we have w z etc because print is used we have supplied as more parameters that is why this C use for P use we see we use the predicate where z is less than y is used by the predicate for doing some if condition so it is called as P use variables data object types so all this need to be verified in data processing so data are as important as code because having the implementation along with the data is as important to flow the program and define what you consider normally you understand the program flow and the data that is used in the program and that definition we need to consider in terms of any anomaly how it is being used what again is to be spoken in the corner so data processing strategies span the gap between all the parts and branch system so it is not just enough to have a branch system by supplying z and y values and make sure that the path 4 is done 1 2 3 4 6 one path is here in this addition testing what we do is paths 1 2 3 4 and 6 are done in the other type of addition testing with the else part executed what we do is 1 2 3 y and 6 this path is executed still we would have missed if the variable is been assigned the wrongly or z is supposed to have x plus 2 and specification says that it should be x minus 2 and we have missed it so we believe that whatever the statement is available before addition testing is being tested properly so it is very important to have data processing also so how do we do by having all the paths excised with the focus on the data so gap between the path and the branch testing takes care of so we have studied about statement testing where all the statements have to be able to prepare and invoke and in branch decision testing we have to box the conditions or the decision sorry not conditions branches will be tested and in data flow testing we do the data or the objects or different types will be tested and in the next session we will study about the branch condition testing, branch condition combination testing modified condition testing and the last type is LCSAJ testing and we will test based on the given scenario so some of the embedded system testing words we will go through what we have added today is a branch condition decision data flow so along with other words we list out for each session how many embedded system jargons are involved of course there is a glossary we have added some more glossary such as litometrics, logical test space master test plan, MIL, model input mixed signals, model based development MBD, output quality example ISO 8402 standards quality assurance, quality characteristics so we will add likewise each session some of the glossary that we need to understand in embedded system testing so with that we will end this session and then next session we will continue on the white box testing other techniques