 ఇరిస్ని నిమాని మావట్ని నిచింది మానినిరంది మారి మాస౿ మాసంది. in the next session it was about test case design and procedures with example test case how it is done what is the template those things we studied also we studied about testing standards what should be used and how it should be used test class of the what is V and V debugging test planning example test plan how the planning should be done the recap of the next session example test specification example test spec test procedure example we studied about those also we had gone through a sample SRS I think I just shared a sample SRS of a simple embedded unit instrument we can touch base like that during our x-ray that SRS software requirement specification talks about different modes of operation that embedded unit does and an example scenario also we underwent during that session in the next session we studied about levels of testing unit integration and testing depiction testing depiction test harness about test setup how test are done and how they are set up and we studied about the hosting target based debugging testing from developer perspective as well as test perspective in the initial stages of the software and also we came across simulator and target monitors these are some of the testing setups how they are going to be used in different levels of testing like it could be unit level or it could be integration level or system level and we listed out a sample test pools what are those with an example and this will be the basis for our x-ray how we are going to formulate all those tests based on this platform okay then in the next session we studied about TN method this is one of the structured testing method is over here little principles that are learning cycle techniques infrastructure organization how TN method is defined then we have categorized the commercial tools and as an example as well in the next session we studied about software life cycle entry and exit criteria of the different life cycle phases with an example of prototype life cycle we defined it and software testing life cycle also we studied we took example of V model life cycle typically embedded industry they follow model life cycle and processes and elements and also in the next session we will talk about what are the process elements and we took an automotive process automotive testing process and consumer electronics life cycle example we had about them and different types of life cycle also basically it is an extension of the V model life cycle we listed multiple V model testing by developers testing by independent testing then as last session session 11 we studied about master test planning that means for a bigger complex systems how we are going to test how we are going to plan the tests that is nothing but master test planning 10 principles of embedded software testing based on the art of software testing also we just gone through those are all going to be helpful in terms of understanding the next level of how to do test and what are the different types of testing so these are about recap of different sessions that we had in the unit one so unit one is all about fundamentals understanding templates standards process etc so in this session we study about the testing method and its details to start with let us study a little about IV and B that is an important term that is used in the industry it is nothing but independent validation verification there is a dedicated team they do this activity in a typical embedded software development testing there is a IV and B team as I said in my earlier questions there is a plan, execution strategy, execution team who has to do what and all that so all this comes under IV and B concept IV and B terminology so why it is called independent validation verification is there is a dedicated team the testing has to be done with independence about some of the embedded software development or testing mechanisms are all will be done in the independent manner so that it is not biased okay so what is verification so the basic difference between verification validation are we building the product right and validation is are we building the right product so two things are there their product means the embedded software first as per the requirements the product is in build so what is the verification validation is the product once we develop are we building it right that means whether the product is acceptable from the customer perspective so we are going to validate that so these two are very important terms the process of demonstrating the product build is right so on the left hand side you can see the verification concept on the right hand side you can see the validation concepts and its differences you can compare it verification it is a process that is used to evaluate whether or not a product for which system complies with regulations specification for conditions imposed at the start of the development phase that means basically verification takes care of the development aspects also like whether the life cycle that is followed during development phase whether it is compliant to all the regulations whether the product or the process of service that are used or the conditions imposed during the development phase are in line with what is expected all this will be evaluated under the verification activities whereas in validation the process of establishing evidence that provides a high degree of assurance products or we saw a system of complies with regulations that means we validate once these things are in place whether that is meeting the negative requirements is what we are going to validate under the validation activity as I said the process of demonstrating that in the right product is what validation does okay the next one is verification can be develop a scale up or production which is often and process based so basically the as per the process the process could be anything that is going to be defined within the organization how it is going to be developed and tested all these aspects will be verified under the verification plan so this verification can also be part of development whereas this often involves acceptance of witness for purpose with end users and other products based that means from the acceptance perspective the product is validated Example of verification is a model testing requirement is testing integration testing that means all the primary inputs that are used are going to be applied for verification purpose whether the product that is built is correct is what is getting done under the verification whereas in validation example is that system testing carried out from the end user who is going to use the product that means the product is built right that is the right product I mean what is needed for the client customer is getting done is going to be validated so all these aspects verification and validation are done with independence because it should not be biased that is the purpose of IVND okay now there is a term called static testing and dynamic testing what are those so let us define about dynamic testing so we will understand what is it then we will in contrast to that how static testing is done so we will know the differentiation and how both are important we will study about it okay so what is dynamic testing definition a process of evaluating a system or component based on its behavior during execution that means testing done during its execution is called dynamic testing it is as simple as that we will put that unit under execution and put the unit under test while it is executing the program that is the simpler definition another definition is testing by executing the program with real inputs that means we do that testing while the system is under execution or the system is running the intended program that is the definition of dynamic testing the other method in context is that is what I am understanding this testing is done without the need of execution so basically we do the dynamic testing process conclusion of all the real codes provided to the system and we collect all the outputs and all that along with that we may not be sufficient to have just dynamic testing along we need to have a static testing also so that doesn't require any so this includes software instruction in some forms of analysis so basically analysis is used from all are done under the static testing sometimes some scenarios are the test cases that we have defined it may not be possible to test it when the program is running or executing so we may have to do some sort of a analysis of the program that is running under the target or we may have to inspect the code or we may have to analyze some of the results that we have captured so with all that with the help of that we will arrive at a conclusion that is all under the static testing so that is the definition of static testing dynamic testing static testing we do not need to execute dynamic testing is done when the program is executed dynamic testing in continuation of the definition there are two types we can categorize in dynamic testing explicit dynamic testing and implicit dynamic testing what is explicit dynamic testing so in explicit dynamic testing the most of the system functions are tested by means of test cases specifically designed for that purpose that is it is basically a test case driven with the help of test cases it is specifically designed for that purpose to test it that means explicitly we will test it with the help of test cases in implicit dynamic testing what we do is during the test process and execution the metrics are analyzed to conclude on the specific testing scenario that means while testing some methods or some strategy we may come across some of the execution details or some outputs we may get it partially or fully all these are used in terms of testing process that will help in concluding the specific test scenarios these are all implicit dynamic testing that means testing aspects are done implicitly we do not have a specific test scenario for this while doing some of the explicit test scenario we may come across testing outputs which is enough for concluding the outputs so that are all part of the implicit dynamic testing so usually most of the systems functions are tested by means of test cases specifically designed for that purpose for testing so to this end many test design techniques are available we will discuss about that in next session testing can also be done by analyzing the metrics as I said that is all implicit testing or by assessing the initiated measures on the basis of a checklist that means as a conclusion of explicit we might also cover some of the other scenarios those are all part of the implicit testing for example performance can be tested during the testing of functionality by response times or recovery with the help of some of the inbuilt tools so we may not need a separate test scenario for doing these conduct tests so no explicit test cases will be designed for this so it is tested implicitly that is why it is called implicit testing another example is security for instance it can be tested statically by reviewing the security regulations security aspects of how it is implemented how it is behaving all this can be tested implicitly so that is about implicit dynamic testing the next one is structured basis testing we know that we have studied about TM method applying the data principles of life cycle infrastructure techniques and automation these principles are called structured approach so dynamic testing is one of them so we apply the structured basis methods for doing the dynamic testing so similarly any other testing approaches to align with the test process will define in the beginning of the embedded software testing all coming under dynamic testing now coming to static versus dynamic testing in software development static analysis and dynamic testing are two different ways of detecting defects unfortunately there are too many thoughts of completion or complementing each other and developers are sometimes encouraged to favor one of the exclusion for other one like some people prefer static some people prefer dynamic that way so what will happen is we need to segregate between static as well as dynamic when we draw the plan or when we do the static also we need to define the tools that are very much important there are tools for static analysis also so aggregation of the strategy in alignment with development aspects or the development team also because we need to be in conflict with the team also because they are the one who will develop the system and we should provide sufficient inputs or they should be knowledgeable enough in the sense that how this system can be tested so that testing goes smooth so in that aspect we will clearly define how much of the testing I will cover with the dynamic testing how much I will test it and what are the tools static testing tools that I will use and dynamic testing tools so that the development team has a gist of what is getting done during the testing so this will be clear in terms of any inaccuracy or harmful implementation that may be there in the development software so that will be clear vision for the development team and the development process and the role will be clearly defined who is going to do what so the team will be aware of what is going to be done so two techniques basically they complemented for static and dynamic testing static analysis were explicitly used earlier days because we didn't have much tools for doing this dynamic testing so most of the testing were done in terms of its inclusion and reviews so that is what is all static analysis either tools were extensive or the ability was not there so all these were some of the reasons why we didn't have much techniques for dynamic testing so we used to have more of a static testing so now we do a lot of automation a lot of tools are available for doing the testing as much with the minimum intervention of humans that is why the dynamic testing is growing but still it may not be enough to have a dynamic testing someone needs to do a static analysis so now we will having understood what is static and dynamic testing we will go in detail of dynamic testing so dynamic testing is further divided into basic groups we know that black box testing and white box testing pause please so dynamic testing is basically divided into black and white box testing we will study about each of them as I have been in my earlier sessions black box testing is something like we don't need to know the the final details of the structure of the implementation of the programs which are part of the template software whereas in white box testing internal details of the embedded software will be detailed for testing also we study about black box testing techniques and white box testing techniques so these are all part of the dynamic testing basically for doing dynamic testing black box or white box whatever techniques that we want to apply we need to have these things minimum taken care while doing the black box or white box testing first thing is the strategy that we need to have it for doing the test next one is the test case selection that means once we work out the strategy once the strategy is in place we need to select the test cases we are going to add test cases more and more or less then finally we are going to have a coverage how much the code is covered or how much the function is covered so these three aspects have to be part of the dynamic or dynamic testing which could be black box or white box so we need to have a strategy the strategy will define the purpose the goal and the goal once we have the strategy how to attain that strategy is what we have to define under the strategy it could be a performance related it could be a functional or related whatever it is basically specification we have we know the system we have understood and analyze the requirements now it is the time for us to define the strategy once we have the strategy broader level defined test cases for each of the strategy per requirements under the test case selection methods we have how to pick up the test cases for each of the requirements or group of requirements or it could be function of features so we will select the test cases or we will define the number of test cases so I am going to pick then once the test cases we have defined for all these requirements because we are going to write the scenario the scenario could be covering the different test cases one test case, two test case whatever it is it could be multiple test cases also so for simplicity sake I will write it it will be easier one or test cases can be part of the scenario it is basically driven from the features and purely this drives the grouping once we have the test cases see what we do is just we define the test cases one set of complete test cases for each requirement or functionality or feature whatever it is then we are going to segregate whether different good enough for all possible test cases all scenarios can be done can we eliminate some of the some of the test cases can we do a implicit testing with the help of explicit testing methods all this will be under test case selection because why this is important is we cannot be writing test cases just like that for a requirement requirement could be talking about 100 test cases but does not mean that all 100 test cases need to be tediously defined and tested so it could be some 10, 5, 20 whatever it is that is again depending on the selection method so we need to be doing a good selection of test cases and methods finally once we have the test case selection we are going to have coverage so I repeat the strategy what we do is we deal with defining the problem of the test to clarify the purpose of testing then define a goal and finally we develop the strategy how to reach that goal once the goal has been defined the test case selection strategy can be constructed we are going to construct the strategy wise test cases the obvious strategy would be to test every that is there always but the possibilities like as I said we cannot have infinite number of test cases as it grows it is based on the strategy how we are going to select the input or if it is not physical we are not going to select those things it is objective again based on the functionality of the feature or the limitations that the system has so we need to be carefully selecting the test cases that are going to be executed eventually these test cases should be good representatives of all the possible test scenarios to simplify the selection there are large number of test selection methods most of them are associated with the coverage category or how we are going to make it a easy test so basically to determine if we can discover failures we can stop the testing I mean basically stop means here there is a sampling criteria if out of 10 tests I can have a stop criteria saying that 5 failures are there I am no more going to continue because those 5 failures are very critical I take up the next so coverage is a measurement of how much has been done compared to the total number of work that is what we do with the coverage okay now we come to test case selection methods in black box the selection method is basically functional or data that is the real input what we are going to use it and what is the criteria for test case selection so based on requirements functional specifications interfaces as we are going to have the selection black box test design techniques are based on the functional behavior of systems without having the input knowledge of the implementation as I said in earlier the user of the we will not have any knowledge about the internal details box that is the embedded system what it is supposed to do what is the specification how it can be fed with the real input what is the expectation what is the output that it can expect so in black box testing the component is subject to input and the resulting output is unless it conforms to the expected results of the behavior test selection is purely based on the structural or the logic that is implemented within the software so based on the implementation or the structure of the code we are going to try the test cases or we are going to define test cases or the selecting test cases white box test design techniques are based on the knowledge of the components internal structure and uses all the information about how the inside of a unit works that means what are the different components of the units that are there and how they interact how it can be driven how it is going to result so all these micro level details are part of this white box test case selection so this information might be code or design so sometimes code may not be enough a well defined or well designed document also will be used for defining the white box test cases white box test ensure that each implemented test statement is run at least once and are tested against correct behavior that means the implementation of the code has to be run or executed at least once and with that execution we are going to test it against correct behavior that is what we do with the white box so these are some of the aspects of test case selection for the black box and white box so we know that what we do is for the test cases that we have selected we are going to have inputs that will be determined the next step is to define the expected output for each of this input so all test cases always take the expected output from the requirements for that particular input to find out how the object under test should react to the particular input so we will have, I think I explained earlier sessions we have an input, we have conditions we have an expected output all this will be laid out so this is the standard we use it or basically black box does not matter because our test case design is such so we have input we have conditions that are required to figure out the input then we have the expected output so this is the basic thing that we are going to have it for test case design so basically we use the logic logic mode box whereas in black box we use the specification or higher level requirements so that is about test case selection here is the diagram which clearly identifies the black box and white box testing how it is going to be used we know that component level testing is where integration testing system testing down the line and as an embedded system we have to cover it so it will be 8 to 9 percent with the help of black box and white box there are cases where we use gray box it is called I will just put few words but we will not concentrate on the gray box it is a mix of both black box as well as white box this is an interesting thing actually I will explain in detail we know what is black box we know what is white box the two types of test case are used a little bit differently in the development white box test cases are mostly used in the early test cases we know that of the development cycle and are of less usage higher up in the testing hierarchy and black box testing techniques are used throughout the development life cycle that is we always focus on the black box level the main advantage with the black box testing is that the only depends on the requirements whereas the white box purely depending on the level design application so both methods are of course important white box as well as black box however there is a boundary between these two so a gray box tester partially knows the internal structure which includes the access to the documental as well as the algorithms some sort of minor details of the implementation will be part of the gray box test and he is also knowledgeable at the higher level that means the detail documents in terms of specification are describing the end application what is going to be the running on the field so all this will be part of the black box so the gray box tester will also have more so gray box testing sometimes are applied explicitly but it could be defined underneath the white box itself or black box itself but just for a definition sake it is beneficial because it takes the straight forward technique of black box testing and combines it with the code targeted systems in white box testing so while doing the black box we have a fair idea about what is happening with the black box that is the implementation of the code but generally have seen not much used in the embedded systems industry they do not have a separate definition though they cover it while doing the black box or the white box usually it is used in I would say object oriented software systems mostly it is used so it has both mix of black box as well as white box so let us not concentrate much on this strategy it is a combination of both just for the definition sake okay so we understood what is black box what is white box while doing the dynamic testing now coming to the coverage very important thing because at the end of the day after completion of the embedded software testing we need to be knowing how much we have tested without that testing is not complete so what is white box testing coverage what is black box testing coverage code coverage is what is that called as white box testing coverage basically it measures the code coverage in terms of how much execution I have done during the white box testing versus the total coverage basically it is a executed code divided by code code will give the code coverage in terms of percentage the coverage is different in terms of percentage similarly here also the coverage for black box testing is measured purely is based on the requirements so how much of the requirements have been tested versus or divided by total number of requirements we need a percentage of coverage how much I have covered suppose both of them will define there is a thousand lines of code that is the denominator and we have covered say some seventy five hundred lines of code that means while doing the black box we have covered the seventy five hundred lines of I am sorry seventy seven hundred lines or we can say ten thousand lines it will be seven to five hundred then it will become something like seventy five percent so that is the code coverage I have twenty five percent I would have covered or I would not have covered so what are the techniques that we can cover that all we will study while doing the white box testing techniques details so we use tools there is a but it may not be possible to cover hundred percent so we have to see the other techniques of the coverage so those coverage aspects are not detailed later okay similarly coming to requirement coverage I have say hundred requirements and I have covered eighty requirements so my percentage of coverage eighty percent so eighty percent I have covered twenty percent of coverage we need to see that is called as coverage shortfalls how I am going to cover these shortfalls that is twenty percent twenty percent for white box and twenty percent for black box so there are different techniques that are used for coverage shortfalls like I could take a credit of white box for black box similarly credit of black box for white box it means to say that requirements which I cannot cover in black box I would have covered with the execution of that particular functionality so I would say I have covered in terms of error free board so that it does not harm but it is wasted not to have this thing the best way to have techniques defined for shortfalls inspection analysis reviews of course while doing all this we might look into some of the research that we have achieved doing white box while we are doing the analysis of the black box the other way also is true while doing the white box you might have to take a credit of black box for inspection analysis so that will be used the complete coverage so the ultimate is to cover 100% 100% the testing is not complete so we should be telling that I have covered 100% of the requirements or I have covered 100% of the code that is tested then only the product is called as embedded software testing completed the next one we will see a difference tabulated here black box versus white box so also called black box is also called input or put in one testing or specification based testing whereas white box is called logic for even testing we know that white box is taken or done with the help of the implementation of the code and software is viewed as a black box without bothering about the structure of the program in the black box testing whereas in white box testing we need to under the equivalent behavior of the structure of the program or the code software is tested against its specific software is tested against low level specification or it could be designed also called as a detailed design or low level design also we use the code as a knowledge for white box testing test data is derived solely from the specification so we have the test selection of the test data based on the requirements of the specification whereas the test data is derived from the logic of the program that is we will understand the logic of the code that is implemented based on that we will arrive at the test data so these are the differences between black box and white box we will see black box and white box okay so what are the advantages of black box so black box testers do not need to have connections with the code it does not have to care about what is the code but all he has to care or his perspective should be code should have bug that means he should suspect always seeing that the code will be having some issue that is where it is failing for a tester to see to that he is testing the embedded software or the embedded system with that energy test cases are designed as soon as he is testing so he do not have to wait for the implementation he can test it as soon as the requirement is done only to wait for the code or because you know black box we do not need to bother about the internal all we bother about the requirements against what the product is being developed there is a need of having detailed function of the system to the tester and its behavior definitely this is one of the important thing that I have spoken last time the testers should have knowledge equally to the development team that means will be done from the end user perspective so definitely he needs to have a end user part of you and he should have some knowledge of the system test this is why because the end user should accept the system basically without the knowledge we cannot test it so to understand the system black box tester so these are reasons sometimes testing technique is also called as because the product can be accepted or not can be told only in the testing mechanism of the black box tester so black box testing also helps to identify the ambiguity and any contradictions in punctual testifications it will bring out if any internal or any ambiguity is there all those aspects will be identified the requirements are complete or no ambiguity is there correct all these things will be tested in the sense that when we have a larger system the basic specification we are able to subsystem requirements what will happen is since these subsystems are developed or specified with different people there are chances that they may disconnect between these subsystems and its developers so all these will be identified with the help of black box sometimes while writing the test case itself we will come to know the problems that the requirement is contradicted to the other requirement so having ambiguity or the requirement is not completely written so that it can be tested so other aspect of the requirement is that requirement should be written so that it can be tested and the developer has to take care so these are some of the advantages of the black box testing so why we need to have all this is because it is useful for the or the efficient for the larger or complex systems okay so that is about the advantages of the black box testing black box testing these advantages there are more advantages equally but we will see how to work from all these things testing with every possible inputs is obviously because we could take an inordinate amount of time sometimes those are the system should accept so and so signals and we should tolerate so and so temperatures so what will happen is it may not be possible to feed all those values it may be difficult or the those situations may not arise but it can so happen that we may not be able to feed the values unrealistic situation it may be difficult to emulate and black box testing mostly present in coverage so if something is there what is going to happen if something is not there it is also called as else portion of it what is not going to happen that portion of it it may not cover sometimes why because we are going to go purely by the requirements especially if then else kind of requirements we may not be able to test it completely exact that specifically for the code actually but sometimes requirement also have to specify if this condition occurs if this condition does not happen that is the else portion of it so what is going to happen that is all sometimes will be part of the code but it is good to have requirements identifying all this in that case it is easy to do the black box testing but black box testing does not care about exact point in the code where the software malfunction cannot be detected we may not be able to sometimes it may hang or sometimes it may not behave the way it should we do not know what is the issue while doing the black box testing so though we say a failure we may not be able to arrive at a analysis of where it is failed so it is because it is due to the implementation issue so in that case we may have to do a inspection code instruction where the faulty code is there faulty code is there but in those for intermittent failures of the code cannot be determined means this is very typical thing actually in the embedded systems what will happen we may not be able to reproduce some of the failures it is very difficult sometimes the failure occurs sometimes it does not work how do you find out so those cases also it is typical to do a black box testing we know the failures are there but to make it point to fail it may be difficult because it varies as told here whether user required functionality is implemented at that point we do not know until we discover with the help of other methods okay the next advantages are some of the errors may not be able to discover until the white box testing is done so all these disadvantages may require to do a white box testing with the help of the white box testing we might analyze it is failing at the higher level and the last disadvantage is that heavy dependency on the test environment and stable system so it is very important we need to have a very predictable and dependent stable system so such environment we need to have it so that is the challenge that is why we have a heavy dependency that is for the black box testing okay so now next thing is we move to white box testing the advantages as the knowledge of the internal coding structure is pre-requisite it becomes very easy to find out which type of data input we can provide so what are those applications that can be tested with the help of that all this will be easier and we can test it and white box testing the techniques helps in removing unwanted lines of code that means we know what lines of code is responsible for what functionality what is not responsible what are the defects that we have to program all can be discovered with the help of white box testing sometimes for example the use of equal to double equal to so all this will be found out if you use equal to and if you use double equal to for couple of variables the variable will roll out that means there is called a roll out problem there is a difference one is for comparison other one is for assignment so definitely there is a chance that purpose of that is of code the next one is discrepancies between the code and requirements can be identified that means it is equally important for the white box to have some knowledge of these requirements or a minimum knowledge that is enough for the white box system so that we can find the discrepancies what is very much important especially for the algorithms or conversions some of the ADV what is being used and representations tolerances because tolerances are defined directly in the requirements it could be plus or minus some percentage in percentage for example these are defined directly in the requirements so it is good to have a knowledge of requirements and we know what code should be there for that requirement particularly so all this can be found out with the help of work box testing and for doing the white box testing we do not have a system we do not need an end to end system because the actual end target may require those signals we can emulate with the help of work box testing with the values that we want to do so those are the advantages of the white box testing these advantages of white box testing are as per below as knowledge of code and influence structure is very clear a skilled tester is needed to carry out testing because he needs to have a thorough understanding of the complete code what he is going to test it so basically this will result in extra cost and the white box testing is usually technically because he has to test from all the aspects and he needs to understand how the code is going to be compiled and the development environment he needs to have because he will be covering the structure of the code and it will be as per the standards that are there at time there is a need of system knowledge for the special details of the implementation especially with all the states of the software so all this knowledge he needs to process so that is why white box testing is bit I would say lot of methodical efforts are required so that is the disadvantage but there is a dedicated theme usually allocated throughout the testing electrical of the white box testing for an embedded system next we understood the white box development areas black box advantages and disadvantages having understood the black box and white box testing and knowledge dynamic testing now we will come to the test selection criteria for the testing of the black box what are those so equivalence partitioning is one of the test selection criteria boundary value analysis state or event transition these three categories are used for test selection criteria we know that test selection criteria is based on the different methods black box we have functional data these are some of the inputs do not get confused with the methods and the criteria that we have for test selection equivalence partitioning boundary value analysis state or event transition so we know the requirements what are those all these should be covered under one of the criteria or we should have the criteria used for selection these selection are primarily based on normal or robustness conditions we will detail out all these things and it will be in the next class as we complete this class we will go to some of the words embedded software testing that we have added SSIT and IONV independent validation verification robustness any other words that we want to add we can do it I think white box black box is better white box black box what are the embedded software or the system words that are required from the extra perspective okay so that is the conclusion of this class before that I have few questions of the class what is the difference between static what are the systems white box and black box systems are advantages what are the selection criteria that need to be considered for black box system how coverage is defined for requirements based okay so we will go to the next class with equivalence partitioning to recap on today's class the methods of dynamic testing black box and white box its advantages and grouping of that and what is the selection and what is the coverage for white box as well as black box thank you