 now you may see you make bypass more for the second session of introduction of testing in it two series where we study about testing methods. For the the state or event transition testing so today's session we will continue that because the last session is important in terms of state transitioning and the model based design and development and testing is based on the states, state machines, grammars and all that so what we studied last time is the state transition technique in which we studied about composing the states, composing the transitions three and the test kept legal cases, test kept illegal test cases it is something like robustness, illegal means it is not unethical or something like that it is beyond the normal test cases so called as illegal test cases and the script cards which is optional so other thing is we took an example of a catheter recorder or a video which has about five states standby, play, record, fast forward rewind and as per as the practicality is concerned we studied about all the transitioning gods, conditions and the events that are going to happen between these and accordingly we know that for all of these so we listed out a state event table which identifies about 18 types of state events and the dotted ones are cannot be achieved which are illegal but the system should not accept so the testers should try to do this dotted ones as a robustness but the system should not accept as a state event that is why these are all dotted and it is called as illegal combination and different events are standby sorry different states are standby, rewind, play, fast forward rewind record also we had drawn a tree which talks about the path which is nothing but the control path I will again rewrite some of the very important term in memory system control path there is also something called as the data path which we do not need to control coupling data path very important term it is being used in memory system for now let us focus on control coupling or control path which is nothing but the execution of the program taking or considering the different events that are happening within this system events so that is nothing but the transition tree so we have about 14 test paths starting from standby and it is a sub state is rewind it is a sub state display fast forward standby and standby like this play also have so all these combinations we have seen the different paths so that are taken up in the state event transition tree where we have the primary event as a standby then we have rewind play fast forward record so all these paths have to be tested then we have drawn a table or a state chart table for the VCR having identified with identification legal 1.1, 1.3, 2.2, 2.3 etc all these are basically event based so there is a input and the condition the next one and the expected result so this is how this needs to be done in terms of state transition testing continuation of that so we have a illegal test cases also what we have seen as the dotted ones having identified I1, I2, I3 till I16 the setup for this will be similar to the whatever legal 1.3 where we need L1.1 setup that means rewind state is required and rewind event rewind we need to move which is an illegal script basically similarly all these are have been listed out so there are practical issues which we have seen hierarchical state charts to draw it may be little challenging we need to have a practical knowledge as well as the system knowledge requirements of course will help as well then coverage extensiveness we need to have it so state charts are very important in doing this then detecting the fault practicality and feasibility in achieving all these so these are some of the practical issues in state condition testing so that needs to be handled very meticulously there is another term called mainstream state testing which talks about we should not stick ourself into boundary cases only we should also see some of the good values as well in terms of a normal input as well as normal expected output so typically that will be used when the system is running that is how we do it okay so model based testing and design we see today I will go back to that slide which we have seen it I think yeah okay so before the model based testing the models have been developed in the development phase with the help of a model based design and development such as a MATLAB or simulink there are of course different tools also very long VHDL etc those are specifically for the hardware for complex systems we use model based development and testing basically it uses the different models for the system under the development or test those models are nothing but different diagrams charts so those will be done once we have those models and those models with the help of the tool can be used to generate the code and that code can be deployed on to the target basically that code can be generated in the host and it can be deployed on to the target and the target can be verified with the help of the target based execution but before that we may have to do something it is not 100% what is it called directly usable with the help of auto code generated code so that we need to little modify the auto generated code similarly for verifying it we use the same modeling concept for testing that means basically this model is a framework for us to go for the testing which is nothing but the model based we will study that before that I just want to describe in brief about model based design so philosophically there are lot of tools conceptualize we have a UMM required a modeling language then we have a system modeling language this are basically dealing with the all the useful transition diagrams state charts all this type of what is it called representing the system in terms of how it executes how it is getting modeled so that is how these are very useful once they do the models they generate the code and with the help of the target they will add additional code in terms of wrapping the generated code into the target then they execute it same philosophy we use for testing also the same model concept is being used so that is how it is called as model based testing so let us see about model based testing in the next slide so basically model based testing is a process of test generation from the models as I explained in the previous slide for the system under test there are number of methods that can be applied for MBT MBT is nothing but model based testing test cases are created automatically using models instead of manually that means we have created the models and the tests are being generated automatically for this model test cases are created automatically as I said the same test cases with the help of test hooks we will generate the script and execute on the target board the target board is already having the modeled code which is executing on the target so basically three classes are getting identified model test generation and test execution these are basically the classes they say about MBT model based testing okay so definition let us see what is model based testing going by the Wikipedia although there are several different most of them carry the same basic idea about model describing the system and the system of some sort and some way of using that model to get the test cases or generate the test cases basically it is called as test sequences or test cases are derived from the models so that is the fundamental about this so model based testing is a software testing in which test cases are derived in whole or in part that can be completely for sub-module model or smallest of the model or as an entire model where we have a top-down as I said in one up or bottom-up any of the approach where we consider the smallest of the units and go towards upper side something like a integration approach as a bottom-up so we can use otherwise top-down where we have the high-level requirements definition is very clear and we have the models we can use the top-down approach in terms of holistic testing so model based testing is software testing in which test cases are derived in whole or in part of a model that describes some usually functional aspects of the system on the test that means the model basically defines and describes the functional aspects each model has its own functionality features inputs outputs what are the signals signals could be analog discrete whatever it so all this will be part of the functions which are basically used to describe the model so all these are useful in terms of developing the test that can be as a whole as well as a unit so that is the definition of model based testing okay that is some of the advantages of model based testing as I said in one of the thing it is very useful for complex system what are other advantages model based testing has let us see model based testing is very easy to understand because the model is visible that means user can see the picture representation as well as the complete representation of each of the sub-modules that it comprises so basically it is very easy for tester developer or from the customer perspective so basically the complex systems that can afford to have some time spending on developing the model in advance so they use this model based development testing so this is useful because it is easy to understand from all the perspective designer as well as the tester primarily the developer and then the tester model based testing separates logic from the testing code that means we do not have to have a code at the individual level we can have logic at the business level that means the functionality as a whole and the functionality as an individual level both can be separated and the code that we have which we want to test it that can be separated so it will be very easier segregation of the different levels so easier segregation and integration so this is the basic thing that will help in terms of separating it so that is the advantage model based testing increase of the test coverage with the same effort as classic test so classic test automation we do not have a model so we basically go by the requirements and the requirement pertain to that we will try to understand the functionality try to fit the signals and all that so where the visibility is much unclear that means especially complex systems so very useful where we have a complex systems to have model based design in testing so in terms of classic test automation the coverage is somewhat difficult it is called coverage control that means we need to spend a lot of effort in terms of having the coverage and understanding the coverage but whereas the model will basically help which models are done what are the signals covered so basically concrete idea it gives for the coverage purpose model based testing is the fastest way to get use of automated test that means somebody to have understanding on develop the automation it will be easier to have a model available for testing so that is the meaning of the last advantage okay so the other advantages are something like though the models not have been implemented completely there are design and other things that could have been done already but without the need of the implementation somebody can start the test cases or test scenarios definition that means earlier verification is that means by seeing the models we can find the test loss that means a tester can find some of the issue that the model has by visualizing it and going through the model itself having his own sort of test cases identified so that is the advantage then verification simulated using based on high level requirements so verification can be simulated we do not need to have a target actually we can do offline as it is told in the next one the test can be accomplished on the PC instead of the target we do not need to have a target based systems in the beginning we can have the verification done using simulation of the models for the high level requirements we can basically have a coverage from the high level requirements we have under test model based testing enables us to switch testing tool if needed or support multiple platforms using the same model that means if the model supports the platform that is being deployed same thing can be used for testing also and the tools that are used is very easy to switch that means the same models which are developed and deployed with the help of the tools can be used for testing also so that is the advantage of that okay so the next one is model based testing focuses on requirement coverage not how many test cases you have last week etc that means you do not have to care about the schedule and all that of course that is a different aspect but we go by model by model and each model we cover the requirements and focus on the model you automatically bind to cover the more requirements automatically so that is the advantage last one being model based proved to positively affect the maintenance problem nemesis of all test that means the maintenance of the models as well as the model based testing is easier compared to the requirement based testing of the model what we have what we can say as a classic automation testing because models once it changes it is easier to adopt the testing changes as well so that is why model based testing is easier to maintain compared to the classic automation test okay we will take an example this is again a reference form one of the good example from the web actually a model describes how a system should behave we can understand with an example below so we need to supply the action and see if the system responds as what you expect that means we have a states defined as a blocks you can see there are the lower right so we take this as an example where we have notepad as an example here you can see we know all know that how we operate the notepad we have notepad executable opened we are going to start so there is a blank notepad that is going to open if you do not want we can close it so the operations that can happen model is that start and close the next what we are going to do once we have started the model is that we are going to type something and if you do not want to type that we are going to delete it it will be back to the blank sheet and once we are done we are going to close it once we have typed it we are going to close it while closing it will ask whether to save it or cancel it so cancel means it will be back to the same state what we take which is having it so that means the third state or reaction will be done with the help of this transition and we do not want to save it will be back to the notepad execution mode where the notepad is closed so simplistic approach here we can see the model should work so the model should work something like notepad as an executable should open should be able to type should be able to close and cancel so we know what are the actions that system the notepad system can respond as a model then we try to subject under test because this is already been developed so that is our fundamental approach model based test so once we understood about the model based example with this notepad we will try to understand what are the tasks of MBT model based testing so there are about 6 steps or 6 tasks understanding the system choosing the model building the model generating the test learning the test result collection and reporting the typical system testing what we have seen in the unit 1 classes similar things will apply here also where we do the model based testing first we need to understand the system here system is the models we need to understand the model first once we understood the model we are going to start the test by choosing the right model which we want to target first before we attack the sub models in terms of the top down approach so all the selection criteria will be done the help of choosing the model once we have this we do the build build the model that means we develop the test model what we want to do it model that is ready for testing we generate the test that means all scripts for this model can be done or we will do a run through of this model with the help of various tools such as UML and all that then once we identify the test we want to run it to see that how the model behaves once we are executed the test or run the test we have the result collection and reporting in terms of coverage and all that understanding the system basically forming a representation of the system's functionality lets say prerequisite of building the models this is a small task basically because already we have the model and complexity will be self-explanatory the software whatever is being developed for that particular model will be deployed on the target so all this information will be part of that model by understanding the model we know that what the model does so basically we determine the components features what are need to be tested that is the objective objective particular test so while doing the understanding of the models we also have to establish a sort of communication called communication with the requirement requirements design development team whatever it is that is very important because we need to establish the model mapped with these aspects so it is very important along with that we are going to identify the users who are all there so how we can interact so we will also identify the inputs outputs how it can be driven all these aspects will be studied domain study is called basically there will be number of documents that we need to understand to map what we have understood the model basically so based on that we will write the test this way it is very important to have an understanding the system next we have choosing the model so basically what we do is we select which model how we can test it that means this basically help in terms of grouping because there are number of models we need to understand what are the models which can be grouped so that that can be tested in terms of testing it so that is why choosing the model is very important so then we have the procedure for building the model this is also very important step so where we will basically build that means we put the constraints inputs outputs layout in doing the building of the model and of course behaviour behaviour and behaviour constraints also important so these are some of the building building the model aspects that needs to be taken care for doing the model based testing so something like we have a finite state machine for state diagrams so we need to try how those paths can be done and what are the sequences so there are different state charts, state diagrams sequence diagrams activity diagrams so many are there may be when we have a class for the UML we will study all of the aspects of model based those things needs to be done in terms of the generating tests because having understood that events and sequences we can have the tests where to trigger and all that then once we have that we are going to execute with the help of a script which can help in terms of executing those tests providing the inputs running the conditions and providing the outputs also so those are some of the running the tests once we have run the tests we need to evaluate it whether tests are correct so it is very important to understand how it has passed similar to fail why it has so very important points pass does not mean that it should be passed it could have been passed wrongly erroneously passed so how do we evaluate that so that is also very important so we need to evaluate or pass and fail criteria while doing the model based testing so those are the tasks that we have for the model based testing we will take a one reference examples of model based testing so you can see there are final state machines called which has different states example here is given as two states two events basically one is door open second one is a door close the first state is open door second is a closed door so there is an action open door from closed state it will move to the open state similarly from the open state it will go to the closed state with the help of closed door action so this arrow is a transition condition similarly we use a state chart if we have seen entry and there is a content so and what are the entry state it has in terms of various actions so there is an author created modified setup so there are a lot of aspects that is involved in terms of this is called a class diagram this is done with the help of unified modeling language is one of the very popular to represent the models or use the models and we have other models model based software testing also that is called the Markov chains grammars which are bit complex in terms of understanding testing we need to adopt any one of these models and go in detail and give that will help in terms of model based so these are examples of models that can be used it can be FSM state charts, VML Markov chains, grammars any of course so we will take a simple model which you have studied we will start with how we can start with the test a simplistic approach is that very simple is that you identify two states very simple first one is not running next one is running so the model can be entered with a running state model can be executed with not running state so we will start that way so that will help in terms of understanding complexity when we go in deep running start will go for running stop will go into not running state so these two are represented as one model so how we can do a test is with that we will apply a condition which is good for triggering not running to running so something like we will set the start as true or start as triggered that will enter into running so we will identify that whether the state has changed to running similarly the next test case will be whether it has run from whether it has changed from running to not running with the help of a stop command so two test cases we have identified similarly we will see the next level of complexity in the next you can see there are about four states I think this is an example of a calculator so what we can do is different states that calculator can have we take that as an example not running are a standard state it is called we know that it will go to running with the help of a start and it will go back to not running with the help of a stop so two test cases can be there similarly for the second one running standard it can go further deep as another running state which is called scientific we know that calculator can run operator will open a calculator I guess we can see the calculator so we know that the calculator can run in standard as well as non-standard non-standard could be scientific which we can choose with the help of a calculator here from scientific it can go to not running scientific so likewise we can have a model based where we represent the model in terms of different states and how the model can be triggered into different states we can understand with the help of a standard example calculator going with more complexity we can see here not running running standard empty running standard non-empty scientific empty etc so each one has its own sort of a conditions all this can be executed with the help of the model based testing where we apply the different states into considerations here you can see one set of other set of existing likewise we can have the complex model based testing where we have more states so here we have second one the next path you could take to 4 or it could next go to 3 then from 5 then from go to 6 from 6 it can come back to 1 so likewise we can have try to separate it out this is the original model this is the test model it is an exact replica replica of the the model so here we have the model here we have the MBT that is one path where we have tested so this is looking like same as original model so we apply the model based testing to apply all this test scenarios into the model where the model is represented at the left hand side different blocks so that is the concept of model based testing so I will repeat again all this model blocks can be tested with the help of different events where the events can be from not running to running running to scientific scientific committee so all this 5 blocks in this complex example we can have and each can go into different path with the help of a model based testing approach which is represented with the help of a model based testing model which is another replica of the original model you can see on the right hand side so that can be called as a model based testing so that come back to the original model so there are 3 aspects that we have tested application status mode status display status which can be many things actually going by the example we can see the application is running or not because that is the primary states which was made with the help of a running state from the standby state in that way we know what is the status of the application similarly we have a model that also can be tested so what are the modes that can be triggered to enter into different status we can choose standard method or a scientific method so these are primary inputs for testing the mode status written in such a way then we have a display status display can have a A, B, C in terms of hexadecimal in terms of numerical you can show 0 to 1, 2, 3, 9 whatever the number and it also can show plus, minus logical operations and whatever it is so primarily the calculator example has 3 types of statuses against which that calculator model will be tested so application status mode status display status application status has running or not running mode status has a standard scientific and display status has empty and non empty so right hand side you can see the various steps that are involved the start scientific stop start standard stop clear stop starts scientific clear execution that can be done like a test scenario there are about 14 test scenarios that have been listed on the right hand side all this can be applied to test one of this 3 aspects or 3 features of the functions so calculator may be having a requirement something like the calculator application should enter into a running status calculator should be able to do not running should be able to enter into so this one requirement requirement 1 requirement 2 so instead of writing calculator should go into running not so it is very easy to have a model what we have seen in the previous example with the help of that model we start with the simplistic approach then little more complex then full complex approach considering the model as a whole so that is how model based testing is carried out okay on the right hand side you can see the different scenarios have been listed out the various what is that called taxonomy of model based testing this will cover basically all aspects of model based testing this is what is that called complete model based testing overview this is again described in detail in model based testing for real time and systems in the automotive domain by justina zander nowika there is a book called model based testing for an automotive domain so that has a more depth description about model based testing where he has basically have categorization in terms of classes and categories and options you can see the first column have in the classes model test generation test execution test evaluation so these are some of the classes that are used for model based testing then we have categories model itself is a category where we draw the model next one is a test generation where we have a test selection criteria we know that test selection criteria can be based on the black box approach having events or the the various dynamic testing that we have seen in our earlier process and that test selection criteria can be applied with the help of this options column that the test selection can be structural model coverage data coverage, requirements coverage test base specification random and statistic fault based so all this are type of options that can be used for test selections so with the help of that we are going to have a test generation for this model for this class test generation is under class then we have the technology test selection criteria as I said in my earlier slides it could be automatic or manual black box or white box then random generation graph search algorithms, model checking symbolic execution, theorem proving algorithm proving online or offline there are number of things number of options that can be used in terms of tools or technology next we have identified this test generation as a class we go for execution class how we are going to execute so execution options are model in loop software in loop, hardware in loop etc so it could be reactive as well as non-reactive now we are going to observe in terms of non-reactive mode in terms of reactive we are going to inject it so that is one class the next class of the model based performance test evaluation test evaluation has specification in technology the specification will refer in mapping the test that we have done with the above test execution after we have the test generation as another class so what it helps is we refer all the features functionalities that have been tested or test executed and all the signals whether they are called robust moves, normal etc all these test selection criteria and the execution options will be specified and also that specification will have a coverage in terms of how many requirements are covered so what are the evaluations that I have done for this coverage all this will be part of the test evaluation class similarly technology also we are going to evaluate that means for example I am going to qualify also for example if we take an automotive or aerospace because we need to subjectively understand whether the tool is producing the right results the intended results that should not have any errors the tool should be error free all this will be part of this technology evaluation or tool evaluation it could be automatic or manual it could be online or offline with the help of the vendor whoever has produced the technology or the tool output so that is about the model based testing classes categories and options which are used in the industry so that is the end of this session described about models and the tasks of the models model based software testing example taking a calculator for the different states it will enter and when we have the complexity we define the paths that models can take and we replicate that using a model separately as I explained so that is the philosophy of model based testing and we list out the features versus it is a test scenarios categorized in the model based testing and we can have a classes defined for model based testing in terms of a model where the model is a category and test generation, test execution, test evaluation or the other classes with the help of all this model based testing carried out coming to the embedded system words that we have gone through embedded system testing words so far in all the units test harness test bed, test bench test equipment model based testing which we have studied today in the previous class test hubs, test drivers we can probably study in the next classes fault injections, MCDC test hooks, boot software, boot photo interface control document break point, simulators emulators, trace profilers or profile data sheet, data in circuit emulator, test equipment checker, static analysis static analysis, core checking in all that will be part of the next future sessions and in terms of embedded system jargons we need to embedded system testing jargons we need to transform them into analysis X, disassembly reverse engineering, life cycle baseline, V model all we have studied we will not go under one software integration test IV and V robustness normal average states transitioning likewise okay some more glossary we studied equivalence class we have studied in software class a portion of the components input or output remains for which the components behavior is assumed to be the same from the components specification error a human action produce an incorrect result evaluation reviewing and inspecting of the various intermediate products and process of the system monitor development expected result we know that it is a behavioral prediction of the specification for a particular input under specific conditions failure a deviation from the system from expected behavior failure a manifestation of an error in the software a fault if encountered may cause a fail as a result of an error basically functional requirement required functionality of the behavior of the system function specification is a document basically that describes the characteristics of the product with regards to with regard to its intended capability high level testing we know as a complete product we are going to test it with the help of high level requirements specification hardware in loop where we put the test with the actual target or the real hardware is used in terms of a testing and initial situation is a condition the state which a system must be in the start of the test case that is a precondition also called it is very important is usually described in terms of values of relevant input data and internal variables okay so that is the end of the this session of the embedded system testing we need to so in the next session we will study about the next methods and all that