 సితోతోతోతో ప్నంథనింద ఆరోసిక్చ్ ప్ప్ ఆరియ్చిచిక్టంగాట్న్ నెవట్గ్మ్. అవట్లులాన్. and its details and in the end we will analyze the unit 1 details what we have studied so far and in session 10 we have understood that different models multiple we started with life cycle models prototype and formal life cycles then we study V model having requirement design implementation on the left hand side and on the right hand side we unit testing integration and system testing and we also went through the multiple we model life cycle model prototype final product these were considered for multiple we model life cycle we also went through the activities that are involved techniques test levels types other issues the mapping to the multiple we model we also studied a allocation of test related issues on the development cycle and on the prototype and on the final product how it is applicable for the multiple we model then we had gone through a nested multiple we model which will be having multiple we models nested based on the components that we have usually in a larger systems and also we went through top slides on testing by developers that means at the initial stage of the testing by development team they can do it since they have the necessary debugging and similar testing environment what we think that for testing so that they can fix the errors or the bugs issues we also through way testing is important at the developer stage is in terms of finding the defects early to gain the components to go ahead with the next level etc then we had gone through slide on testing by independent team we know that all the tests of the team driven by them including the test plan and all the activities so there are five stages of formalization for independent testing planning and control phase preparation phase specification phase execution phase completion phase we had a exercise on that okay we study about mass test planning what is mass test planning typically testing of a larger system with many hardware components hundreds of tons of code lines of code in the software it behaves complex right that means I say complex endeavor so involving many people especially performing different testing tasks at various points in the project very much required and especially complexity in terms of the states software states or the system states and the transitions of certain components exhaustively we need to test it which is something like a higher integration level or that is based on the complexity that the system is involved and the features all this have to be considered as well in doing the testing so what we do is we will have teams say remember here testing me testing is done by larger team so larger team means we can further subdivide that larger team into sub teams and sub teams they can focus delicately on performance requirements specialized features functionality level requirement etc it can be aggregated for the entire software testing for these complex situations mass test planning is very much necessary so what it does is it formulates a mechanism to control the overall testing process so hence the mass test planning has come into picture it basically requires the complex systems requires good plan so that is why we need to have a mass test plan so that is why the term called a mass test plan okay so larger system systems having complex and we just have a high degree of potential testing master test planning provides the control the overall testing process for complex systems what it does is first of the difference between the test lines of test levels is explained so what are the test types what are the test levels that we will see it exercise so basically it will be differences between these two because that is very much important in identifying the elements of the plan so that is why mass test test planning is conducted based on this it is a difference between which aspects are being tested and which organizational entity is executing the test the master test planning then deals with the combination of what so what it primarily does is which or what aspects are being tested that it will identify and it will differentiate with who is doing the test or executing the test that is what is the organizational entity this is nothing but a set of team or a set of allocated specialties or person who will be involved for the entire testing activity so they finally call these all these will be finally put together into the master plan that is why it is very much required to mass test plan in order to test the complex systems complex systems for example I can quote something like break control systems engine control systems we have plate control system fuel measurement fuel management do not assume that fuel measurement is just a lot of control systems are involved along with the sensors and their feedback and there is something called here redundancy management is something like we have quarter plus systems or systems where one or the other channel it could be second one second subsystem takes over for controlling the intended operations or it could be one or the other systems which will be used in order to arrive at a specific decision so that is where redundancy is very much important so we have two channel system four channel systems where almost each channel will have a similar functionality each channel inputs are very much used to arrive at a decision which one is good which one is bad which can be discarded which can be used so typically in a flight control system so we have 11 or actuators this kind of redundancy is used very much so definitely they live with the complexity of the underneath the software hardware what are the subsystems that is involved within that system so to have a development life cycle or to draw a test execution we need to definitely have a master test plan so because it is a complex systems where we identify lot of activities lot of types mechanisms or methods etc so that is why it is very important to have a master test plan okay so what are the elements of master test plan these are just basic elements what I am trying to highlight here or this can lead to further or this can further broken up into minute details as we discussed in the earlier sessions about test planning test cases procedures broader level test types are defined that is the first thing of the master planning a test type is a group of activities with the aim of evaluating the system on a set of related quality attributes that means test type identifies a evaluation mechanism for certain things like it could be other form as it could be meeting certain criteria it could be a quality aspect in terms of coverage or standards etc so basically it does basically accumulates the group of various activities aim is to evaluate the system based on certain attributes of measures the steps state what is going to be tested and what is not so something like a scoping it does the test types will define what will be tested what are the activities how I am going to group it and what are the tests that will not be taken care or that will not be done so that is what the test types state about it means system can be tested for different parts of you it can be tested funcest at the performance this is requirement user so these are some of the quality attributes that describe the various aspects of system behind of course there are standards exist for example ISO 5126 I will just list out so that will give the 9126 O178 etc so some of these standards also can be used as of arriving at certain approaches for test type so these are the quality attributes so that can be used for test types that is so several quality attributes are also combined single test that means single this can have a single quality attributes likewise and test types state what is going to be tested what is not for instance a tester doing the functionality test is not I mean it will define basically who will do what functionality and performance related or vice versa another way like performance and the functionality in terms of which tester is going to do which one the below table we can see some of the test types based on the applicability so as I mentioned here means that it is again depending on the complexity of the underneath embedded system so this table we have a test type its description and quality attributes that are included what are the test types that are considered for master test planning they are functionality interfaces load and stress support it could be manual production recovery regression security standards okay so let's see one by one what is the functionality test type testing the functionality behavior that means functionality is being addressed here so all the features of the functionality of that system will be categorized under a functionality test type basically that is given with the inputs so input definitely will have an error in the inputs to test the behavior of the particular functionality and interfaces as I said basically it is an impression between a couple of subsystems or an external system or a signal or a speed whatever it is so that is what it does testing interaction with other systems other systems could be subsystem or any external entity so this is coming under quality attribute called connectivity sometimes we can also mention as communication may be communication is more sensible compared to others I mentioned it as communication as one of the quality attributes that we can attribute to interface then we have load and stress allowing large quantities and numbers to be processed that is we have system capable of taking certain load or certain inputs certain quantity of inputs say 5000 rpm or 7000 rpm range is something like 1000 to 7000 rpm let us say 7000 rpm let us say for example those kind of requirements or those kind of test type is coming under load and stress so what we want to subject the embedded system is allowing the higher number with a fully loaded system so that we will test the behavior of that particular condition with the large number of inputs or large quantities when it is fully processed with these inputs okay next type is the support providing the expected support in the system that means the types the test types which will help us in terms of directly or indirectly arriving at some of the results that is the expected support in the system that means overall understanding of the some of the features of functionalities we have to arrive at some analysis etc all this will be part of this support testing it is attributed as suitability next one is a production test production process that means we have a production environment all the production based testing will be done under this so that will have a quality attribute in terms of operating operating and continuity that means a embedded system should be able to operate with a minimal introduction or maximum automation and it should continue the same way as it is expected once it is through with the given set of test procedures then we have the recovery testing recovery and restart recovery that means the like reboot power of requirements those kind of testing could be coming under this type of recovery where we have a test recovery and the rest of the facilities will help in doing this type of test this is the standard recovery quality characteristics then we have a regression you all know what is regression testing whether all the components function is correctly or the system has been changed so why system will change because it could have been bug fixed or any upgrade would have happened or any testing failures to be able to reproduce etc so all this function of regression regression can have any of this quality characteristics that is where they are put as all then we have a security testing security so all security requirements sometimes we can add safety requirements also under this type called security then we have standards rating the compliance to standards like this much percentage of coverage should be there complexity coverage etc we will all come under standards against the standards these steps are defined this also is categorized under security safety because for example if you take airspace projects what happens is they call it as a airport safety that is how the product is safe when this subject to certain tests all those features come under these standards and also it should be a user friendly in terms of its operate operability and usefulness then we have a base in the resource system some of the systems that embedded system primarily can use like power regulators memory communication all these the measuring equipments all are called as these resources type of testing capital under this type of course we can combine with this also interface as well as we can the broad day you can categorize communication this again as it is objective based on the complexity of the system so these are some of the elements under test type element some of the test types that are listed we can add more based on the need of the complexity that we have we need to draw this because this is a very important in terms of master test planning the next one is test levels the second element of the master test planning is test levels a test level is a group of activities that is organized and managed as a activity so test type is a group of activities with the aim of evaluating the system on a set of related quality attributes here we define the test types under the quality attributes what is going to be tested in test levels what we do is we list out a group of activities and how it is organized and managed as a set of activity is going to be defined so the table below is some of the common test levels for a test subject to supply okay so before that and few words is that test activities are performed by different testers and teams as I said because we are dealing with larger systems so we may use a different environments or various environments pertaining to certain set of features that is under test so all these organizational aspects are the reason for introducing this test level that is why it is important to have a test level defined for a master test plan test levels basically state who is going to perform the test and when it tells the states basically who is going to perform the testing and when it is going to be tested the different levels are related to the development lifecycle of the system it structures the testing process by applying a principle of incremental testing you know that incremental testing is a development process we develop small small pieces first we integrate one by one and then incrementally we build the system when the various components of satisfactory quality they are integrated into larger systems or subsystems so likewise we will develop the bigger systems these in turn are then just try to check if they conform to higher level requirements so we will start with the small subsystems and components then we will grow up with the higher the incremental testing finally we check if they conform to the higher level requirements what is integrated for that particular system and we know that low-level testing low-level testing focus on isolated components executed early in the cycle in the bottom most part of the remodel high-level tests are tests on the integrated systems or subsystems executed later in the life cycle of the retention of the remodel it could be simulated target based based basically on the real life and so we will go through the table of test levels all of this we have gone through in our earlier slides in a different class sessions in this unit we have detailed but different about mass test planning which defines each of those type of tests or levels of tests categorized under the mass test plan okay hardware unit test hardware integration test model in the loop software unit test post or target testing software integration test hardware software integration test system test acceptance test field test again these are all subject to applicability of the particular embedded systems as we need we will draw a different levels put an integer and that is test hardware unit test test level is low and it is done in the laboratory in the lab testing the behavior of the hardware operation in isolation as I said it is a component level unit level it is called you can also mention it as a unit test hardware integration test that means it is also a low level this will be done in the laboratory testing the hardware connections and protocols basically we have CAN AFDA XETA whatever it is all this are part of hardware integration where we basically plug all the hardware processes underneath the basic embedded systems and we will test it then we have a model in the loop this is categorized as high and low depending on what are the type of models that we use so we know that in initial stage of the project we develop the model to prove it whether it is going to work so basic purpose is proof of concept testing control is design optimization some of the complex algorithms may need to be proven first before we actually develop and develop of course in terms of memory performance you may have to optimize so in order to architect additionally you may have to do design optimization all this will be covered under model in the loop also called as MIL the next type is software unit test host or target based system of course I told here as hardware unit test basically different hardware components want to be tested we can do it or if it is not required then we can go ahead and use software component test actually because this is a component that those units are fine because what will happen is we will do software unit test on the particular unit we may come across bugs or issues so we presume that sometimes there could be hardware problem there could be software problem etc to make it clear that we are good at hardware we start with the hardware unit test this can be done by a hardware team or a specialist who know about or who is knowledgeable enough on the hardware components is again a low level test laboratory this host target processor is used of course laboratory has both of this typically it is used in the lab testing the behavior of this software component in isolation then we have the SIT software integration test it is also low level it is done with the help of that output connected to the host we know that how the target is interposed with IDE then debugging in one break point and all that we use it to do the integration basically here we try to integrate the different software cases within the system and integrate it we will pass the parameters test against the parameters whether the intended behavior of the flow could be control flow or data flow that will be the aim of the test the testing interaction between software indication test similarly the next level of test is HLID where we focus on the system and their system behaving when it is integrated with software components and those software components how they are doing on the other one we focus only on the software to solve the interactions here we focus on the software to hardware or hardware to solve the testing interaction between hardware and software is taken here HLID here also we use the same lab equipments measurement equipments and the target board with the same language whatever it is through that the software is working fine on the end target system then we have the system test as a integrated unit the embedded system unit whether it works fine as expected is done with the help of this level called system test is a high level of course the HSIT is also high level simulated real life that means accept few of the items we will almost have the entire system say suppose exception is what similar to real life that means it is not the actual system running on the field suppose it is a fuel measurement or if it is a speed so we will not take the embedded system target onto the road with a high speed of 100 miles 200 miles etc we will not do it instead what we do is how it the speed or the fuel actual fuel is going to fit into this we are going to simulate that those inputs and those inputs with those help of those inputs how it is behind what we are going to test it so this is under system test and AB is to test whether it is as specified in the high level requirements basically due to the HLR or SRS or SRB software requirement specification software requirement document or high level requirement a subset of high level specification is business requirements or user requirements this is tested under acceptance system is whether the product can be accepted this is also done with the help of simulated real life with the actual end target testing that the system for a purpose for the user basically we look from the customer or user perspective in all aspects of acceptance and last one last category is again subjective we can use the field test some of the complexity some of the features need to be tested with the help of the environment something like ambience or weather of course weather also taken care like we have a temperature same or more they will be doing but other integrated system sometimes we may have to test on the field those things are categorized under field test testing that the system keeps working or the extreme conditions could be the attitudes whatever it is all this will be taken care with the help of field test so this is the test level under the master of test planning so we have two types test types which deals with what is going to be tested and what is going to be tested we will summarize all the group of activities basically to evaluate with the help of the quality attributes then we have the levels all these attributes have been defined how they are going to be tested that means what level you are going to test it let us say model in low or software integration system test etc for the component test all this will be considered for master test planning so in continuation of the master test plan here is a picture we all know that we understood already about this is another way of depicting master test plan do not get confused what we have studied in our earlier it is same where we have drawn the picture of different levels of what test we have a system test it is all part of the master test plan so master test plan can be viewed as the combination of what has to be tested it is test types and who is going to perform those test activities or test levels so that is about the master test plan of course we need to coordinate the various test levels it should prevent our redundancies and options it ensures that a coherent overall test strategy will be implemented decisions must be made as to which test level is most suitable to testing testing which system require quality attributes so that decisions have to be taken care of course the specialized resources such as the specialized test equipment measuring equipments and expertise to use that all those have to be allocated to the various test levels of certain functionalities or features of course there is a timeline or a great period that also need to be drawn inside the master test plan so an overall test plan has to be drawn up which defines the tasks responsibilities boundaries for each test level very important to understand that boundaries in terms of expertise or limitation etc at each level of the testing methodology which spans across this master test plan so that is why this overall test plan is called master test plan basically this is done by the project manager and the project manager it could be a test manager depending on how it is organized in the master test plan so he will delegate the above ones all tasks who is going to do what all these responsibilities are done by the test management team of test managers or project manager so overall is responsible for this so that is about master planning test level and test types so three main areas of interest for master plan that needs to be considered by a project manager or test manager before he does the master test plan test strategic choices what to test and how it is how through how thorough it should be there is one of the important test strategy choice that tends to be done allocation of resources expertise resources could be from the material for the components one said and sources could be from the human resources in terms of expertise or the experience that is required to be taken care and how all these different entities for the resources of the components all they are going to be communicated there could be an element of a vendor also sometimes we may need to help some of the power supplies or the measuring units how we are going to communicate within these disciplines there also is an interest for the master test plan and of course the master test plan will know exactly what is expected from each of them and what level of support and resources they can expect the master test plan serves for each test level we can have a detailed test plan for each of the level we can have a test plan for acceptance test we can have a test plan for a system test integration testing test etc all this combined together the master test plan is formed it does not need to prescribe for each test level activities must be performed in detail but it can point to what can be done that will be there in the detail test plan so whatever the details are decided so so the master test plan deals with decisions that concern areas where the various test levels can either help or hinder each other so all these pointers will be proud of the part of the master test plan of course there are other aspects in master test plan also just list out just for sake of listing it out is not mandatory but for understanding sake we do it that is a strategy we know that we need to have a test study and team for the same infrastructure and organized organization I would say who is going to do what this is basically some of the key elements of the master test plan and we know that in the TM method all these above are drawn with the help of data principle what is data principle lifecycle infrastructure techniques and organization so this all will form the master test plan so master test planning activities I will highlight a few points which are important formal the assignment global review and study that means what is the strategy in the previewing outcome of the test and how the team can start in terms of study then determine the master test plan what is the basis on which the test strategy can be determined specific infrastructure define the organization determine the global schedule it means based on all this five above schedule is drawn in terms of using this and coming up with an execution plan for the same so these are some of the activities so this is done by a test manager and he starts as an early activity in the development process itself as a overall test process this is done by developing a master test plan having formulation of overall objectives responsibilities what is the scope then a global analysis of the system to be tested and the development processes come out the next step is to use the information what is being discussed and carried out about how we can measure for taking the next step in terms of it could be a quality performance basically we should have the aim of successful system having done with the complete testing so this is the determination of the master test strategy and involves choosing what to do what not to do and how to realize the test managers may be the responsibilities it accomplish this whatever we have discussed so far is getting accomplished with the help of master test plan the required resources infrastructure or personnel should be defined to manage this the communication between all required reporting must be defined how it's going out the individuals are going to report and what frequency and to whom all this will be part of the organization and and the test plan a good master test plan is not created in a quiet and isolated office it is a highly what is it called political task I mean it has a lot of buzz lot of people involvement all that basically it requires a discussion along with the bargaining and persuasion throughout the organization so that is a good master plan drawn by the project manager of the technical test managers so that is about the master test plan the last part of the okay so of the session I'm coming with couple of slides just to bring some of the pointers which will highlight overall what we have studied in this unit in terms of principles this is a good set done by Glenford James so just want to point to this so 10 principles of and necessary part of a test case is a definition of the expected output or we know this we need to have an expected output and the programmer should avoid attempting to test his or her own program that means the independence should be maintained that means the programmer should always like to test as an independent way or he should not be biased way testing a programming organization should not test its own programs that means one is a programmer level another one is a relation thoroughly inspect the results of each test means once we have the results outcome even though it is passed we should have a justification or we should have a validation why it is passed we should inspect it you should watch it you should make sure that the right results are achieved test cases must be written for input conditions that are invalid and unexpected as well as for those that are valid so both have to be taken care examining a program to see if it does not what it is supposed to do is only half the battle other half he is seeing that the program does what it is not supposed to do so that is both of them are very important first we will do what it is supposed to do the example telephone instrument what it is supposed to do and the same telephone instrument what it is not supposed to do when something is not required or something is not specified in the specification so in principle avoid throw away test cases unless the program is totally throw a program so unwanted things we should analyze properly before we make it a unavailable or unwanted do not plan a testing effort under the tacit assumption that no errors will be formed so we should not waste having testing effort just for the heck of putting some effort should have appropriate effort on achieving the success of the test it could be hundred percent at one go or it could have some errors but effort should be there always and effort should be on the right direction the probability of for the distance of more errors in a section of a program is proportional to the number of error already found in that section is a very interesting principle is always proven I have seen this and I am for this actually basically why because I have a source code of say thousand lines and I found say some 20 bucks initially and at the end of the test of this thousand line I found say 100 bucks definitely those 80 bucks could be based on the 20 bucks which are found it in the interest so there is a proportionality that already found errors could result in more errors of the new new errors or more errors that could be another setting is an extremely creative and actually challenging task that means testing is equally important and equally well rewarded task considered in the industry so that is about 10 principles of software testing we have also beta this is basically did not be part of the embedded software testing but some production environment is being used especially on the application side but I have heard that alpha and beta testing are also used in some of the embedded systems production environment so just to understand what it is typically used in production the focus is focus of the testing is to simulate the real users by using black box and white box techniques alpha test done by the customer sorry alpha test at developer side by customer this customer will have an idea of the product at the development site so developer looking over the shoulder recording errors usage all this from development perspective is taken as a test controlled environment it means still it is not a full-fledged or released environment with certain conditions that are carried out so the next type of test after alpha test is done we have test at one or more customer sites by end user that is the end user real user of that will be testing the product at the karma site and there is no developer involved for beta testing is something like a live situation where the product is on the market or the field developer will not have any control only he will be reported of the issues that is coming out of this customer records problem so that is about alpha and beta testing there are a few differences between that alpha testing performed by testers who are usually employees of the organization beta testing is performed by clients or users who are not employees it is on the field alpha testing performed at developer side beta testing is performed at client location or end user can have the product and test before he uses actually for his main reliability and security testing are not performed usually this may not be applicable for embedded testing but in general is how they are using it reliability security robustness are during beta testing alpha testing involves both the white box and black box it makes beta testing typically uses black box that is system level alpha testing requires lab environment or test environment beta testing does not require any in-house environment or testing environment software is made available to the public or the end user and is said to be real-time environment. Long execution cycle may be required for alpha testing only few weeks of execution is required for beta testing critical issues or fixes can be addressed by developers immediately in alpha testing most of the issues are feedback is collected from beta testing will be implemented in future versions of the product so alpha testing will be have a still chance of reverting back with updates of the product whereas beta testing it has to go through different chain of implementation versions test to bring you back to the factory and all that alpha testing is to ensure the quality of the product before going to beta testing beta testing also concentrates on quality but the other user inputs on the product and ensure that the product is ready to ready for the real-time so I have covered alpha and beta is some embedded systems can be covered under this this is basically they using industry like telecom where they have OSS like Android or Google so there is alpha test released products and given to the customers group of customers will use it in the real-time environment and that type of testing is called the beta testing is the most of course end of session we have words repetition of all maybe we can add some of this some of the right terms from this class something like schedule strategies strategy strategy both of them which look at this plan of course part of this plan we all know that I think one word is a model in the loop so this is an important it's also called as ML the will also be the master test model but safety and just safety hardware software integration software software okay the last part is the question what are the main elements of master test plan list the principles of embeds of co-question and their importance and maybe we can have a question on beta testing this is the differences okay so with that we will conclude the session in the next session we will take up for the next unit of software testing maybe I will recap of all the sessions that we had for some duration to begin with that in the 11 sessions we have then we will take up the second unit on testing methods dynamic testing model based testing college testing etc as part of the next session okay thank you