 we have seen some of the basic concepts of the embedded software system and the embedded response the basic in the user system so highlight what are the types of systems and what are the basic requirements of the embedded software system and the embedded systems and the embedded systems. In the course of the initial slides we had course objectives, the process in C learning and some of the embedded systems basics we have defined what are the differentiators between general and embedded systems also we had a explanation about elements of the embedded system so what are the contents of the embedded system, the typical embedded systems how to do the problem also we had an understanding on the embedded system development how the target is connected in terms of the development similarly we had the embedded system development. This is required why because some of the language testing we may have to understand in the three language which will be for debugging and testing perspective on the target side. In that aspect we understood that language three and embedded three is a complex where we had the first code compiled using the host machine and the linked in the same machine which will be the x86 for example which will be run on the same host machine whereas embedded C we have a C and embedded assembly code, host compiled and assembled and those object files are linked together to produce a target table and we also had a basic understanding of the embedded system we had gone through some of the testing definitions from i-tryper me and the source we understood that the role of testing is very what is the process for defects if the testing is done at different stages it just requires the line coding of the screen for the application to run and the C process elements for embedded software testing like we have planning documents, technical inputs, guidelines and testing process for discrimination as part of the process planning, specification, execution for each network. We had gone through the system test setup, embedded system test of how it will be and embedded system testing setup in terms of end-to-end testing setup how it will be, there is a host PC connected to the target system there are tools which will be running on the host PC with the help of tools the testing ports will be covered to the target system and the results are zero and it may not be enough to access the target system with these tools so there is a need of a real time input, the real time input can be covered with the help of a test panel or a prequel panel which will provide input to have a lot of these tools etc. And we have gone through different test methods, access and testing, system testing, integration testing, open testing, the level of test methods and we detailed out about access and testing, user level specification to be tested, criteria setup so also we had gone through different testing, integration testing different levels at the bottom of the integration and user level testing where small units of the system are under test and these could be core functions which can be considered as in-temp test also we have a core as the type of testing which is called spectrum which can have different brand, manufacturing conditions and production conditions the tools are used for testing or LRA, multiple units also can be used or they are called logical units which are connected to the target system we know the two methods of testing, black box and white box testing black box testing where expression specification without the intricacies of the embedded system are constructed and tested whereas the white box approach we need to make the embedded test box to understand the internal structure for the knowledge of the flow that is happening within the embedded system of course there is a regression testing multiple versions of software we could have fixed some of the parts that had some of the outcome of the previous testing and those can be tested using regression testing I have a few questions on the past sessions probably we can make a note or we can come up with an answer on this or we can touch this in the next session for an answer on this so there are about eight questions first one being what is an embedded system and how it is different than general system please define one of the questions in the answer identify at least five embedded systems many embedded systems that we have discussed earlier we can list them so why embedded system testing is needed we know that embedded system development is there so as an entity separately we need to have an embedded software or embedded system testing why it is needed the next question is about what are the different types of embedded software life cycle that embedded software life cycle is DLT followed or embedded software at least name them one or two what are the test methods under embedded system testing next one is what are the main differences between the black box and white box testing under what systems we use white box testing what are the different types of integration based on the different types of integration these are some of the questions if we start getting answers then our fundamental is the concept of embedded software and embedded software will be good to clear up for the next session so except few answers for all these questions in the next session we can proceed and if there are any doubts we can continue in the next session okay so coming to the next session session three we will go to the test case design development and test procedure what is being used or followed the contents of this test case procedure test case design and development are as below we use the different methodologies and all that so general testing philosophy we will highlight and testing the concepts in terms of VNV and debugging what is it we will go through and what is the test plan so before starting a test activity we need to have a definite test plan test plan will cover all the aspects of the embedded testing so because the test plan embedded testing is not complete so that is the first thing that we will do so we will touch upon test plan along with an example next one is the test specification the test specification is the thing that identifying the test cases and how it is designed so what are the different methods so what are the various requirements that we can start with an example template of how the test specification looks like we will try to go through that the next one will be on the test procedure given a test specification identifying the test cases how are we going to develop a test procedure test procedures are something that is done by step action basically they are practical steps in order to achieve the test procedure to do all these test plan specification procedures we need a set of standards that is a test standard so the standard will basically define how the specification should look like so what are the elements what are the rules what are the guidelines that we need to consider for developing the test that we will discuss along with an example and one of the test method that will be used in embedded testing it is called the EMDA method we will brief and go through few slides what is it about and this is one of the method that is being followed whereas whatever we are going to highlight in the test specification procedure and standard is of generic method that is getting followed in the one of the typical embedded industry EMDA method we will try to see what are the different types of that it is getting followed what is that mean that is separately followed in embedded software industry adopting EMDA method and we will end with a few questions on the above contents we will see if we can see we can have the questions in the next session or based on the progress we will see for example the specification and standard we can conclude in the next session so general testing philosophy so to test an embedded product so we need to adopt a philosophy how we are going to test it given a product what are the elements that are required basically to go through the testing method or how a user should look for what are the philosophy so first thing is to go through the requirement or analyze the requirement for what is going to test it then I am going to identify test cases and conditions here what we do is we are going to identify test cases put it into the requirement so each requirement is going to be adopted and to test it what are the conditions in it so that will have the identification of test cases once we have the test cases identified we are going to create procedure so as I said test procedure we are going to start with what are the conditions that are mapping to the test cases so basically anything test cases and conditions are optical steps whereas testing test procedures are practical testing method then we have exhibition so basically when we are done with test procedures mapping to the test cases the practicality of the test procedures are done with the help of execution so testing execution are conducted with the help of test input as per the test procedure test input will produce the input condition and once the execution is done we are going to come up with a result then once we are done with the execution we are going to have actual results and we are going to compare the results with the accepted results with the accepted results will be part of the test procedure to what should be accepted so based on the comparison that we have with the actual results to what is the expected results we are going to report the particular test as past or state I repeat the output of the execution that will be compared with the expected results the expected results are identified with the test cases and the result of the comparison output of the actual results will highlight past or state that will be reported for each of the test the phase test should discuss the fault I mean the file will identify the issue with that particular test and also it will highlight of the identify the faults of the particular unit this is the overall test philosophy for any it does not matter which domain whether it is a telecom product or it is an automotive control unit or it is an cockpit control unit or any of the networking crosser or any embedded system so once we understood that general test philosophy of the various elements we know that we are going to do verification because we have done the system correctly correctly the system has been developed and with the validation we do we have done the correct system two aspects one to develop the system correctly and the correct system is there or not for example we are going to develop a telephone instrument and test it whether it is functioning properly or verification validation is that whether the telephone instrument is built as per the need instrument is correct as per the specification validation another point is that testing does not mean that verification of some product or the product running all the time or a program or a a project is not that alone it also has some of the requirement we need to have a review of the documentation we may need to do some core instruction so we may have to do some static analysis so testing involves all these aspects but then only testing is complete so part of the testing so we need to understand that the end is one of the process we compare all these things so it comes under testing every testing does not mean only B and B or verification of a running program it also has various other aspects like requirements analysis review of the documentation that is pertaining to embedded system development it also can have a debugger work through core instruction and some of the complete executable code analysis static analysis dynamic analysis etc so in that aspect testing is complete now the next testing and debugging two terms will be used for embedded system testing they are using they are used interchangeably definition is for debugging it is the act of attempting to determine the cost of the function of malfunction detected by testing or by completely use the function that means debugging will help in understanding and identifying some of the malfunction embedded system so that is what we have mentioned and purpose of debugging is to find the interpretation which is which is like a failure and rework on the issues to implement the program field that can correct basically we use debugging the debugging for to identify the cause of the symptoms to determine the cause of the symptoms so that is causing some malfunction or some issues in the embedded system second thing is to fix the first convoy so by fixing we will reproduce some of the symptoms for whether it is reoccurring or not with the debugging we will reoccurring and once we have the program debugged and fixed we will do the testing so the testing will clear the bugs so purpose of testing is to detect and detect or identify whether the program has a cause of the cause of the one so said all this test design like now requirement the first step for us to do a test planning test planning is the most important method that will be followed as the first step for any of the embedded system testing so what are we test planning items that are which can be charged here there are various tools various blocks to start with we are going to have a project specification or project plan the project plan as an input we use the program the test planning will have an input from the project specification that means the entire project is the plan under the project specification it will have the requirements are made out how the coding is done what is the environment includes what are the development tools so this will be part of the project plan similarly we have a testing process defined during the planning so what is the testing process I am going to follow so what are the technologies or the review checklist or the guidelines and standards for the testing process then we have test approach like how the various requirements are the functionalities of the embedded system product will be tested using what approach as we discussed earlier the approach one could be using system testing approach two could be using integration testing so all these approaches will be used in terms of test planning approach one approach three etc test approaches etc will be part of the test plan for doing the test planning as we said there is a testing process which will be used for testing the testing process along with plan input and test approaches will form applied test process the applied test process will be a specific set of test processes that will be followed for all the approaches that is going to be used for the embedded system testing finally we will come up with the test plan the test plan will have all these elements for the embedded system testing and of course while developing the test plan there could be some deviations the test strategy the strategy for each of the approach like system level test there are different functionalities for each functionality I can adopt some strategy I have a group of functionalities in terms of performance group of functionality for user specified requirements group of functionality for customers etc so this all will be categorized under test strategy and we may take some deviations which may result in alternate approaches all will come under test plan of course this test planning needs to be aligned within the organization so a company test approach together with the test process which is defined in the organization adopted to the current project based on product specification as I said between primary input so all these together will result in an applied test process and that is an overall vision how we will test the embedded system in particular the product this vision is the implemented test plan in a test plan that means the complete vision or the test approach using the particular applied test process will be part of the test plan so basically the test plan is a document written in a natural language I will explain the test plan as an example so that you will understand the process of creating a test plan is the test planning mostly it is done as I said in the early stages of the project sometimes it will be done during the development phase also that means while we are doing the development sometimes it may be required that some prototypes may be required to be developed so that develop the prototypes will be then identifying some of the approaches so that we know what should be considered for testing otherwise we have to be developed during the specification project plan the project plan will have after development plan after quality plan after configuration after testing plan all together will be developed for the initial phase so of different method we will discuss in the later session that time we can develop all this development planning of what will be used in the initial phase this is about test plan next is that the goals the goals of the test plan so the test plan is a high level test plan and the more detailed the test plan also will be there there will not be enough to have a high level test plan sometimes it may have to require further detailed test plan for the specific embedded as I said earlier the embedded system can also have a subsystem subsystem subsystem also can be called as embedded subsystem there may be a need for subsystems so those can be calculated so that is what is the high level test plan and the detailed test plan so this should be aligned with the project plan project plan as I said is comprising of development plan test plan, peer plan and all that and the test plan should relate to the project plan and of course the test plan should follow quality analysis will point to some of the standard checklist review process and all that so there should be a good pointers in the test plan it will follow the player plan configuration management preparation management all this will be highlighted in the test plan how the test are taken care how the defects are maintained how different requirements are ingredients are managed so all this will be integrated together as part of the test plan so before some of the test plan go and the purpose of the test plan is to basically plan it organize it and then follow what it means is we need to define a workflow for ender software that workflow will be defined in the test plan basically to give directions on how testing should be performed which means to which test parameters processor, methods tools and templates to be used all this will be part of the plan and once defined all the test parameters, methods and goals it needs to be controlled and it should be performed and basically we need to coordinate our control and view continuous feedback to the workflow for a period while testing is in place also we need to create control adjustments of enforcement although there are certain deviations that may have to happen while doing the testing there should be provision in the test plan which will allow us to control such enforcement all this will be part of the test plan and the typical test plan contains as per the IEEE standard as below there are different contents across the organization so they follow not only different organization between the organization also there are different domains on different projects they follow this is typically driven basically from the need of the customer or the need of the product some product may need a stringent test plan in detail again depending it depends on the definition of the product at what level that means for the plan in terms of indirect system testing so as per the MCR IEEE 829 and 298 test plan the contents are test plan identified introduction to the test plan what are the test titles and what are the features what are the features that are not required to be tested and the approach the test plan and criteria to define parts of the test and suspension criteria and the the requirements we have the requirements and while testing we may come across initial stages so in that case we may have to that test and go ahead with the next test and one so it is fixed how we are going to resume the test and how are we going to define the criteria for pass or pay so next once the test is done how are we going to deliver the test before and what are the testing tasks what are the environment we need for doing the testing so who are all responsible and what is the responsibility and what we can get responsibilities of the staff what are the staffing needs and what are the scaling them when we are in the system tasks basically why they are listed out it is very necessary that embedded system there is an equally available in terms of the end product what is the end product what is the embedded outcome of the embedded product so all these aspects in terms of the specification requirements may be some of the design elements have to be undergone that means the system knowledge is very much important in terms of the embedded system because the system knowledge will be difficult to test it basically test it when the user cannot define a test case or test procedures without the system knowledge you cannot do a test this is the testing so this will be defined in the test plan I mean it is the testing aspect like when the test is to be defined when the test plan is to be available who is going to be deployed for which testing when the request is going to come when your registration test is going to happen we can have an example of a test schedule how it will be on next may be in the next test what are the test risks for those who are in what are the risk to involved for doing that and once the testing in all aspects are done who is going to approach in the organization like there are different departments which are responsible for approving the test plan available the testing outcome so those will be part of this approval so these are typical test plan in the example that I follow in the next we will either directly or indirectly follow this standard if we have this test plan so the test plan identifier will not take place much as introduction items so as we did maybe we can detail out all these aspects as a point so once we have the test plan contents so we will have part of the contents the test strategy definition these are some of the important things to that define so what is test strategy so we will define which design techniques needs to be applied for what sort of like we have test identified for a particular requirement so how I am going to test test design what is the test case or the technique that I will apply for that requirement and to produce a good test strategy or what is the test condition or that are needed or to be developed to do the test of requirements or the function of the MRA system so this is the first thing that I need to have an awareness of that means the test strategy which needs to be done first before or as part of the test plan test plan will strategize or cover the aspects of system level, user level integration level, component level that means the test plan will identify where we given a embedded product how the embedded product will be tested as a whole in the what level what is going to be tested the system level or user level the fitness to use parameters and integration we have top of bottom top of top of bottom of process we use in terms of integrating the various model similarly we have unit level, component level testing that also will be part of the course test strategy or the test plan strategy is depending on the positive size and complexity this needs to be without and test planning can be divided into a hierarchy of test plans for example high level test or master test and some detailed test systems can be powered and detailed test can be coming under detailed test plan so I will go through an example of test plan how it looks like basically this provides an example of how a test plan looks like looks like how test plan test plan example test plan example let's understand so basically this is a template typically used the product I use here with name as xxx it can be anything or instrument any name can give it so we have a header of the test plan and the name of the product if you have any questions probably you can go through this I can share this and we will have all the document distribution icons like test strategy and the standard strategy covering all the levels of now we will go through an example of test plan so this is a typical software test plan which is followed in the industry I took an example name the product as xxx you can name any of the product name such as the control name or any instrument name or anything the header will test plan identifier identifying the products distribution the various people or the various stakeholders required so we will approach this particular test plan so this is very important because as I said test flow this document is controlled in the configuration and this will be released as a beginner for the test cycle so these are the header footer and the and we have the revision and whenever we update this test plan as I said there could be some variations there could be some changes in the strategy for the period of testing that will be listed in the industry at any time and the impact and of course we have the let us this is the introduction of the test plan the applicable or the input document the responsibility the method verification environment I have used verification here because as I said it is used as a interchangeable reports along with the testing we can observe verification so we have verification activity verification environment traceability to compliance the method mapping to the how we are going to trace it to the embedded test and then additional computations standard and the part of the class okay so in introduction we are going to introduce the purpose of the test plan so what are the responsibilities what is the theme control what is the compliance and the evaluation and action I have not put any text under this because it is all implied or understood may be in the end create an example or the audience can create an example probably I am going to assign activity which will help the audience all the students to come up with the good test plan hopefully that will be an activity we will take up so first thing will be the introduction with the purpose in control compliance and abbreviation as part of the test plan next we have the applicable or the referenced part of the test plan so what are the external documents we are going to refer as part of the development of the test plan this could be a customer given an document or any user standard or a different standard or a specific standard which are used for developing the test plan similarly we will have an internal applicable that there is an organization that plays a review guideline whatever and verification responsibility for every project or product there is a definition of responsibility that responsibility will have identifying identification of the organization the organization will identify what is the team who is responsible for what that could be 10 member team in the 10 member team what are the people responsible for doing these tests who is responsible for delivery who is responsible for process arrangement who is responsible for quality and who can do an independent validation or verification etc so this will be part of the verification responsibility then we will have verification methods so this will basically identify the testing methods like high level testing, lower level testing all the testing methods are defined under the testing methods and as I said earlier testing is not the only method that is followed there are other strategies which needs to be used so other strategies could be static analysis or any of the dynamic analysis or it could be a review if I do a review so what is the review method I am going to do like offline review or peer review or self review whatever it is for the testing review analysis in terms of code analysis or the code instruction or static analysis of the entire project view these are all will be part of the verification method next we have as part of the test plan verification activity once I define all these methods so what are the activities I am going to conduct to verify this it could be planning process verification that means the test plan as per the test plan the process is used or the process has been followed or not and verification of system requirement that means the verification activities are taken care of system requirement and similarly software requirement or the high level requirement and verification of outputs of integration or verification of testing results that means entire test life cycle will be verified with the help of these activities that means the planning process or the requirement verification or some of the software requirement or the high level requirement or the models are integrated the outputs of the integrated model then the entire testing the output or reports or the results will be verified whether the results are as per the expected output and all that the total outcome of the testing so once we have the verification activities we will identify the verification environment what is the environment I am going to use in terms of laying out the complete test plan element low level testing high level testing what is my environment I am going to use for the different testing strategies and to have that environment what are the tools I am going to use test PC desktop less possible assembly code less possible tools that are going to be used for doing the testing similarly if the tools are used from how they are going to be qualified or whether the used tools or qualified tools are applied to the standard whether they can be acceptable by the customer etc all this will be part of the verification environment next we have the feasibility to compliance how I am going to test all the test testing requirement that means I have a testing strategy for the low level requirements etc so the objective is to cover 100% of the environment so how I am going to achieve how I am going to map the various testing all this will be part of the procedure for example it could be requirement based testing for testing the requirements so that should cover up the 100% of the requirement I mean do an integration testing mapping to the design of the environment system so to map to the 100% of the design similarly low level design or any of the low level requirements which are done with the uncomfortable testing for the compound testing that should be compliant with the low level requirements so how I am going to trace it in the testing and any other additional configuration that I need to consider for doing the testing it is something like a partitioning partitioning means here embedded system will have mostly two units something like low level partition or high level low level partitioning with different application for example low level partitioning is a boot loader or boot software high level partitioning is an application fitting on top of the boot software so both are needed so given an embedded product it can be comprised of such similar partitions and the applications of course will also have different partitions connected into the where we have the multi task embedded applications so if there are any partitions to use the embedded system product that also needs to be highlighted that needs to be considered similarly the for compiling the test software or the embedded software that has to be and for doing the testing we may use a lot of tools or we may use some of the standard library or provided by the windows one of them the group of them is called as commercial official tools it could be a library or third party support tools all this will be configured for doing the test that needs to be highlighted last but not the least standard techniques and guidelines as I said these are some of the process and the next that needs to be added for doing the embedded system testing we will go through them separately in the next slide so I will again highlight the proper test plan or verification plan it will have the verification responsibility verification method the verification activities and environment that is used for the verification and how I am going to trace it the various verification activities etc and of course we have any considerations like partition the compiling of tools and last we will list out all the standard techniques and guidelines we will also point to the various process that are used as part of the techniques and guidelines so this is all about this plan which sees the embedded software test plan as template as I said we can create an example test plan any that we can have it as an activity or as an event some stage of the last session of practical so that is about the test plan example so next coming to test specification test plan layout we are going to start with the test specification so what is the test specification test specification defines what is the test test specification is part of the test web I will tell you what is the test web test process basically identifying test test data and test condition this all will be under configuration this is my name separately only coming and test specification will have basic building blocks of test papers test specification will have instruction to use the scripts test specification will identify the requirements test specification will report the complete test test cases so this is about test specification and I will go through a test specification exam and later we will test based on the test case design of course the test case design is part of the test specification this is typically a test case development or test specification as it is called the template I have used is similar to the one in terms of the structure how the template is like same as the test plan we will have a correct name we can identify with our own name the software test cases will be identified as part of the test specification same as in the earlier one where we have various stakeholders in the first page then we will have the same so basically we will update the document whenever there is a change in the test cases once it is a baseline there is a possibility that the test cases themselves will have some challenges which needs to be reworked test specification basically we will have two basic sections the first section will identify the test cases for each of the requirement the next section will identify the group basically once we develop the requirement we are going to group it why we need to group and all that we will understand when going through the test specification for doing these test cases there is a precondition that means there is a precondition so that means to be in place for developing the test case so test cases identification for each of the requirement that is the first section and next section above that means all the cases so whatever we have identified here that means each one is a test case each one is a test case test case 1, 2, 3, up to 1 for each of the requirement there will be multiple requirements and each requirement could have multiple test cases once we identify test cases all these test cases will be grouped so why we need grouping and all that I will explain later we will understand the need of grouping so that will be done under the second and for doing the test cases we need to have a minimum preconditions that will be done for doing the test cases of course we have a test scenario for different static analysis method these are something like different different tests that are taken under this what are the different types of static analysis which are the functions that are taken care in this test situation what are the non-functional requirements all these should be highlighted as part of the appendix so these three basic sections will construe the test cases these are the test cases and grouping of the test cases I will detail out or we explain each of this in the next session so in the next session we will test upon test cases design how it is going to be done we have seen in the test case section that we are going to have test case of identified and test case conditions also we have grouping so that is all part of the test cases design and what are the elements of the test case design and once we are done with the test cases we are going to have test procedures as I said in the beginning test procedures are nothing but step by step actions of the test cases in terms of practicality in theoretical nature but test procedure is practical in nature that is all about the session we will have the next session starting from the test specification and the test cases and also we will have in the next session test case design test procedure and the last thing will be the test standard what are the test standards that are used we will go to an example and the typical test standard we will go to that will be part of the next session