 and this unit we will try to study the test management aspects in terms of the effect management, test management and control and test management tool or defect management tool. and test management tool defect management tool then after this we will go through all the units what we have to be quickly so that we are done with all the critical parts of embedded software testing so before this I will just go through what we have understood in the previous session about test management so we had gone through change management document the change request analyze the impact approval is required from ccb ccb something about change control board who will approve for the changes after analysis is done and the final approval with the senior management and any minor changes will be taken care within the team without highlighting to the higher management so making of changes will be done to checkout checking process which is nothing but the configuration control process where we check out we will take out from the repository and once we update all the reviews are done we are going to check in back so that the new version is created and the changes are available in the repository so that is how we do the changes onto the corresponding document code test aspects test cases anything that are part of the test cases sorry part of the configuration items then we are going to close that change particular change saying that it is baseline or it is available for the release or delivery so what is the change closure then we had gone through incident management so we understood what is the any significant unplanned event that occurs during the testing or any other event that requires some investigation and resolution of that particular incident so the incident could be due to a fault intermittently it has happened unexpected so the results are incorrect test was not performed correctly so we can raise the issue incident again is the unexpected behavior or it could be self whatever that particular test or engineer trying to test it as well because testing may be incorrect way of doing also that is also can be called as an incident so that is a need for correction of the incident or the solution for that particular incident so in incident management we first identify or detect and record what is that incident about and details of all that incident then we do an analysis and classification priority like whether this five incidents have happened how we can prioritize what is impacting that whether it is a show stopper for the rest of the program to continue or it could be testing execution or the next one is the investigation and detailed analysis we will try to investigate what is the cause where the problem was before that incident cause incident had happened and we try to put a mechanism or recovery how we can recover that particular incident whether we can fix that code fix that design or requirements or the test approach itself is wrong all this will be part of the resolution and recovery then we will have a different stages of incident like new open weight rejected include into the build verified closed etc and every revision that we are going to release deliver as an artifact should identify a clear history of what is going on in the particular CI or configuration item so every configuration item is supposed to have something like this mandatory which will identify the particular revision number why it is what revision number it is been numbered and on which date it has been revised who is the author who is the reviewer or approval and what are the changes that particular revision number has undergone similarly upon the next change we are going to highlight it as revision number 2 3 4 like this each one will identify clearly so what is the revised details or updated details or the change details that are undertaken within the particular revision that is what we do that is about the revision history how we are going to mention for each of the CI then after understanding the configuration on incident management and all that we went through the need of CM tools so all the testware has to be having identified in a physical location or a particular repository so to control and manage that we need CM tool or a configuration management tool such as MKS VECS dimensions SVM etc so this is basically used for the CI it could be a development it could be a testing it could be a test case design any documents any accessing anything that is part of the project all this will be controlled and managed through the configuration management tool. A glimpse of how the configuration management looks we try to understood and it has various windows which will highlight different class in terms of what is the status when it was updated within that particular project and whether it is inline it is a baseline and if it is baseline how many elements of that particular baseline is ready for these etc all this will be part of the tool as a display you can see it you can configure the tool also but does not make usually the tools will allow user to configure the way that project has to be fashioned then we understood about the test management in terms of planning of the test project various aspects of test management and it is effort in terms of test management methodical support technical support domain expert test configuration management and the tester all these efforts for each of the activity under test management will be updated with the start and end date it is something like a planning document or a management document here is this tracking will be done how much effort has gone here and all and in one of our sessions we study about the variance like this is a plan this is a management plan and yes which how much we have consumed how much is left over what is our trend and what is the variance from a schedule what is the variance on off all this will be part of the test management we are going to take care also we studied about how the test processes are related to the software view model we understood about multiple view development life cycle you can see model prototype and each has its own view model and that is why it is called as multiple view development or life cycle model and that life cycle model has on the right hand side edge as a test aspects or test activity and that is on par with the left hand edge of the particular model or prototype of the development that is how it is aligned with the view model in detail the same thing is going to explain this problem is from that book called testing embedded software by Bart Brokman and Edwin Nautenboom what he says are provided and we have gone through this in our one of our earlier sessions I think it is in unit 3 so similarly the different view models various view models we have gone through on the right hand side you can see how the testing is aligned in terms of regression testing random testing statistical testing certification all that ultimate aim is to reach the right most edge of the upper side of the view model basically to release that particular activity which is identified for that particular view model and we are going to define a release criteria the next type of testing for as for the interest in embedded software testing testing design by contract here the approach uses the documentation only to capture the design but it will encourage basically sort of interaction among different developers so that particular developers are allocated to work like a contract and they will provide the support in terms of interacting with each other and providing the solution so that is how we do the test this specification is called as the contract and that contract will be developed after the interaction is done with the different module oneness and the system oneness then we had gone through a process called agile development process they basically use in short duration time to market embedded application or embedded systems which is totally away from normal and impossible development process so the current process are too heavy weight or too cumbersome in terms of documentation process and all that so all that will be bypassed in the agile development process and the current software development is too rigid it is difficult in terms of changing requirements, incompletion on short development cycles all this are very difficult to align with the current model so the process itself is deviated in terms of doing the agile where it is totally aligned with the business process or value to the business basically it brings up and different methods of agile are available like such as extreme programming scrum like this so most popular ones are scrum and XP extreme programming so basically the values are something like integrals and interactions process and tools working software over comprehensive documentation customer collaboration we take care more in the agile process then we contract negotiation and all that for changes and all that responding to change following a plan so as soon as the change comes we are going to start working on that implementing and all that instead of going to the plan process analysis and all that like a process mechanism extreme programming, adaptive software development dynamic system development all are different types of methods that we have can see more details say non profit organization they define and promote agile development basically companies like nxp or any company reference group they follow this to develop the prototypes and all that they adopt this agile scrum development process of course testing is also part of that scrum development so they will identify their own mechanism to align with the agile development so you can see that example what I have put they call it as a strength backlog so everyday based on the frequency that backlogs will be cleared and identified action is taken care for example if it is a 30 day project within 24 hours there is action for each of the development team or digital control they will take to act on that so that there is the deadline as I said agile testing scrum process also will be used there is no specific or dedicated tester in formal scrum process testing is carried out by developers with unit testing mechanism testing coverage is carried out by the owner or the client he takes care of how much power and all that by seeing the report against the specification so the testing is taken care by each spring spring is something something like spring backlog on any basis or how it is as per the definition and there is the acceptance criteria for each of that and that will be taken care next type of thing is a test drive or a test drive on development so basically we write the test for specification and against that we will try to develop the project so basically the project is oriented towards test or test driven surrounding the test cases and all that we are going to develop it so we write a test case that fails to fix that fail mechanism and repeat that the complete testing is taken care and the development is done test management and control what to do when things happen that test plan so if there is a hit on the test plan we are going to definitely work on the test and we need to control it so we need to reallocate the resources we need to change the schedule we need to change the environment we need to redefine the entry exit criteria and number of iterations changes we need to take care and we need a suit related changes are there that also we need to see the effect and release it also we need to postpone or update that is what we have studied in the earlier session today we will try to quickly understand the test management in terms of test management tool so we know that reallocation of resources to the entire test cycle artifacts need to be managed and controlled so that is what we do with the test management tool so the tool will basically take care of all the artifacts and it will highlight and it will provide the tracking mechanism in terms of where we are in the testing project so that report will be used by the test management and I will try to adopt as per the strategy that is going to work out for the release so basically the test management tools provided by tool vendors offer only the functionality to store the test gf of scripts and scenarios and sometimes integrate defect management something like bugzilla test management example I will try to tell you what we had used in one of the project test management identified test spaces scripts scenarios all this will be taken care of the test management tool such as test link practical sessions we will go through this how this tool works and we will try to create an exercise sample project identify the test management test cases and scenarios similarly we need to identify the defects using defect management tool all this will be basically used for managing the test something like bugzilla tool is used for test management and defect management so basically it offers to store all this artifacts scripts, scenarios, bugs and all that and they do not store plans so plans test management manager has to see and this tool will provide the numbers and all that artifacts and we need to align that against the project schedule so they have no facilities to store project schedule but this type of test management tool is not basically useful and controlling the activities but they need to be aligned manually test manager that is what they do but the good thing is they have the ability to link system requirements to test cases such as test link we can link so how many test cases have failed passed etc will be taken care so these tools afford the ability to keep track of the coverage of test cases is very important thing how many test cases have been done how many test cases are failing all that coverage reports specification requirements these tools will provide in addition they become very useful if the requirements are changed or if it changes the test manager is able to show the impact of these changes and how many test cases we have to execute or re-execute if one requirement changes so that test so those things can be highlighted with the tool basically and reported accordingly and the show stoppers in terms of any serious issues or bugs that have been uncovered all this will be highlighted with the test management tool the other part as I said is the defect management tool such as bugzilla can be used defects detected during the test process must be collated in an orderly way basically because all the defects need to be used and controlled that is why this tool is useful for a small project a simple file system with a few components is sufficient one excel sheet sort of a document is enough where we will highlight manually but as we progress throughout the entire project it may not be sufficient to maintain through simple file system mechanism we have to live with the complexity and that is advised to use defect management tool such as bugzilla usually in the industry they have their own tools defined bugzilla is a royalty free it is not a royalty free thing so commercially we want to use it you need to pay them you need to go for the royalty licenses and all that cooperates and all but in general what happens is embedded software industry they use their own INOS tools to maintain the defects and the test artifacts and against that the management will try to collect it and report to the customer and all so more complex projects need at least database with the possibility of generating progress reports showing for instance the status of all the defects or the ratio of solved to raise defects several tool vendors have developed the defect management systems all these are designed around a data system and have tracing and reporting definitely all the repository with regard to defects will be stored in the database systems and the database systems will be smartly used by the UIs and they are used to report and can be used to track it the tools have the ability to implement a workflow with the security and access level so admin can define who can check in the files who can put into the repository who can update it so that workflow or the work instructions can be defined so that the individual having different levels of access but developers can have access for development level they cannot have development access but they can read they cannot write and all this sort of a control mechanism should be defined by the admin for using the defect management pool now so as long as the defects are opened closed in progress hold and we can set up a notification in terms of email facility and some of them are available and trigger that in the web itself so that once the stages have been moved automatically communication will be happening to the respective stakeholders the defect management system is used to store defects trace them and generate progress and status reports the tracing and reporting facilities are more indispensable as control instruments so very important aspect of the so with that we come to the conclusion of test management tool and defect management tool in this session so with this we are going to end the embedded software testing over now so now we will try to okay so this next few minutes I am going to discuss and highlight about what we have studied overall in the embedded software testing right from unit 1 to unit 5 okay so the units are divided in such a way that all the aspects of embedded software testing is taken care and and in unit 1 we studied regarding which one we have embedded software testing right that is the first session we had and in the second session we understood about testing methods third one is on static analysis and code reviews the fourth one is basically integration this one is test management because each these 5 sessions are very 5 units are very important part of the embedded software testing so that is how it is been the course is been divided that way we will try to go through as a pointers what are the things that we have so we started with the fundamentals then different testing methods like white box, black box and all that next one is on the offline part of the static analysis code reviews type of reviews and all that then we understood about the hardware software, software integration methods and regression and all that the last one is about the test management in terms of test planning test life cycle alignment with the development life cycle and tools that are used for managing the effects as well as the defects so that is how the entire course is being parted and we have gone through them so in unit 1 it was about fundamental software testing it was an introductory session to embedded software testing and we had gone through embedded system basics embedded system environment embedded C glands because embedded C code is a specific code and there is a separate course of course by itself and we try to understand the basics of that and compilation, linking and all that basic parts of that embedded C language and the embedded systems we understood in that class why I took embedded C because most of the 90% of the projects that are there in the world the embedded system are on the are developed basically using the C language that is why it is very important on this embedded system basics with the C embedded C language and fundamentals about that then we had studied about V and V validation and verification cost of embedded software defects and graph plotting various stages of embedded systems V find defects it will be difficult to control as we progress towards the end of the project so it has to be fixed in the earlier stages so that is why we need to have a right sort of a testing mechanism so that is what we understood about complexity then test process basics one of the processes that are followed like test cases, procedures then the scripts, then the execution all these are basics and embedded software that system testing basic setup how are you going to have a setup so we had gone through the setup how the system is communicated with the target board as a white box how it is communicated with the target board as a black box and how the reports are being used how the scripts are driven all these basics we understood with a good example of a diagram and we also understood about the development the environment, test environment as well so continuation of the embedded software testing fundamentals test methods we had gone through like acceptance system integration and component level testing the various levels of testing then principles of test case design and procedures how it looks like with an example we had gone through and how we can do the testing and debugging on the target and on the system we had gone through test planning like what is the goal purpose what are the contents of the tests planning in the planning process we had gone through then we studied about example test plan documentation and test specifications also we tried to know example test procedure also we had gone through testing standards in terms of guidelines and rooms and all that how the testing document should be developed and testing should be carried out we understood also we had a few questions and answers exercises we have made in the first unit and we had drawn a block depicting various levels of testing and those levels are integration system testing testing definition of test harness test bed, test setup host and target based systems in development stage how it looks like during testing stage how it looks like then target based debugging and testing we have three various types of them simulation and target monitoring each of them we try to understand and advantages and disadvantages of them we use it for the embedded software testing and tools what are the tools entirely in the embedded software testing that are used how they are categorized and we try to put a list and snapshot of the embedded software testing tools and it will be listed as part of the planning, test planning and setup software environment configuration and tool we had gone through definitions of entry and exit for embedded software testing the various phases that it goes then PM method we understood with data principles software life cycle, entry, exit criteria prototyping life cycle and example we have gone through a formal life cycle and example we understood we model life cycle mechanism and embedded software development testing we have gone through and different life cycle processes with an example of entry with criteria we have studied life cycle process example we took a consumer electronics example where different stages of life process how it passes through each other and automotive embedded projects testing phases and process also we studied to understand then multiple life cycle based on that famous embedded software testing book we referred and understood the same way we also went through the multiple V model test activities nested multiple V model testing by an independent team master test planning principles of embedded software testing about I think 10 principles we have studied about embedded software testing which is highlighted in one of the embedded book that after that we came to conclusion about the embedded software testing embedded software system basics setup planning and all that next we started study on the unit 2 in detail of testing methods and testing methods have various chapters various sessions about dynamic testing static versus dynamic testing context dynamic testing types black box white box techniques testing techniques strategy in general and in specific depending on the complexity and the network of embedded software testing under test test case selection methods and white box testing coverage aspects advantages and disadvantages black box testing how to carry black box testing how to draw various test cases in selection and in that we studied the fundamentals and basics about black box testing in terms of influence partitioning boundary value analysis state transition state or event transition testing so these primary methods of black box testing we had gone through black box testing examples valid and immobile classes boundary value analysis its applicability examples and I think we took the temperature sensor example with high and low values so what are the values we can feed or we can use techniques in terms of black box testing using boundary value analysis equivalence classes coverage aspects also state transition testing I think we had gone through the clear example of something like that and the various events, actions activities, states and transitions how it is going to occur and how we can test it we had gone through we had an example with a UI embedded instrument instrument operation or modes in that example we went through we tried to understand and put the various testing strategy test our state transition testing techniques state event table we had gone through with an example transition tree legal, illegal and god test cases state events we see an example of state events and transition tree drawn and gone through the next type of testing method is the model based testing we understood the basics of model based testing model based testing taxonomy of model based testing we use the various instrument for that models such as matlab and all that and accordingly we are going to draw test strategy for that particular model and we are going to take care of the testing aspects of the model based design then dynamic testing black box and white box testing approach coverage testing in terms of structural coverage and statement coverage decision or branch coverage condition coverage other types of testing like data flow testing branch condition combination testing modified condition testing which is also called MCDC SCJ its principles then unit testing and software instrumentation what it means and how they are used with the various tools then test driver and test steps definitions we had MCDC from D1 student B perspective of importance we had gone through few slides then coverage testing tools vector cast we understood what they do and I think one of the practical session we try to see possible like one of the tool we can use it with an example project performance analyzers testing timing analysis profiles we can develop the test strategy and to be testing of the embedded software testing testing tools and life cycle test automation techniques test suits risk based testing we understood and we had defined about driver and formal runs definitions the next one is the third unit which is about static analysis and code reviews where we do static testing and analysis reviews and all that and we had drawn differences between static testing and dynamic testing both are complementing each other both are needed static analysis has control coupling static matrix analysis ak by complexity cyclomatic complexity is called off and few examples we had come through with a called tree example generated from the tool lines of code of the execution plan in plan out testing levels in terms of generating the matrix static analysis tools policy space quality QAC they are used these are rule checker PC line logic rule checker are used for seeing the report of software testing aspects in terms of code how they are used called tree analysis first case execution stack analysis stack workflow and code standards and rules which are applied in the embedded software development will be checked in the tools and the reported test matrix cycle matrix type software testing matrix in terms of various matrix that are being generated for embedded software testing and defect acceptance, defect projection, execution productivity test efficiency, defect severity index automation coverage effort variance, schedule variance scope change all these definitions we had method representation how they are being reported on a daily basis weekly basis train charts how they are shown in the home environment customers it is a burn down chart suppose the mix set of activities is going to take one month and how are you going to use that one month using the burn down chart we are going to create and report and as we progress we are going to update it and report it and matrix capture tool there are ways tools that are used for managing the test aspects defect aspects and all that defect management test bugzilla is being used the next unit is about software integration integration testing which is also an important part of the embedded software testing we had defined about integration testing what is integration definitions of system integration system hardware integration system software integration we use hardware and software together we are going to build and integrate and test it and types of embedded software integration testing big bang bottom up top down and considerations for integration testing and it is important importance we studied integration test strategy comparison of various strategies like top down that we had gone through we had a comparison about it then hybrid integration strategy which amongst both the top down as well as bottom up integration test approach then we have a centralized integration layer integration client server integration collaboration integration etc based on the type of nature of the project we are going to decide and do the software integration testing environment how it looks like system integration test integration from use case perspective specifically used in UML type of projects and we use that use cases to convert the test cases then we had gone through regression testing which is nothing but retesting and regression testing definition of regression testing test strategy test areas test automation in terms of stage execution and processing change analysis and relative importance use case example we had seen prioritizing the regression test cases and what is the basis for doing the priority and all that test case maintenance because very important because once the embedded software testing basic testing is done it is bound to update or modify over a period and how are you going to maintain it the project what are the steps that are involved for maintenance what are the test cases we are going to exercise for the future versions of different embedded software releases and build process we had gone through and the last unit is about test management which we have concluded in today's session it talks about configuration management test management configuration management elements CM process SCM software configuration management software configuration management activities such as planning software configuration index control status accounting software configuration orbit so different activities that are used for SCM example and tasks configuration control and SCB software configuration board controller board configuration items guidelines and life cycle also we have gone through and SCM phases like initiation planning execution also we had gone through in one of the session and the important thing is about version control or the revision control and how are you going to do a baseline that effects workspace management the repository how that is being used by development team testing team of the configuration control people that mean basically manages that and change management and instant management aspect also we had gone through last part was about CM tools SCSV and MKS and test management we had studied in terms of testing process which is relation with software V model designed by contract what are the testing agile development process agile development SCM process test development, test management and control test management tool defect management tool so these are part of the test management so that is about unit 5 so there are 5 units of embedded software testing which we had gone through with that we come to the conclusion of all the units of the embedded software testing so in the next 4 or 5 session we will talk about the practical aspects of embedded software testing we will divide accordingly the various elements that are used for embedded software testing in terms of practicality okay