 Hi everyone, we have come from APT Software Corporation of Vietnam and University of Engineering and Technology Vietnam National University nice to meet all of you in the open source submit Japan and Automotive Linux submit 2020 event as you know because of the COVID-19 pandemic Therefore, we are conduct this event as a virtual meeting We are very excited about this kind of meeting because we feel saving and safety Today we would like to present to you about our technical approaching for an CNC plus plus programming language automated unit test case generator and its application in embedded automotive project in general and HGL in particular The table of content for this presentation is include six parts At the first part we would like to describe about our motivation of unit testing in embedded and automotive software development At the second part we introduce the technical methodology of automated unit test generation and some our research about this technique and From our research result, we have developed by ourself an effective platform for CNC plus plus programming language automated unit test generation named ACA-AUT The feature of this platform will be introduced in the third part Recently we have applied the ACA-AUT in some HGL software components and seen many potential results The fourth part we will share with you about our effort for this task and this result The fifth part is some our conclusion and the feature plan of FED for ACA-AUT tune What is unit testing? We almost know that unit testing is a type of software testing where individual units or components like functions are Macro are tested and The unit testing process is about validating each that you need or component of the software Such code performs as desired or not Unit testing have many Benefits, some most important benefits of UT are it possible to identify the problems earlier it helps reduce bug ratio and development effort and In most industrial sector unit test is recognized as a up-to-date development document for example in automotive domain Totally the earlier we process the UT activity the lower cost we have achieved Although there are many benefits of UT But recently it has some challenges Actually to design a good unit test case the tester or developer has to understand the entire of the source code clearly But to understanding the entire source code is make us Very tedious and terrible Almost of the proper initial condition to design a test case is like in actual Then almost the manual test case will not be exercised the code in Realistic context which reduce the value and the the accuracy of unit test result The configuration management process of a UT project is very painful also To find the dependencies of the source tree as known as the system under test SCD On the other hand manual test creation is low productivity and need to improve an accessory The last one is in some minor aspect Almost the industrial sector has their own and during development process need to integrate with UT solution But currently we have only high cost stand-alone commercial UT solution like Vector Gas or Cantata for instance Therefore developing an automated test case Generation is a good approaching to resume these challenges for industrial sector at current time For more detail as we have known There is no scope for errors and bugs when we are developing automotive embedded Software of critical vehicle ECU in general and in Instruction cluster in particular And so the software testing is our in need because the automotive Intrusioning cluster has to respond several functionalities Each embedded software is quite complex for example meter advanced driver assistant system IVI interact functional safety So the software testing activity take many effort of Development team and it seems that currently almost a GL middleware layer UT is developed manually and the UT activity is in floating status AGL has to development mainline at this time IVI and cluster The most important task of AGL contributors in cluster development is QM quality management activities To be able to release a mass production Totally we need to improve the productivity of software testing process to ensure the quality of software Deliverables to catch up an effective embedded automotive software development Engineering process with higher productivity And now is the theoretical basic about C and C++ programming language Automated unit test in AKA UT I will mean approach is developing a platform that can generate test data automatically in white box testing The white box testing it allows to detect potential bugs in the source code some mandatory input arguments to conflict the UT project is C and C++ projects for tree and some coverage criteria For example statement branch MCDC With this input argument AKA UT will create a set of test design Test case test report Name test data as the output The most inspirational papers using to develop AKA UT to list as the following Especially we have our main publication for the automated test case generation for C and C++ project as in following information These six steps need to be able to automate the test case generation The first is remove some pre-processor Directive line if depth and platform dependencies to scope Each pre-processor directive based on the configuration can recognize as a different SUT The second step is to construct the source tree This test will model the constraint of each UUT into a tree data structure We choose trees data structure because it has parent and child relationship and can be passing easily By experiment, we choose node in tree is a folder a source code a namespace or class and the tree leaf is mental attribute or variable The first step is instrument on functions We can catch the instruction of UUT and we insert more information into the instruction and mark it as instrument After this step we have an SUT actually The first step is generating function control folder graph Cfg Each function has its own describe graph When we pass this graph we will have is Cfg The function graph will be different with different coverage The fifth step is generating test that are automatically This is the main part of our turn and the detail will be described in next slide After have the test data Aka and UUT can execute the test case based on this test data and The interface with some test framework like g-test or check-c to create the test driver Components and generates the coverage report It is the final step when using current version of Ika and UUT tool The fifth step generate test data has several seven sub steps The first sub step is generating random initial test data Based on the specification of unit or component After this step we have the initial test data The second sub step is executing this initial test data by create the test driver interact with the test framework The third sub step is computing the current coverage based on the current Cfg We have the status of coverage after this step The fourth step is create the path constraint that relative with the current test data Next step we will negate the path constraint based on the current test status with the objective to generate new test data that responds to increase the coverage Each fifth sub step In this sub step we use digital Algorithm to evaluate the negate point by finding the sorted part 3 in the Cfg In the sixth sub step, we will solve the new path constraint to finding the new test data And at last we will execute the new test data This sequence will be executed until we meet the input requirement for the coverage The visualized information of all this of all steps will be described in next slide For visual all of the steps we have this diagram like an example We choose the source code of three type function as the unit under test From the source code, we create the structure tree from the structure tree. We will have the instrument instruction In the fourth step, we have the control follow-up Three type function Almost of the source code path we use CDT, CNC plus development tooling In the fifth step We have the DD1 as the first test data and the second test data is generated based on the test data the first test data and The third test data is generated by using the first and the second test data In actual, we can use the random method to pick up test data in parallel In practical, in some case, choosing random method have more productivity Because we can have the target coverage result by random test case In the sixth step is execute the test data based on the test framework like ZTest or check C these are ACA-AUT main features The ACA-AUT-Tune supports CNC plus projects with the following main features The programming language version is C99 and C++11 and C++14 version We support multiple compiler support GCC and G++ and we can integrate it with any compiler Currently, ACA-AUT platform can execute in Linux and Windows environment ACA-AUT supports Various standard data types, like primitive types, integral, boolean, float, etc The right type like class, structure, enum, union, and string C++-STL standard template library is supported by ACA-AUT-Tune or so ACA-AUT also supports primitive execution block like inline, overload, override abstract class, private, public, protected function And some function context like normal class, namespace, and abstract class ACA-AUT supports multi-dimension array and multi-level pointer This is very easy to set up dimensional size by using GUI of ACA-AUT Based on the configuration parameters, ACA-AUT will generate the suitable test data automatically for multi-dimensional array or multi-level pointer supports Especially, we support VoIPoiter The VoIPoiter is used in embedded automotive source code as a best practice pattern for flexible design And therefore, it has to generate the test case for VoIPoiter parameter But in ACA-AUT, we can configure a VoIPoiter as a primitive structure or use code type to respond with the real context The same with the VoIPoiter, the function pointer is also used widely in embedded automotive source code To respond with the function pointer, we implement a smart method in ACA-AUT to pass the project source tree to suggest the function list that can be matched as a prototype of system under test function pointer In C++ and C99, the macro is widely because of its flexible uses ACA-AUT also supports the macro time, as in the screenshot, we see that ACA-AUT can recognize a macro and suggest users to conflict for this macro After configuration, ACA-AUT can generate test data for the macro and after executing the test data, ACA-AUT will report the result to us about the test data Of the powerful feature of C++ is the template And the most impressive feature of ACA-AUT is support template type Template is generic type and based on the theoretical viewpoint, it has to handle automated test case generation But ACA-AUT can pass the template and suggest to the user to choose the same type of data for some template and generate the test case as primitive types We also can define the custom source code for the template Some other features of ACA-AUT are described for the environment configuration For more detail, we screenshot some windows of ACA-AUT We can create a multiple testing environment for a C&C++ project We can stop a function, it means that we can choose any return value for the function of SUT using to testing other functions We can define some custom user code In case we cannot generate the test data automatically, we can use custom user code as a powerful feature Besides the generate test case automatically, ACA-AUT permits to create their test case manually to respond for some specified use case We deliver the source code days by days Therefore, ACA-AUT provides a smart mechanism to process the incremental testing activity ACA-AUT will report to you the duplicate, the change of the source code need to regenerate the test case or not in the next test activity You can debug the test case using current ACA-AUT environment And last but not least, ACA-AUT will generate the test report for management and development activity ACA-AUT environment setup, we have eight main steps The first is choose compiler, the second is naming the environment The third is choosing the testing method The fourth is some build configuration and the fifth is locate the source file And the sixth is choose UUT and start function as in it And the seventh is we create the user code, custom user code for some demand This is an optional step And the last we will see the summarize of all configuration for environment After the SUT is changed on demand, we should use incremental testing ACA-AUT will pass the source tree and know that the effect of modification in SUT at the current time for both color and quality function ACA-AUT will report to you the effect of test case the unusable test case and the return test cases One of the powerful feature provided by ACA-AUT is missing both manual and automated generated test case for each UUT By using this kind of feature, you can increase your productivity in unit test case generation In case we have some redundant in test data, ACA-AUT can evaluate the effect of each test case and suggest you that the test case can be removed but keep the expected coverage For example, we can remove the test case if there is other test case in the same path ACA-AUT provides an utility to determine the source code for the function of the macro The utility will have you achieve the expected coverage in some complex business logic source code We have some custom user code There are two types of custom user code The environment user code will develop the source code that needs to create the testing environment For example, in case we have missing some library or function in the SUT The parameter source code using to define the value for the input argument And the asset source code using to compare the expected value and actual value by the runtime execution function And we can debugging the test driver as similar in other IDE with some feature like brokepoint breakpoint Now we can see a screen record of using ACA-AUT in action by some example We can create a new testing environment The great testing environment has seven steps And the first step is choose compiler Currently ACA-AUT supports some compiler like GCC, GCC++ Each compiler has their own instruction But we can reconfiguration In this project, we will update the compile option and the link option Because the IGL application framework might be using AFP version macro Therefore, we will define this variable value ACA-AUT supports some code rich type like statement branch and MCDC User can choose enable the wipeout Testing for private and protected class The user can choose the directory of the search tree Each search tree can mark as a search directory Type handle directory is additional source code For create the environment And the library included directory is the directory store the extend library In the seventh step, user can add some user code need to build the configuration environment And after success to create a new environment We can see the structure of the project in the project structure panel User can choose manual generate the test case for what function And input some suitable argument And execute the test case and see the report The report includes the coverage And beside the manual test case generate User can automate generate test case Not only generate test case automatically for one function ACA-AUT can generate the test case automatically for multiple files or whole of the project On the another feature of ACA-AUT is compile test User can generate the test case the combine test case and execute in one execution To achieve the suitable result with the interact between each unit and the test Now is some result of ACA-AUT tune in action apply for ACL As in the demonstration we choose a GL application framework by the repository as a system under test This code has about 24,000 lines of code The expected coverage for this demonstration project is 80% We have allocated 5 testers with junior skin for CNC++ ACL but has familiar with Linux environment and ACA-AUT tune The hard way we use to build testing environment is Intel i7 CPU and Ubuntu 16.4 version In the result we can do the UT activity for the ACL application framework by the in 6.5 months with junior testers The allocated effort for study the ACL application by the source code and build the ACA-AUT environment has a high proportion about 38% because the tester is junior with the ACL platform and it takes about 2 or 3 days for the Intel PC to generate the test case for this repository In total the average productivity using ACA-AUT tune for ACL application framework by the is about 4,000 lines of code per man It is more effective than manual unit test activity about 80% And now is our conclusion We see that automated CNC++ programming language unit test generation is very effective for embedded automotive software development project It increase the productivity and reduce the manual mistake We should consider about the standard engineering process to using automated unit test case generation in industrial sector but still keep the expected quality In the near future we eager and have a short-term plan to enrich ACA-AUT tune by adding some feature like integrate ACA-AUT platform to some standard IDE like Eclipse etc And we win steam pay effort to support C++ 17 version And we really want to cooperate with OEM or T1 and make ACA-AUT in action to acquire certificate to using ACA-AUT in embedded automotive project with locals than other current platform And if you have any question or any discussion about our ACA-AUT tune please contact us with this information Thank you