 Okay, so now we're going to go into the conformance test suite version of the overview. So the face conformance test suite or CTS is often referred to. It is the official tool for verifying adherence to the requirements. In the conformance verification matrix that are to labeled test. And that they use the test suite as the verification method. So the reason I am very clear about and very precise about that in that bullet. Is that passing the CTS does not mean that you have you have achieved conformance. Because after all conformance is a test and evaluation process. So not only do you have to provide your UOC for test, you also have to provide artifacts to show that you have correctly adhered to the requirements of the face technical standard and not all of them can be resolved via test. So that is why in this bullet. I'm very careful to explain that is just to make sure that all the requirements that are labeled test and test suite in the CVM is to verify that you have adhered to those. So moving right along each version is aligned to a specific version of the face technical standard. There are different distributions depending on what version of the face technical standard you have aligned to. There is not one quote unquote god version of the CTS where you select which version of the face technical standard you are using. Just simply because the CTS would be way too big if that was the case. Like it would be probably two gigs. And it also has separate distributions for windows and for Linux. You can test your software on either one, however, just make sure that whichever OS you're using, make sure you download the correct distribution. It can be a headache if you don't. And then iterations of specific versions reflect the implementation of improved corrections. What I mean by that is when you don't have one CTS for face 3.1, you have multiple. Just simply because along the way we figure out better ways of doing things. We fix things that were broken or we find defects in the CTS. And so there are iterations that are put out dot revision, so to speak, that implement those approved corrections. And also when you go through the conformance program, you have to specify which version of the CTS you are using, which one aligned to the face technical standard and what revision you are using. So that's sort of the rhyme and reason for that. As far as the testing methodology, so the goals of the CTS are one to verify strict adherence to the face interfaces, verify that face interfaces were implemented correctly. And then for PCS and P triple S UOCs, verify that the USM adheres to the requirements and constraints of the face state architecture. And then the same goes for DSDMS, aka domain specific data models. It is an all or nothing verdict. You either pass or you fail. There is no partial conformance. There is no, yeah, you tried your best, but no, it's either met all the requirements or you didn't. And then for UOCs that are required, that are allowed to make inner UOC restricted method calls. This is for things like TSS UOCs and IOSS UOCs. If you used an inner UOC restricted call that would normally trigger a failure for a PCS or P triple S, the status that is returned will not say pass or fail. It'll say inspection required, meaning that the reviewer or the person at the VA that is actually performing the test has to go into the test report that is created and look at what triggered the inspection to verify that you're not doing something you're not supposed to. The overall methodology you can see in the bottom graphics is you create a project, you generate your goal center libraries, you compile your UOC, and then you test your UOC. Sounds easy, but nothing's ever easy. So, as far as requirements for a test, for one, you have to have your UOC, you have to have your USM if it's applicable. Granted, it's a PCS or P triple S. You have to have your header files. And then once you actually get to the CTS, you have to create a project file, a tool chain file, and then you have to implement the CTS factory functions, which are generated by the CTS if it's applicable. Typically, it's going to be applicable, however, usually it's OSS UOCs that are not subject to this. Now, I'm going to stop for a second and talk about the CTS factory functions. So, see, whenever you implement a face interface, whether it be one that's defined by the face technical standard as a major interface, or if it's the injectable, you have to provide a factory function for the CTS in order to instantiate it. Because for every interface that you declare in your project file that you have implemented, which includes injectables, the CTS is going to generate a test for each of those interfaces where it is going to instantiate your interface and then call all of the methods that are part of that interface to make sure that you implemented the pattern correctly. Now, granted, these tests are not actually going to run. It is a link test, however, if you have not implemented it correctly, or you got an argument wrong or something like that, it's going to trigger a failure or something like that. The CTS factory functions is an inspectable part of the CTS testing process by the verification authority, just to make sure that you're not doing something dishonest, like returning null or something like that. Now, as far as project configurations, these are created by the user. The project configuration in a nutshell describes a UOC. It tells what it is. It tells what programming language is written in. It identifies dependencies. It references our UOC object or source code if you didn't compile it against the gold standard libraries. And this is where you basically tell the CTS everything it needs to know about your UOC, as far as what interfaces that you have implemented, which ones you use, what OS profile you're using, whether you're using POSIX, ARIC, or both. And there can be only one project configuration per UOC, or one UOC to every project configuration, rather. You can't reuse a project configuration for multiple UOCs in 3.x like you could for 2.1. As the old Highlander quote goes, there can be only one. Now, as far as a tool chain, this is also created by the user. This tells the CTS how to compile an external client and link with the UOC under test. So everything that's compiler specific is going to go into a tool chain. Now, the tool chain can be used with multiple UOCs that use the same compiler and linker settings. So as an example, all the Balsa UOCs use the same compiler settings. So there is one tool chain for all the Balsa UOCs. However, each UOC has its own project configuration. So project configuration one to one, tool chain can be one to many. Now, in your tool chain, this is where you're going to identify a language, OS profile, what segments are under test. Now, segments under test basically is for anything OSS or not OSS. So because they have to be treated differently. You're going to include your compiler in linker settings, identify your compiler in linker, whether it be GCC or something else, your linker settings, what your target system defines as null and file extensions that are specific to the language, like header file dot HPP dot H, things like that. And that's your tool chain. Now, another thing about the tool chain that I like to warn everybody about is when you go to your tool chain list view in the CTS, it's very important to change your tool chain directory to the location that the tool chain you have created is in. Because if you don't see your tool chain here, if you try to import it, it's going to import a copy. So if you want to change anything about your tool chain, switch to the tool chain directory that it's located in before you start making modifications. And once you've switched your tool chain directory to the directory that your tool chain is in, it should automatically come up in the list. If you have to import, if you're not trying to import a copy, you're doing something wrong. So that's something I like to really warn people about so that they don't run into chaos later when they're running their project verification that says there's something wrong with your tool chain. No, I just corrected my tool chain. Nope, you corrected a copy of your tool chain. So void chaos when possible. Now, when preparing a ULC for a test, make sure you read the manual first. There's a lot of things in the manual that are not covered by this kind of training overview and vice versa. So make sure that you're not ending up flying the plane while reading the instruction manual. So read it first, become familiar with it before you start, you know, tinkering around. So when preparing a ULC for test, first and foremost, you're going to create your project configuration if it doesn't already exist and your tool chain. And you want to verify each just simply because the CTS requires that verification be run in order to create some temporary files. You're going to add your USM and your share data model if applicable to your CTS project. And then what you're going to do is you're going to generate your gold standard libraries. Your gold standard libraries are a set of headers and archive files based on what you have disclosed in your project configuration. So for every interface that you set that you set that you use or implement, it's going to generate headers for them. And also it's going to generate your CTS factory functions. And what you're expected to do to do with that because it's only going to generate the header where it's going to have the function declarations. It expects for you to implement those functions in a source file and then add that source file to the project. And so once you have generated your gold standard libraries, you create that CTS factory functions file if it's not already done so. And then you're going to compile your ULC object using the gold standard library headers and the OS headers that are provided by the CTS. Do not use any standard in glutes and do not link in any standard libraries when compiling your ULC for tests. Only compile the objects and list those objects and its non-face dependencies in the project tool and in the project file. Now before running a test, one, like I said before, create your source file for the CTS factory functions because you have to add that to your project before you can run the test because if you do not do that, the CTS has no way of instantiating your interfaces to make sure that you implemented them correctly. So if you try to run the CTS without a factory function source, it's going to bark at you. And then you have to add the following to your project configuration. You have to add your ULC object code and then once again the CTS factory function source file. And then it's always a good practice to run your project validator before you run your test just to make sure that you've supplied the project everything that the CTS thinks is needed. Now when you run your test, you have two options. Either you can run the full conformance test, which is going to include everything including, you know, your objects and your USM. Or if you're just wanting to run your USM, there's like a strange little icon next to the run conformance test that looks like sort of like an atomic snowflake. And you click that and it just runs the USM itself and not any of the objects, which is very useful for when you're creating your USM before you've actually started putting down some code to make sure that you implemented your USM correctly and are adhering to all the constraints of the face data architecture. And then once you actually get a result back, whether it be test fail or inspection required, it's going to print out for you the location of where you can find your test results. It's going to print out a PDF that has all of your configuration settings as well as the results of each test. So it's kind of long. I wouldn't bother trying to print it out unless you actually want a physical copy because it is, it can get kind of long. So anyways, that's that. All right, now let's do a quick demo. Okay, so I have my CTS up. Let me just go to home so you can see it with the what it looks like when you first bit it up. Now for this, I have downloaded CTS version 3.1.1 just simply because I'm running Balsa. And I'm not quite sure that it's going to pass this one because we never tested it with this one. This one came out after we cut our final Balsa snapshot and delivered it to the consortium. So I'm not for safety and to make sure that I, you know, don't look like an idiot on this WebEx, I'm just going to use the one that I know will work. So I have downloaded this one. I have already unzipped it and I've brought it up and I've given it two project files. And I'm going to use this one, the ADSV out project for Balsa. So here's the general tab where I give it a base directory where it's going to find all of my dependencies. Segment is Peter bless language C++. I'm going against the safety base API subset of the OSS. I'm using only POSIX, not using multiprocess, and I'm not using a rink, and I'm not using OpenGL, and I've listed where my tool chain is, and I've just given the UAP a name. For my data model, I've given it the shared data model, which is version 3.14. And I've given it my, my USM. And when you add your USM, it's going to query for UAP models within the USM. And I've selected the one that applies to my UOC inner test, which is ADSV out. And here's the associated views or templates rather. I've figured a GSL directory where I want it to generate my GSLs to. Here's all my header dependencies and include paths. There's quite a few. Now I've already given it the location and file name for what's going to be generated. However, if I run project configuration right now with this in-ear, it's going to tell me there's a problem, because this does not exist. I have not compiled it yet. Here's my factory functions header file, no, not header file source file. And then here is the list of all the interfaces I have implemented, and I've indicated which interfaces I'm using. It automatically assumes you're using a TSS if you're implementing a P triple S or PCS. So it does not give you the option of anything, but the TSS component state persistence. But I'm using configuration services and I'm using a generic IO interface. And I'm using the health monitoring interface. And these is just project info and notes if you wanted to add any notes in here for anything. So what I'm going to do now is I'm going to generate the GSL. So I'm going to go over to where I'm going to generate my GSLs to. And as you can see, oops, it is empty. There is nothing in it. I cleared it out earlier. However, I do want to show what the CTS factory functions look like. But I'm going to generate the GSLs first, show the shell, and then I'm going to actually show the source file. So let's go ahead and generate the goal centered libraries. Now, anytime you're generating them with a USM, it's going to parse your USM first to make sure that it's correct before trying to generate anything off of it. Otherwise, chaos and insanity will ensue. So it's going to take it a couple of seconds to actually parse and verify your USM. All right. It successfully generated them. No problems so far. So now let me do another LS and it's created a bunch of archived libraries. And let's go to the include file. So there's my CTS factory functions header. And there's also my face includes my configuration header, configuration injectable, all the data types generated off the data model, my TSS include files. And this includes the type TS as well. However, let's go back and let's look at the CTS factory functions. So based off of everything that I told the project configuration, this is what it is generated. Obviously, I've implemented the initializable interface, the configurable, the stateful. It assumes I'm going to provide the base injectable as well as the injectable for the type TS, the injectable for the IO generic IO service and the injectable for configuration. So it expects for me to provide implementations of all these where I return pointers to each of these interfaces. Now, in order to implement these, I have to create my own source file, which I've done in the past, and I just reuse the same one. Now, because of my inheritance structure, all I have to return is an instance of the ADS-B out to have basically, you know, provide these interfaces. So each of these calls is going to return a new instance of my UOC. Because in the in the inheritance structure, all of these are inherited and provided. So that's the CTS factory functions. Now that I have my GSLs, I'm going to compile my UOC to those GSLs. Now, let me really quickly show you how I am compiling, because once again, don't include standard, standard headers and libraries. Where is my config? There it is. Okay, so I'm establishing my GSL base as the location where my face conformance test suite is, my generated GSLs to the location where I've generated the Gold Center libraries. And then for my C++ libraries, I am using GSL base, which is the location of the CTS on my system. And I'm pointing to the C++ base directory in the Gold Center libraries that's provided as part of the CTS. And in there, there's going to be a source files for OSS and POSIX. I'm going to use those as far as the standard C++ libraries. So once again, compile for tests using what's provided by the CTS. Do not compile based on what's provided as far as your OS. And then also, your tool chain is going to generate a series of files that you have to point to as well. Let me kind of show you what that looks like. Because as you can recall, in your tool chain, you define a few things, one of them being definition of null. So this is my file that's created. Created five minutes ago when I generated my GSLs, it's going to generate the tool chain files. And one of the things that the face interfaces is going to look for is these two guys, the exact types that I have defined as being the types that are part of my targeted system and my definition of null. So you have to point to these files as well. So now I'm going to compile. I'm going to invoke my make file structure, but I'm going to point it to my make file.conformance, which is going to compile using the settings in the config file I just showed. It is compiled. Let me make sure that my .release file is there. Yep, that is my object file. Now, for Balsa, we instead of having to add a bunch of object files to the CTS, we just combine them into a releasable file that has all the object files associated with the ADSB out and all the library dependencies, because we do provide a library as part of the ADSB. And that's the ADSB library. And since it doesn't provide a face interface, we have to include it as part of our test test objects. So now everything is ready. I should be ready to just click this start button here and run the test, or I could do the atomic snowflake and do the test data model only, but I'm going to run conformance on the UOC itself. So let's run it. Yay, I passed. All right, so now let's go look at what was generated in this location. See, I put it in the CTS projects and tool chains folder. 3.1 files. And there it is. Whoops, not too much. Says I passed. There's the contents. It's got configuration, tool chain configuration, performance test, data model test, and then all the individual tests it ran. I'm sorry. Here's all the data from the project configuration that I gave it. A little less readable than in the GUI, but it does its best. Tool chain configuration. Here's where I put the exact types, which is generated, my definition of null, file extension names, compiler. And then here we get into the USM tests. These are all the tests it completes. Meta-model validation, OCL constraints check, view specification, share data model conformance, and then we get into our PEE-TRIPLE-S segment conformance test. Test my initializable, tested POSIX entry UOC calls, test the injectable for all my injectables. Each one here is listed, TSS base. So it tested all my injectables. Make sure I did them correctly. It did its GSL link test to make sure that I'm not using something naughty. And of course it passed because I'm only using the calls allowed in the safety base OS API profile. It did an inner UOC communications check. Now granted, if I was using, well, two things would happen here, other than getting a pass. If I was trying to do something sneaky, like put in a socket call to something, it's going to flag fail because I'm not allowed to do that. However, it's not uncommon for a socket call to be part of a TSS UOC. So what would happen here is you'd get that inspection required, meaning it used an inter-C restricted call. You got to look at it just to make sure. Hi, Chris. I just have a question on that. Yeah, you went through like a lot of test paths and lots of checkpoints are passed, passed and say that one of them fail. Do they still skip that first fail portion and move on with the next test or how it stops there? It does not stop there. It tests everything. However, one failure means you failed overall. But it goes ahead and runs all tests. Okay. I mean, one fail is fail. As a report, you get a whole fail and you have to go through which one fail. But internally, you get which one pass which one fail, right? Yes, you do. Okay. That's great. Thanks. Let's see. And that is about it. This last one was for the configurable. And we got two blank pages. Yay. All right. So that concludes our CTS overview. And we can open it up now for questions whenever we stop recording. And can you stop the recording?