 Hello and thank you for joining us for C++ for real-time and safety critical systems. I'm Robin Rowe and this is Gabrielle Panterra and we're going to talk to you today about how to make C++ code faster and more reliable. Let me switch over to my screen here. So here's a real-time safety critical system. This is a Tesla autopilot. So this is the kind of thing that we usually think of when we're talking about safety critical real-time systems. But here's a system that is also an autopilot, but this time we have an autonomous robot that is driving a conventional truck. So we never quite know where a software is going to end up. So it's a good idea to use good software design principles in whatever we build. Bugs in uptime. Fewer lines of code equal fewer bugs. Clusters, technical debt, timing errors. At Google, 70% of failures happen when releasing a new version of code. To reach 9-9, no. To reach 9-9. 9-9. Sorry. It's confusing to see it written that way. To reach 9-9, you must bank reliability. 9-9 is one-tenth the time of the blink of an eye. So that's one-tenth of a blink of an eye in a year of downtime. So that's pretty much no downtime at all. And you can see this chart on the right where we talk about the various number of 9s. 9-9 is a lot. Safety critical standards. So there's ISO 9001, which many of us are familiar with. That is how we get a repeatable QA process and make sure that we actually have a quality assurance process. 23360 is the standard for Linux. 13485 is medical software. 178 for aviation software. 26262 for automotive software. ITS-ATC for automotive traffic light software, which I'll talk about some later. And there's also MISRC, which gives some standard best practices for safety critical C systems. But the one that I'm really excited about is ISO 56007 because I'm actually the chairman for that. That is a future standard in idea management that is looking to extend 9001 into the innovation realm. Process types. Agile, waterfall, unstructured. What process do we have? So unstructured is the no-process process. We have no goals. Top-down directives just come out of thin air. Deadlines and milestones make no sense. We don't learn anything from past mistakes. We try to catch up. There's a lot of personal baggage and team stress and people freaking out. There's expectations of project failure voiced in our meetings. The managers are consumed. The team doesn't actually know what the managers are doing because they're off trying to fix something else. The budget is out of control with both binge spending and illogical cost cutting. Waterfall process indicators. Top-down requirement analysis, preliminary design review. Lots of reviews. Lots of reviews. Yes. Critical design reviews, too. Charge numbers, which is bug tracking, Microsoft project, daily team meetings to discuss what happened yesterday. Managers spend much of their time absent for planning meetings. Good plans that demand sticking to the plan no matter what, and Big Bang finished deliverable deadlines tend to slip. So charge numbers aren't exactly bug tracking. It's more a matter of that you get charged for fixing bugs. So here is Robert Overcracker. He had a great waterfall plan. He's literally jumping over the waterfall at Niagara Falls. Great plan, except his parachute wasn't tested to operate when wet. So this turned out very tragically because he didn't have a complete waterfall plan. And this is why people like Agile that plans can change. Agile. Manifest for Agile software development. As you can see on the right, there's all sorts of stuff going on. A lot of interaction, working software over comprehensive documents, customer collaboration instead of just what's in the contract, and responding to change over following a plan. So that means you're able to move and go. Agile process indicators, user stories, sprints and retrospectives, released early and often. That's key. Yep. Pair programming, Kanban boards, meetings are forward-looking or retrospectives, cloud-based project management, JIRA, and Git. And many other tools as well, but those are the two that are most popular. So why do we like C++? Well, I like it because I've taught it to universities and used it for 20 years. But that's not the reason everyone else likes it. The reason that most people like it is that typically it's going to be 10 times faster than other tools. Also there are 20 million C++ programmers out there, so it's possible to staff a team. And we really like that it's subject-oriented, has constructors, destructors, encapsulation, all that good stuff. Then for reliability, it also has an extensive set of tools, which is very important in doing safety-critical and real-time systems. And also, C++ is the software if you're in wish and picture visual effects, as I am, VR as I am, or autonomous vehicles. C++ is everywhere, it's in... Everything. Oh. Windows. Windows, Mac OS, Linux, iOS, Android, embedded systems, IoT, cloud, aerospace, AI, and of course, in databases. Smooth task, Gabriel. Sorry, it wasn't paying attention, I was a little off in the lala then. Object-oriented design. The most important single aspect of software development is to be clear about what you're trying to build from Bjarne's true strip. Software that snaps together like Legos, nouns or classes, verbs or functions. Encapsulation hides data from code that shouldn't change. C++ is, pardon me, is as easy as Pi. Inheritance, encapsulation. Elegant design simplicity is what's left after removing complexity. Yes. And the reason that C++ is so complex is to remove complexity, not to add it, which some people do. So, embedded system design. We want to think small and fast, we don't have a lot of hardware here. We want to avoid the heap because we may not have much heap and we don't know how long the heap will take because any time that we go out for memory on the heap, we're doing a global lock, we're pretty much stopping the world while we wait for that memory to be allocated. We want to avoid termination because there may be no place to terminate, too. There's maybe no operating system. And so, therefore, we don't like exceptions. If rebooting is feasible, we'll use a Highlander. That is an automatic restart to reload our program. Now, we want to avoid implicit initialization of static objects before main. The reason for that being that we don't know in which order they will initialize because that's undefined by the programming language. So we want to do our bring up explicitly in main where we initialize all of our objects that we're going to have in memory. And I'll bring all of our files online, bring our devices up, all the stuff that we need to bring up. We want to avoid C++ optimizations, so we're going to profile and test, not just as soon as we know where the code is slow. And we're going to avoid risky coding practices like using a lot of CAS and other things like that. And we're going to use C++ type safety and encapsulation and be constant correct. That is, use, mark memory as being read only wherever we can. Safety critical C++ concepts, encapsulation, memory management, thread management, hard and soft real time, static analysis. Single code base on Linux, Windows, Mac, embedded, pardon me, embedded, audit, simulation, playback, SQA, unit and regression. Yeah, so that's software quality assurance and, you know, audit, simulation, playback, SQA, unit, regression. We're just doing everything we possibly can to test this software. We're not assuming because it ran once that we're done. And in safety critical systems, we may actually spend more time and test than we did in the software development. C++ type safety. Locked out in compatible code. Typical type errors will be found at compile time. A major way of static checking, we can still cast when we must size a pointer, unknowable. You get the last one. And that's used in pointer type. Don't cast to an integer, assuming that an integer and a pointer have the same size. The reason that our size of pointer is unknowable is because of platform differences. We may know what the size of a pointer is on our development system, but it might be a different size on our actual target cross-compiled system. So we just assume that we don't know the size of pointer and make sure that we use types that are compatible. C++ encapsulation. Watertight compartments, classes and objects, private, protected and public. Encapsulation is a form of data hiding. Encapsulation can ensure consistency of state. Don't use inheritance where you meet encapsulation. Const is also a form of encapsulation. Yeah, so we've got a picture of the Titanic here. And this is an example of where compartments certainly help, but they're not foolproof. So yeah, there's a lot of things that we can do, but compartments will keep you afloat a lot longer than not having compartments. C++ memory management. So that can be the stack, it can be static or the heap. The stack memory is automatic. That's what just appears magically when you create a variable inside a function. Stack memory is allocated when the program loads and exists throughout the time that the program is there. And the heap is variable size memory that can vary in time and space, but the heap is prone to problems, including that we don't know how long allocating a piece of memory on the heap may take, and that can be a real problem in a real-time system. So if we are using the heap, we don't want to call a naked new call. We want to use unique pointer from C++, from more recent C++, because it leaves no garbage to collect. But that pointer is encapsulating new so that when the unique pointer goes away, it automatically self-distracts. And then there's shared pointer. This is really more for legacy code. This is a usage count pointer so that you can go back and have a pointer destroy itself at the appropriate time magically, but it's not as efficient as unique pointers. So we use unique pointer in new code. C++ concurrence. Pardon me. C++ concurrency. Concurrency. Yeah. Concurrency. We're talking at the same time. This is concurrency. Yes, I know. Message queuing, threads, joiner, detached, mute hex, locks, condition variables, lock free, double buffering, and volatile. So in C code, we would use volatile to use some sort of integer that works across different threads. But in C++, we have atomic. And the reason for this is that with multiple cores and that sort of thing, we can't be quite sure that volatile does what we think it does across all those cores. And they're really getting the one and only volatile. Whereas atomic will make sure that that variable is actually synced across all the cores without locking it. So with threads, join and detach. With join, that means we're going to wait until the thread finishes before we do something. In general, that's what we don't want because that's not concurrent. So we're going to launch a thread and detach it and keep going. And that's when things get complicated and we need all these fancy lock systems. Now it is possible in many circumstances, well, maybe not many, in a few circumstances, to have lock free code where it has no locks at all. But just by the cleverness of how we've coded it, it can't collide. That's not very common that people do that, but it's lovely if we can. So be on the lookout for that. And then there's double buffering. With this system, we might have an off-screen buffer that we paint all of our screen changes to so that we don't see the screen update. We just flip the buffers so that a fully-painted screen appears and we don't see components on the screen being drawn one by one, which would be visually annoying. Pointers. Oh, OK. Avoid garbage by using unique pointers. Yeah, we mentioned that. Keep going. Avoid wild pointers. Yeah, so point to null pointer in the constructor. Zero is still a valid thing to point to, but we have null pointer now, which is really just zero, but it's a little clearer what the code is doing, that we mean for this to be a pointer that we never call or that we'll at least check that it's not zero before we call it. Avoid dangling pointers. Yeah, so we're going to point back to null pointer after we release the thing. We only need to do this if we're literally calling delete, which we very decided we're not going to do because we're calling unique pointer. So this should not be necessary, but sometimes it is because we've got legacy code. So delete those dangling pointers because in many cases, that will be where you discover memory errors. Hide pointers. Yeah. By making them private. Make them private. So don't show your pointers to everybody, just only your closest friends. Or a pointer could never be null. Use references instead. And then we want to avoid unintended modifications by using const. That's the same as read only for people that prefer English, but it's not called that. It's called const. It's in a debugger. All code you write that does pointer math. Yeah, because every time I do pointer math, I find some quarter case or something that I've screwed up. And so I'm really glad that I've traced it through the debugger and said, oh, yeah, what about that? So trace all that pointer math and the debugger and any other code that we're highly suspicious of. You know, in theory, we have unit tests and we'll also catch this stuff, but that's not the same as a human brain just looking at the thing and saying, does this really do what I intended? C++ casting. We don't want any, but sometimes. So this is really more of a thing for legacy code. We try not to cast a new code. Here's a C-style cast. It's just, you know, int and parentheses. But constructor style casts are also legal and sometimes easier to write where you put the type on the outside. Then there's a const cast, a static cast, reinterpret cast. The one here that's really interesting is dynamic cast because where we need to do upcasts, which we really shouldn't do, but sometimes we've got to. This will tell us if we're upcasting to something that actually exists or we're out of our minds and casting off into memory that doesn't actually go where we think it belongs. So if we've got a base class and we need to upcast it to the derived class and we're not sure that that pointer is actually the derived class, this will keep us from catastrophe. And then when it comes to doing integer casts, we may want to declare but not define some function templates because that will make the compiler complain that it's not going to automatically implicitly not that we want that do those casts for us when we really shouldn't. C++ exceptions, don't. Signal, in a type safe way, return false. Used to add simplistic error handling to legacy code. C++ exceptions are termination based. If a second throw happens before the first throw is caught, not good. If we don't like new because it can be slow and terminate unexpectedly, we don't like throw for the same reason. So in real time and safety critical embedded systems, we just stay away from exceptions. Exceptions are much better for batch process types of things where if things go bad, we want to terminate the program. But in this realm, we pretty much never want to terminate a server or an embedded system. So we don't want to take the chance that we do a throw somewhere and before we can catch it, something else goes wrong and throws again and terminates our program to nowhere. Testing methods, tracing unit stress, regression monkey, screen scraping, keyboard mouse macros and catch library. I should also mention C test here, which comes with C make is also very nice for doing the unit test. So C++ traps. Infinite loop. Recursion. Casts. Wild pointers. Segfault. Division by zero. Floating point exception. We're fatal cashmats, but we won't get into that. Initialization before main. We already said don't do that. Complexity and obfuscation. Obfuscation. This is also a speech thing to learn new words. Cohesion. Spaghetti. Okay. Say it three times fast. No. Okay, so I want to do this one. She wants to do this. Agile safety critical mindset. How could you not select a guy who wears a woman's hair band for sunglasses, but seriously, Jordy saves the enterprise from certain doom and countless episodes. Sherry so did Scotty, but Jordy did not whine about it like that. I'm not going to do that. I'm not going to do that. Captain. I'm giving her all she's got. She can't take much more. Now. Jody's got her. All done. Whoops. Jody's got. As ridiculous as that visor journey sport of luck. It enabled him to see things that other crew members couldn't. The bar burden. Yeah. So. Yeah. If you, if you come from a realm of termination is okay. You can come here on out so that you can see things in a new way. And actually if you do watch it, if you notice in our photo, they did have a lot of scrum sessions. Yeah. Yeah. They're scrubbing here. In the 23rd century, they're, they're charmingly retro. They're still, they're still doing a scrum. Yeah. So here's some real life, C plus plus examples. Literally everything depends on C plus plus. Real time systems, safety critical systems, embedded systems, national critical infrastructure all of that stuff. Here's some systems he's touched. So at GoPro I worked on defining their IOT API so that we can connect all the cameras in the world together. Robbie V. Lager this is a Mattel animation series that I worked on the I designed the animation motion capture software that animates us in real time. So this is done with actors on a sound stage and comes out as a cartoon on the monitors. Right they're in the suit and they can see it right there with the amazdic. So we're in the white suit which is so flattering and it comes out as a cartoon. Whoops I think I went past one there. So here you can see some some dudes in the suit. They're in this fake Jeep in the in the studio and this is being used to do a game trailer. So these actors are actually inserted physically into the game. They're actually playing Call of Duty in the physical world here. Then here's the genie set top box. I got to work on the security and satellite channel expansion for that. Here's a casino gaming system. This is a very high-speed real-time server with online gaming everything is about how many simultaneous players you can support. So server efficiency is very important and then this is national critical infrastructure and a real-time system. This is the traffic control system for the country so that when you cross the street the light going the other way is not wrong and you get run over. So the way that this system functions this is a Linux system. If you see those silver boxes next to the traffic light that's where the system lives and there is a single board Linux computer in there that is running the traffic light and there's a second processor in there that's independent of that that is pinging that computer every one tenth of a second and if the computer if the Linux computer responds saying that I haven't processed all the commands on my command stack in this time slice then the traffic light will go into flash and it requires a truck roll that is it requires technicians to get into a vehicle and drive out to the light and reset it because we don't want that light to reset automatically if there's something wrong with it. So every time of a second it's asking the Linux process is everything okay with you and if it says no or if it doesn't respond at all that other processor the watchdog timer will take over and turn the light to be read in all directions. This is the USS Lincoln I got to to sailing this for a system I built for national security and that's it I'm Robin Rowe and I'm Gabrielle Pantero and we are ready to take your amazing questions here we go.