 Hello. So, we're ready to start. It's my pleasure to introduce to you Joss Wetzels and Ali Abhazi with taking a scalpel to Q and X. You'll get a deep dive analysis of the Q and X operating system and with that I will just give it straight over to you. Thanks. All right. Thanks for that. The great round of applause, please. All right. So, welcome everybody to our talk. Taking a scalpel to Q and X, analyzing and breaking exploit mitigations and peering Gs on Q and X 6 and 7. My name is Joss Wetzels and I'm currently an independent security researcher with Midnight Blue where I mainly focus on embedded system security. I previously worked as a security researcher at the University of Twente where I focused on critical infrastructure protection and most of this work was part of my master's thesis at the Eindhoven University of Technology. Hi. Hi. My name is Ali Abhazi. I'm a PhD student at Eindhoven University of Technology and a visiting researcher at the Chair of Systems Security at Rohr University Bochum here in Germany. My research mostly are related to embedded binary security and programmable logic controller security. All right. So, we'll start this presentation of an introduction to Q and X and discussing the general operating system and security architecture before moving on to discussing the pseudo-random number generators and the exploit mitigations themselves and finishing off with some final remarks. So, what is Q and X? Q and X is a UNIX-like POSIX-compliant embedded real-time operating system. It's closed source and proprietary. It was initially released in 1982, so it's quite old and was later acquired by BlackBerry. Q and X versions up to and including 6.6 are 32-bit operating systems but as of Q and X 7, which was released in March of this year, it's a 64-bit operating system. It's most famously known for its use in various mobile devices because it underpins the BlackBerry 10 operating system as well as the BlackBerry Tablet operating system. But really, this is only the tip of the iceberg of Q and X usage because especially these days, it's far more prominent in automotive systems. Especially in infotainment systems, it holds more than a 50% of the market share and it's set to be used in various self-driving car initiatives. For example, Delphi Automotive has partnered with BlackBerry to use Q and X as the basis of its self-driving car initiative, so that's very interesting from a security point of view. Second very famous use of Q and X is in carrier-grade routers like the CRS series, the 12,000 series, the ASR series. And here Q and X is used to underpin Cisco's iOS XR operating system as you can see on the right of the slide. And that again makes for all the obvious reasons for an interesting security target. It's used in many, many more critical systems. These are just two examples. You can find it in industrial control systems like the nuclear power plants of Westinghouse, surface mining control, turbine controllers, and various military systems such as UAVs or military radios, anti-tank guidance systems, medical systems, railway safety, you name it. So the security implications are obvious, I'd say. Last year, some people might remember we also gave a talk which covered some of the subject matter called Wheel of Fortune. And here we focused on PRNG issues in VxWorks redacted OS, which we can name for NDA reasons, and Q and X versions up to and including 6.6. So in this talk, we'll discuss a lot of different stuff. We'll discuss the new user space and kernel space PRNGs of Q and X7, and focus on the exploit mitigations of Q and X6 and 7, which haven't been discussed before. So hereby I hand the introduction to the OS and security architecture to Ali. So OS and security architecture. Q and X is a micro-carnal, a true micro-carnal. So what it means, it means that basically most of the components of the operating system within the kernel will be out of the kernel. Things which you expect to be in the kernel are not anymore there. So things like file system, stuff like device drivers. Well, protocol stacks, all of them are actually located outside of the kernel. And what you will have is just really tiny, tiny micro-carnal, which have some benefits. So for example, the biggest one is actually the higher reliability for the operating system because there are less chances for the buggy implementation, which cause a crash in the entire operating system. But also, it will provide something we call less rope for hackers to hang on on it because there will be a smaller surface for attackers to target the kernel or the micro-carnal. So it will help some micro-carnal, generally micro-carnal operating systems to get higher EAL levels from NSA or other certificate bodies. So that's micro-carnal. But how then, when you are putting all of these components outside of the kernel or the micro-carnal, how then they are actually going to communicate how they are going to work? So what you will have here is a message boss in the Q and X specifically, which provide a functionality where I assume a program like a network communication. So it will be very similar to the network communication. So what you will have is that an application in the user space wants to communicate, let's say, file system. So how it does it? It send a message to the micro-carnal, which the micro-carnal pass it to the target application. So let's say file system and the file system respond and this message will be then passed to the application. So that's the message boss basically. And the micro-carnal basically, the task of micro-carnal basically is to forward these messages to the other components. But one interesting thing about Q and X specifically is that this architecture combined with something called Qnet and Q and X provide a functionality where you can have multiple micro-carnals running and talking with each other. So let's say they can actually via Qnet, two micro-carnal can have like a talk over Ethernet, which provide greater functionalities, for example, for networking or network communications. Beside that, actually Q and X also support C-scalls, but it's not as big as generally Linux, which you have more than 300 C-scalls, but it's like less than 90. And also Q and X is a POSIX compatible, meaning that you can have those standard Lipsy functions which you write in your code, but here you are using a specific Q and X compiler and then this compiler converts these Lipsy functions to message passing stops basically. Regarding the memory layout, so you will have kernel space and user space, but the only thing which is remaining or stay at the kernel space is the micro-carnal itself. And basically also there will be user space separation, so it means that there is no possibility for some processes within the user space to just like touch each other, for example, because some of them are sensitive. So basically Q and X provide a virtual private memory support via memory management units, but also Q and X provides Unix-like process access control, which we'll talk later about. With respect to Q and X memory layouts, if you look at the user space parts, there's not that much significant difference than the stuff we know in other operating systems. So you will have program image and basically L-floater in a micro-carnal and basically load it and then you have your shared object or dynamic libraries which will be loaded by the dynamic linkers. However, in the kernel space, one thing which is interesting is that all of the address, like basically the base address of the micro-carnals, starts at a static location or static address. And per CPU you will have a different stack. So now let's look at the process management. So process management is a little bit different, so let's say there's a process called PROC NTO, which is basically process manager, but part of it is located at the micro-carnal, but other part of it is located at the user space. So the process manager is actually running by root process, like with the PID 1, and basically it invokes the micro-carnal the same way as other processes, but the only difference here is that it has a flag, NTO-PF ring zero, which provides ring zero privilege for the micro-carnal itself. Besides that, as we said before, QNX actually supports the usual POSIX stuff, so spawn fork, exec, all of them are provided. Also, as I said before, QNX actually uses L-format as a file. But here's an interesting thing, is that if the file system is on block-oriented devices, the code and data are actually loaded into the main memory. While if the file system is actually a memory mapped, code can be in place. So basically multiple instances of the same process share code memory. Also, QNX provides some sandboxing, so it's provided via product manager ability, similar to Linux capabilities, so you can obtain capabilities before dropping root, and also restrict certain actions even for the root user or root process. But these abilities are like significantly big, so you have domain, range, like being locked on lock, so you name it. All of them exist similar to Linux, but hey, here it depends on system integrators on how they are going to implement it. It's not a problem of the operating system, but it will be depending on the system integrators how they are going to use these functionalities. Also, QNX actually supports usual stuff, so with respect to user management. So you have ETC Password file, ETC Shadow, ETC Groups, and also usual utilities such as login, SU, and also it supports the mandatory access controllers. With respect to hashing mechanism for the Passwords, well, QNX 6 basically supports SHA-256, and by default SHA-512. However, it actually has a backward compatibility with MD5 encryptions, which are weaker, so basically if one can crack those devices, which have like their passwords based on MD5 or DES, then they might be able to crack it, and once somebody can crack it, then they will have a long shelf life for attackers to use it. Well, things are much better in QNX 7, and of course patch QNX 6.6, so they are basically using PBKDF2, and with the SHA-512 as default. So looking at the history of the security of QNX, majority of research actually done by BlackRemo while research, which is owner of QNX from 2011 to 2014, and also very, very interesting talk in 2016 by Alex Plaskett about inter-process communication in QNX, PPS and kernel calls. I recommend you to watch that, and there were also various individual vulnerabilities from 2000 to 2008, but the most interesting part is the leaks from WikiLeaks, named Vault 7, which was showing US Central Intelligence Agency were interested in targeting, well, embedded development branch of the CIA interested in targeting QNX, which they didn't do yet until 2014, but we don't know after that. So basically, there were no prior work on exploit mitigation completely, so this will be the first time we are going to talk about it. And also with the PRNG part, we talked about PRNG of QNX 6.6, but here I will talk about QNX 7, PRNG implementation both in user space and kernel space. So let's look at the PRNG. Why we are looking at the PRNG? Let's say like that. Well, because actually PRNG is actually have a broader implication. It's a foundation of wider cryptographic ecosystem. So stuff like SSH, SSL, all of them are relying on those stuff. And besides that, the strengths of exploit mitigation itself are also affected by PRNG quality. So as Yoslater talks about it, you can see that how things like, for example, ASLR or stack canaries can get affected by PRNG weaknesses. So as a recap, we talked about QNX 6.6, the random implementation. So as a recap. So basically the original PRNG, which was implementing in last year, which we talked about, was based on YARO, but not the reference YARO, but the original like earlier version of the YARO, and there were some sketchy design issues. And basically the biggest part, which we talked for example, have a lack of completely broken seat control or not having basically any seat control, and also low-quality boot time entropy and some entropy source selection, which was based on system integrators, and we show some examples of how things can go bad when system integrators don't care about it, and the operating system itself doesn't provide a proper PRNG. However, things got much better in QNX 7 after our assessment, they incorporated some of our suggestions, so right now they are actually using FORTUNA implementation, they are actually using new entropy sources, which I talked later, and a proper seat control mechanism, which didn't exist before, and basically overall quite much better, and still doesn't mean that everything is fine, so still there are some design decisions or implementation decisions which the system integrator have to decide, and still there can be a tech surface, but from the operating system side, things are much better. So let's look at things changed, so you don't have to actually look at all of them, only look at the green parts, because that's the parts which things changed. First, and foremost for fixing the problem of the boot time entropy, QNX actually QNX 7, they right now provide a seat file source basically, which means that at the boot time, you can provide a randomness file which contains some random entropy to the operating system at the boot time, and later once the seat gets used and exhausted at the run time, it can get updated, but the point here is that the frame where you have to actually provide per frame where you have to have different, for example, seat file, but beside that there is also user supplied sources of entropy, so there are different kind of user supplied sources which can be provided, but other part is still the receipt source basically, which is still weird, because they are still using, for example, getUID and getPID, which is not at all random, because it's just completely static, it's just a time of day, which is not random, but only proper one is RC4 random function, which is not bad. Regarding the QNX 7 kernel PRNG, which QNX 7 actually introduced a new kernel PRNG, and there is an implementation of it as a function called randomValue in the micro kernel of the QNX, and it will be used or being used as for the ASLR and stack canneries by the micro kernel, so basically what you see here is that you have different sources of entropy, so for example, clock cycle, you are using the PID or like the current time in nanoseconds, also for example, current CPU, like wake up timer, and also some random seats, which you can pass it to a PRNG input block, which gets passed to a short 156 function, and basically what will happen, the PRNG state or the output will be divided to 8 blocks, and the first block will be used as a salt, and the second block will be used for the output of the random value, the 32-bit, and there will be an iteration issue whenever you actually need a new random value, which this iteration moves each time from location 0 to 1, 2, 3, so each time the location which you are choosing, the 32-bit will change. Basically that's the QNIC 7 kernel PRNG and now the meat of the work actually, exploit mitigation with QNIC 7. All right. Thank you for that, Ali. So let's start to look at the exploit mitigations. Why take a look at exploit mitigations? Well, because the mitigations that were used to in the general-purpose world, Windows, Linux, BSD, didn't come falling from the sky, especially not in their current incarnations. There's a long history of weaknesses, bypasses and subsequent improvements, as you can see for example, for Windows on the bottom of the slide, and because there's nothing like that for QNICs, that means that it's very fruitful ground for finding interesting stuff, which is why we took a look at it. So as of QNIC 6.5, as you can see in the table, there is support for data execution prevention, address space layout, randomization, stack entries and relocation read only, but don't get too excited because these are not enabled by default, so it might just mean that you encounter a firmware image with QNICs and it's fully up-to-date, but if system integrators didn't explicitly enable support for all these mitigations in their tool chain, then you might be just exploiting like it's the 90s. You also shouldn't expect any support for advanced mitigations like V-Table protections, control flow integrity or kernel code and data isolation, so this is really just it. Let's start off with data execution prevention. For those of you unfamiliar with it, it seeks to prevent, well, the execution of injected payloads into data memory, and roughly speaking, you have two main architectural styles for a CPU. One is the Harvard one, where you have separate, physically separate code and data memory, and the other one is the von Neumann one, where you have shared data memory, and in order to prevent the execution of injected payloads in data memory, you effectively seek to emulate a Harvard architecture on a von Neumann one, and typically this is done as on x64 on the bottom of the slide, being facilitated by hardware support in a memory management unit, and here in a page table entry, you will have a specific bit like the NX bit which regulates executability of this particular page. Now, Q and X DEP has support for several of these NX-like flags in MMUs. It has support for it on x86 and x64. It has support for it on ARM. It does not, however, have support for this feature on MIPS, and it has varying support for PowerPC, but, you know, that's PowerPC. The big problem with Q and X DEP is the fact that the defaults are insecure, so the problem is that even if you have hardware support here in a version that has support for DEP, then still the stack will be left executable, even if the heap is not. So this is something to really check for when you encounter a Q and X firmware image. What's more is that the typical GNU stack ELF program header is ignored by the program loader, so regardless of your linker settings or whatever, this will be executable. Now, it's possible to make the stack non-executable by specifying explicitly a particular flag in the microkernel startup but the big problem is that this is a system-wide setting, so if you have executables which require for legacy or backwards compatibility reasons an executable stack, they can no longer be included with these new firmware images, so even though we reported this and we said, you know, this is just enough rope to hang yourself with as a system integrator, this issue is still present on Q and X6 and Q and X7 and this really is something to check for if you encounter a Q and X firmware image. The second mitigation is address space layout randomization, and again for those unfamiliar with it, address space layout randomization seeks to complicate code reuse attacks like returning to programming by randomizing the memory object addresses, so a typical exploitation flow you can see on the right of the slide you find existing code to reuse as gadgets and snippets and stitch them together a bit like a ransom note on the top of the slide. Now ASLR seeks to prevent this by using randomness as a means towards the goal of memory layout secrecy because if you don't know where the various code fragments are in memory, then you can stitch them together to form a raw payload or at least that's the idea behind ASLR. Now Q and X ASLR is enabled by starting the micro kernel with again a dedicated flag which is not enabled by default. Child processes inherent their parents ASLR settings but it can be enabled or disabled on a per process basis so you have a good opt out scheme so it's not an opt in scheme so don't look for mistakes like that. Memory objects are randomized at a base address level so it's not a very fine grain form of ASLR but that goes for most ASLR versions and most memory objects are randomized except for the kernel code addresses and how terrible that is depends on your opinion of the usefulness of KASLR in general so that's not the real problem here. The problem that is a problem in practice is the fact that Pi is disabled by default in the tool chain so that means that unless you explicitly enable it then all the binaries you have and you will compile including the system binaries won't have randomization of code memory and if you look at a lot of firmware images of Q and X in the wild you will find that in fact code memory is never actually randomized which greatly reduces the usefulness of ASLR. To learn how Q and X ASLR works under the hood we reverse engineered the memory manager of Q and X which you can see mapped out here and I'll save you all the details but basically it comes down to the fact that all of it is underpinned mostly by calls to M-map in the microkernel and there are two functions that actually regulate the randomization and those are marked in blue which is the stack randomized function on the left and the map find VA function on the right and these both rely on the same random number generator which we'll discuss in this this talk. Now the first of the functions map find VA among other things randomizes virtual addresses which are returned by the M-map call and it does this as you can see on the right of the slide by subtracting or adding a random value to the found virtual address and this random value is obtained by taking the lower 32 bits of the random number generator result, bitwise left shifting them by 12 and then extracting the lower 24 bits and the problem already here is the fact that the application of this bit mask contributes at most 12 bits of entropy to any address randomized in this fashion regardless of the quality of the PRNG in general which is worse as we'll see in a minute. The second of these functions stack randomized well as the name says it randomized as stack start addresses when a stack is allocated either when a process is started or when a new threat is created. It does this in the same fashion as the previous function by subtracting a random stack pointer it takes the lower 32 bits of the random number generator result as you can see on the right of the slide then bitwise left shifts it by 4 and then at most extracts the lower 11 bits depending of the size of the allocated stack and this contributes to the bit mask again at most 7 bits of entropy which is also worse in practice. This is mitigated a little bit because it is combined with the results of the previous function because under the hood of course the stack is also better because this won't matter a lot. We'll take a sip of water. So these actually these upper bounds are quite optimistic because Q and X6 ASLR uses a very weak PRNG you can't really call it a PRNG because they directly use a source of entropy called clock cycles and as you can probably guess it maintains and retrieves a 64 bit life running cycle counter and the implementation of this is architecture specific on the right of the slide you can see that on x86 it will simply use the read time stamp counter instruction and for PowerPC it will use time base facility and various other kind of architecture specific options. Now the first thing that springs to mind is the fact that if you want to guarantee memory layout secrecy using ASLR then you will also need to keep the internal state of the PRNG secret because that might allow people to reproduce the ASLR settings at a given point in time. There's no PRNG here but just a raw entropy source that means that in that scenario clock cycles would have to be a secret value which of course it is not. It can be requested with unprivileged axis. It's incorporated in a lot of different kind of drivers in network packets broadcasted all over the network so in theory you could mount a reconstruction attack but that's overkill and kind of involved considering the fact that it doesn't contribute a lot of entropy and another more feasible for breaking it. So we measured various kinds of processes across different boot sessions and harvested the memory object addresses then we used the NIST entropy source testing tool to obtain a min entropy estimate for all of these memory object addresses in different kind of classes and here it is good to realize that 256 bits of uniformly random data should correspond to 256 bits of min entropy and we found that the average min entropy of an address on QNX6 was 4.47 bits with the lowest min entropy being 3 bits for shared libraries and the highest 6 bits for the stack and this is very very weak if you compare it to other 32 bit operating systems as you can see on the right of the slide for example for mainline linex varying between 8 bits of entropy and 19 bits or for example linex with the packs patches where you vary between 6 bits and even 27 bits and why is this a problem you might ask well this is a problem because of the potential of brute forcing so if you have a typical networking daemon where you have a forking architecture and let's say that upon every incoming connection you spawn a new child to handle this client connection and a for call will be called and because of memory layout inheritance a child process will have a copy of the parent process memory layout and because this is applied after ASLR has been applied it means that the ASLR randomization is also copied to the child which is static every time this child is respond now an attacker trying to guess the address for a certain code address for example might try an address and measure the response in whatever way and if the child crashes and is restarted they can try the next address and if there is not enough entropy in the randomization of these addresses they might succeed either locally or remotely or both and the address is needed to build their ROP chain and does this work in practice you'll ask well you can see on this slide that in fact it does on the left you have a vulnerable service which runs on the network port 1337 it has a trivial stack buffer overflow it has ASLR enabled and on the right you can see it remotely being exploited over the network brute forcing ASLR in 23 seconds to pop a root shell so yes that works in practice brute forcing ASLR is interesting but memory or information leaks are even more interesting typically you find an information leak in the application you're targeting or you craft one from a flexible enough vulnerability but it's nicer especially for local vulnerabilities to have a system-wide information leak in this case we'll discuss too but there are many many more of this kind in Q and X Q and X the first information leak we discovered is the work of S information leak and this basically works by relying on the fact that Q and X like many Unix like operating systems has a process file system and here you have dedicated entries for each running process on the system and you can interact with these different entries using the DevCTL API where you can request information like the register values or stack addresses or the general memory mapping layout in general regardless of privileges or whatever our world readable so that makes it very easy to write a very simple application that across privileged boundaries for a low privileged user this closes the memory layout of the microkernel on the right you can see that it is made even more convenient but the fact that they include in a lot of Q and X releases the PIDN utility which allows to incorporate this functionality by default so even if you can't write your own application on the system to exploit this information leak you might just be in luck and find this utility there to do it for you the second information leak we found is residing in the LD debug environment variable this is an environment variable which allows you to specify various requests for debugging information and if you specify the all option then it will give you a lot of debug information among which are the addresses of shared libraries and the interesting thing is that for example linux or bsd this option has privilege checking so if you try to do this for a setuid binary and you're not a root user then it will not output that information but on Q and X no such checks are present and you can obtain this information across privileged boundaries which makes exploiting setuid binaries that much easier after we reported some of this stuff they made improvements to Q and X7 and Q and X7 and now still has disabled ASLR there's no KASLR but they do use a new kernel PRNG that Ali just discussed and that's good but it doesn't make Q and X7 ASLR much stronger despite this new RNG and despite the fact that they have a 64-bit address space they forgot to remove these bitmaskings that are applied to the randomization functions so as a result they still have a theoretical upper bound of 7 bits of entropy for stack addresses and 12 bits for the virtual memory addresses or most of them as they are allocated another interesting thing to note you can see on the right of the slide is the fact that code memory is mostly loaded in the lower 32 bits of the address space which also greatly reduces the potential effectiveness of ASLR and 64-bit operating systems they did fix the LDD book information leak but unfortunately for defenders and fortunately for attackers they did not completely fix the PROC-FS info leak so as you can see on Q and X7 above you cannot longer use the PIDN utility but if you're just writing your own application compiling it and interacting directly with the PROC-FS you can still disclose this information across privileged boundaries so this is a information leak free there to use the next mitigation I'd like to discuss are stack canneries they protect against traditional linear stack buffer overflows which are much more interesting on embedded systems than they should be for the people unfamiliar with it it basically works as follows you generate a master cannery value using a random number generator and again you keep it secret and you insert it between the local variables like a local data buffer and the saved return address on the stack and the data variables so when an attacker then overrides the saved return address and upon return of the function traditionally you would hijack control flow but here first the saved cannery is checked against the master cannery and if a mismatch is detected instead of returning to the saved return address you instead invoke a violation handler and thus prevent control flow hijacking now Q and X uses the GCC stack smashing protector implementation of stack canneries so on the compiler side it's what we're used to in linex or bsd for example and that's mostly okay but on the operating system side of the implementation it's all custom and that's where the problems start because the user space master cannery is generated at program startup when libc is loaded now typically in the GCC implementation it uses libssp's guard setup function to regulate this and then on various platforms they have sometimes differing implementations but it's mostly the same on linex for example Q and X however uses a custom init cookies function and that's where the problem lies because again it uses a weak random number generator it draws entropy from three sources as you can see on the bottom of the slide it uses again clock cycles and it combines this with a local stack variable address and the address of the function itself now these last two only contribute any entropy if aslar is enabled and again even if aslar is enabled there is no entropy relies on clock cycles as well so we decided to evaluate the cannery min entropy across three configurations without aslar without position independent executables and with aslar and with position independent executables and found the min entropy on average of the canneries to be 7.79 bits and aslar had no noticeable influence here and this is less than ideal because using a csp or g they should have had at least 24 bits of min entropy like in this case they include one null byte in the 32 bit cannery or if they used a full cannery they should have had 32 bits of entropy and again this is a problem because of brute forcing attacks against canneries in kernel space however the problems are even worse because the microkernel is neither loaded nor linked against libc so the master cannery in the kernel cannot be generated by this init cookies function so they should have implemented a master cannery generation function in the kernel but they forgot to do this so the microkernel is protected across various functions using stack canneries but the stack canneries never actually initialized and so they're always zero which kind of defeats the purpose of having stack canneries in the first place now we reported these issues to blackberry and they're now enabled by default stack canneries they also generate 64 bit canneries on 64 bit operating system the user space canneries they mix in an alpha auxiliary vector value based on our best practice suggestions by taking a 64 bit random number generator value from the kernel PRNG and transporting it to the user space process to mix in with the init cookies stuff in the kernel space q and x now concatenates 232 bit kernel PRNG values during very early boot and creates a cannery out of that so basically canneries at least are fully fixed now and that's good news for defenders at least that brings us to the final mitigation relocation read only or the way this works you can see on the right of the slide is that dynamically linked binaries use relocation to do runtime lookup of symbols and shared libraries so if you have a function during runtime and you have it in a shared library once you hit that function it will be looked up and the address will be stored in the global offset table now for obvious reasons this relocation data is a popular target for overriding to hijack control flow mostly because these addresses tend to be static regardless of ASLR and because of the fact obviously that once the control flow hits that particular function then you can hijack control flow in order to mitigate this partial railroad was invented which works by reordering the ELF internal data sections and making them precede the program data sections and then making them read only after relocations have been done so attackers during runtime can no longer override these entries now the problem here is because of something called lazy binding lazy binding means that most of these symbols won't be looked up at program load time but during program runtime and as a result the global offset table will remain writeable during runtime now you'll have to relocate or have to relocate you have to make sure that this does not happen how do they do that they do it by disabling line lazy binding and then making the PLT got read only at program startup they implemented this on q and x6 and that's very nice but the problem is that their implementation turned out to be broken so as you can see on the left is what it looks like on Debian and what it should look like there you have all the internal data sections precede the program data sections and are covered by the GNU railroad segment and made read only on the right you have the q and x6.6 implementation for the same application where you can see that only some of the internal data sections precede the program data section which is the most interesting of the overriding targets actually does not precede the program data section as a result it's not covered by the read only segment and regardless of your settings in your linker you will be left vulnerable to this attack even if railroad has been enabled the root cause of this is the fact that they did not do proper linker section reordering in practice it looks like this on the left again you have Debian you have full railroad enabled and you can no longer write to global offset table entries so that's a broken mitigation right there on top of that we also found a local bypass for railroad again the LDD bug environment variable turns out to have an undocumented function called imposter which allows us to disable railroad for whatever reason without any privilege checks whatsoever and this is very nice of course for exploiting vulnerable set UID binaries and as you saw in one of the first slides there are a lot of these in the history of QNX so this is actually very nice in practice both of these issues were reported to BlackBerry and are now fixed with patches for QNX 6.6 and QNX 7 so that's good news that brings us to the final remarks so we disclosed all of the issues we discussed today to BlackBerry most of these issues are fixed in QNX 7 patches are available for some of these issues in QNX 6.6 as you can see on the link in the bottom and the table that's displayed a word of warning though to both defenders and attackers most of these patches will take a long time to filter down to the original equipment manufacturers and the end users especially for deeply embedded systems which might be a couple of minor release versions of QNX behind they'll have to upgrade all the way to QNX 6.6 and then apply the patches roll out the firmware update so these issues might be encountered sometime in the wild concluding most of the mitigations turned out to be okay on the toolchain side but that's mostly because they relied on GCC where the problems were really found and this is not just a QNX thing but this is generally an embedded thing is on the operating systems side and why is this the case because QNX cannot benefit directly from any work that's done in general purpose operating system security because it cannot be easily ported to a different architectural lineage and the result is homebrewed DIY mitigations which turn out to be not as good as you'd want them to be what's also really evident if you look at these issues and other vulnerabilities that you find here is the lack of prior attention by security researchers a lot of vulnerabilities feel like they're from the early 2000s and the information leaks are really evident of this and again as a word of warning to many people embedded random number generator design remains difficult many of the entropy issues in the embedded world lack of proper entropy sources mean that the design burden is often placed on the system integrators regardless of the good intentions of operating system designers on a more positive finishing note QNX at least attempts to keep up with general purpose operating security which is more than can be said of most embedded operating system vendors which don't have any exploit mitigations whatsoever as I discussed in my talk at thisyearshardware.io conference they had a very quick and extensive vendor response sometimes directly integrating our feedback into their new code and as a finishing note I'd really like to call for more attention to embedded operating system security in general if we ever want to hold them to the standards we'll hold our laptops, desktops servers and smartphones to which we shoot for things that are deploying cars critical infrastructure and military systems you can also look forward to more QNX stuff in the future from us at Reckon Brussels where we'll have more questions of gone black hat and infiltrate so with that, if there's any questions I'd like to take them now Thank you Ali Abhazi Abhazi and Joss Wetzels now we have some time for Q&A you can just line up on the microphones here, here, here and back there I got one on mic 5 we'll start with you probably did the very first question work on QNX I feel a bit left out of one of your slides oh that was not my intention what is your name or nickname thanks any other questions oh, there I guess for the issue where the stack canary wasn't set up properly for the kernel was that an issue where it wasn't set up at all or where something like it wasn't persisted or reclaimed out of like thread local storage to actually be placed in the spot for it on the stack so the problem is that the way they implemented it is they had no initialization routine at all for the master canary so there were references to the canary all across the microkernel but it was never actually initialized and because the microkernel stack canary was located in BSS which was initialized to all zeroes in very early boot that means that it was predictably zero all the time you know they used it but never initialized it so it's very predictable anybody else with questions don't be shy, come on well if there aren't any questions left alright thank you very much for awesome talk