 Hi, everyone. So Johannes and me are going to give a talk about Nito and for work vacuum cleaners So this is Johannes and Johannes is my student and Fabian student, but Fabian is not here We supervised his master thesis on those vacuum cleaning robots And I am yes guy you might know me from hacking all the other things like Fitbit or broadcom Bluetooth Yeah, I don't know now. Does it work? What so the question is why is a vacuum cleaning robot interesting or why is it interesting IoT? Well, first of all it has a local user interface USB console something so it can quite often clean without the cloud But there's a lot of features where you would need the app or the cloud so for example, you have this app and it has a local Wi-Fi socket connection whatever and then you can Do some direct control so this direct control is really important if you want to run your own shitty vacuum cleaning robot contest or something So that they can fight. I don't know In the Nito system, it's not really useful because this one is not for you doing the standard cleaning map whatsoever But it's funny And you need it for the first setup with the beehive cloud the beehive cloud is a special cloud It's just there for the user account. So your account in this ecosystem Then there is a second cloud because it's everything is better with more clouds the nuclear cloud and There's a web socket to that you can send robot commands with your app so if you have a valid beehive session and so on and then you can Send those robot commands to the robot like clean my house and then you will also get back a status like yes, I'm on I'm cleaning I'm stuck whatsoever and Everything is better with a third cloud. So there's also this This Amazon stuff where maps are uploaded Yeah, so lots of lots of clouds just for one vacuum cleaning robot And you might also not really know what is going on in between those because that's nothing that you could sniff But the remaining parts are very interesting So recently I realized That It's not just the Nito cleaners But also the four-way cleaners I went to a store to a lady and said yeah Can you present this one to me and she really did a nice job with this? And then I asked her like wait what did you say they get Updates and she said yes, and she was very proud because the old version needed it over USB And a new one can do it even wireless with this firmware updates and then asked her like okay But why does it need a firmware update such a vacuum cleaner and she said yeah, it gets new functions like For example, you have a map of the floor that is cleaning and if you want the map of two floors Then this is supported by the number and then you can for example like draw a line So you get a map of your apartment and then you can draw a logo line instead of putting this magnet stripe there So it's kind of useful But it gets updates and then I asked her like how does it work with the updates? And then she said yeah, you know this thing has a CPU and the CPU gets new functions Was a bit funny, but yeah, I mean it's her job So that's all of the explanation that I got, but it's interesting to see how they actually work So what is in there for for both those ecosystems you get the binary for at least those that you can update over USB and Then there's some information about the diversion and so on and the hardware model, etc Then there is a thing that is encrypted and has a certificate attached and so on There's also some plaintext part Which is the user interface and then there is a Footer and you cannot flip any byte without getting an error during installation So there's check some certificate over everything. It's quite high to change something here So everything looks kind of secure here, but at least it's not completely encrypted and you can get some metadata Yes, and as I said, it's that the CPU gets new functions obviously So this this firmware update process I'm going to tell you this now so did you understand what it is for but it's actually what we reverse-engineered in the end So it's not how we did it, but just to let you understand how it works So the firmware update contains a QNX IFS QNX is some Unix like system, but proprietary stuff IFS is just this file system there and it's signed and encrypted for a secure boot and This platform that is running system while it's booting it's decrypting this thing And even the firmware update process That you get the firmware update And then they did it encrypted and signed on to the the hard drive or flash chip and booted directly so During the firmware update process. There's never a state in which you would get a plain text Version of this firmware This actually means that you saw to to extract something from the system You need to break the secure boot in some means So What I did I was looking through all kinds of serial ports, so there's tons of Poids there and the only unlabeled port is actually a serial port This one is there It's nice because for the new forework robots they even have it just as an outside port So you don't need to disassemble your robot to attach it But for the older ones you need to disassemble your robot And then if you have this USB interface and say some test mode on and then reboot whatever action You will get an extended boot menu and in this one you can say Something it says serial download But a serial download is not real download so you cannot download the system But what you can do is you can upload a system which is downloaded and then boot it So like netboot Yeah, so this is pretty useful But still a problem if you don't really know like what to boot there So you need to guess what can I boot over there? And there's this big chip that looks like the main chip has some weird Name you cannot Google it you see it's ti and then there's one thing that you can Google Which is the form factor and that's for a certain chip family and Then I googled a bit more if there's like an IFS for this chip family and What I found is there's some fun found with 27 project and it's for example for the Beagle board plaque And for those there is existing IFS which you can boot if it boots You will get some error. So you will get something and then it breaks during the boot But at least you can see it starts booting and at this So when when you are here and you already like broke the secure boot So that's that's the interesting part. I mean we booted something It does not boot completely, but we can see we execute something Which is out of this secure boot environment and now the next challenge is That you need to compile your own QNICs IFS which boots on this specific platform But yeah, that's breaking secure boot. Well, so how do you? build an IFS so well So the the nice part is that there is academic licenses and I asked for one and then they denied it I don't know in a second try they accepted it But only for QNIC 7 and it's it's just an awful process So back then it was just faster for me that I just download the installer and then hack the license management and It was kind of easy and I started it like binary patching and installer blah blah blah Pretty annoying because then I just found out there's an install lock and this install lock Contains the serial number that you entered and then this compared and then says which features is Available for this license key and so on And then I just googled this format and then I found other installation keys on the internet Only problem. They are pretty old So and and that's the thing why this hack does not really work anymore because what they did is they They just disabled to download Possibility so now before you can download the installer you need to enter a valid license key And that's the fix Which they came up with even though we did not really contact them on this. I don't know it was just fixed some time ago But before you could just download it. It's also pretty annoying because then you need to have your Virtual machine set with everything to the old date and so on so that it works So it's really nothing that you would do in a productive environment. It's really more like a very hecky thing But it worked to to build something So the next part is I Wanted to build something that that boots on the chip and I didn't know anything So there was no documentation for the specific one and I tried like to initialize for example the flesh So my idea was like I initialize the flesh and then access the system because I didn't really know that it also was encrypted and everything But nothing of this work because everything was mapped to different addresses The only thing that seemed to be mapped to the standard address was the rum area and Well, then then then I was just a bit like what should I do? What should I do? Well, maybe it's just reboot and print a ram area that you can access was my idea and that's what I did then so it's very Very slow what I did is just a reboot then and For loop that loops over this upper memory area and make some k print f over the serial with the slow bowdread and so on and just Prince the memory next time when I do this I might like on addition to some I don't know base 64 or something. I didn't do anything of this I just printed like characters and was pretty pretty slow But it worked and then I saw it's actually I just got eight times the same thing because of some memory mapping But okay, so next time it will just be faster, but I extracted the system So what what is done there is basically just a cold boot attack So you you have the original system you reboot the ram is not re-initialized And I was putting the IFS in it so my own IFS Into a different location than the original Nito IFS and so I could just k print f this pirate into a file and The funny part is because it's the system during runtime. It's not just all the binaries That run on it, but in addition is also stuff like the Wi-Fi password that this vacuum cleaner was configured so My next idea was to actually boot my my own Nito IFS And it's pretty tricky because the IFS is Copied to a certain memory area that is specified in its header Then it's decompressed, so you need to compress it correctly and so on and Then you have this image directory which then contains files and so on Yeah, and that's how it looks like but the problem here is so you can actually access everything in it But if I boot the same thing again into this thing then there's already the system initialized and that would mean that you Have for example stack pointer and everything already initialized And then you boot again into the same thing with a strange memory state and this one does not really work So I don't know I I tried it a few times But it's also pretty slow because then over serial you upload the system and so on and then it does not work So yeah, but whatever I mean it would have been nice But then I didn't continue that because it's just a bit awful to do it But at least there's funny scripts like pinky and brain so Brain is this this main robot binary Obviously and pinky is a watchdog Pink is pretty pretty awful too because we tried some some Debugging so live debugging everything on this robot later And whenever you crash any process Then the process is restarted and so on and if the process is not running It might be that the complete robot restyles and so on so it's a bit annoying this this watchdog and The brain is also it's just calling a bin robot and so on So yeah, I didn't I didn't patch it to the part where I could boot it again, but so what This secure boot bypass is something that they fixed after the mr. MCD But it took them quite a while until they they noticed our talk So in the end of November, it was pretty funny. I was getting an email from the core developer I mean meanwhile We were already at a point where we were sending them proof of concepts for certain hex and so on that we had to this to their Support system and people said like we did we forwarded it to the developer sense on and nothing happened And then at some point I got this email from the core developer who said hey Oops, can we talk? and Just one day later at the same Time of the of day I was just so we were calling them if we prepared like everything So what we're already CV is that we hand it in and what is new and so on And tried to talk to him in a structured manner And actually he was writing that email during night in in California time And we didn't know that because I don't know who who writes maize at 2 a.m I don't know and then he also got our call on the following day like during that time but he actually Like noted everything that we had and it was really a professional thing because he really noted down everything and they fixed it But it also takes time to roll those fixes out So you can see the fix actually in the newest Nitto, but what connected the seven update But not for forework because they test firmware on the forework first And then if it not not on the on the needle first and then when it's stable they roll it out to the forework So the cheaper American product is their test platform so to say and then the German platform with this Better battery for example and more sensors. That's then this second platform where they run it Yeah, and the thing that they said about this vulnerability is that we please Please do not publish the IFS that you extracted. Okay, obviously and please also do not publish this Caprint F for everything that we built to extract the IFS. So I mean, that's that's really a rocket science For loop, please don't do that So software hacking Now we had all the binaries and that's the part when it comes to something that that is really More easy to do and more fun Because all the the executables there contain symbols and everything. So it's a really a fun thing and Even better There's core dumps. So they have a script running That is really useful. So QNX has some default behavior where core dumps are written to a certain directory whenever a process is crashing and then this is copied for so it's first encrypted and then copied to a USB stick if you attach that and click some things on the robot So that's pretty useful. I guess for any developer who gets a robot which is doing weird stuff So what was the binary doing when it was crashing and so on? But it's also useful for us if we would have the decryption password Um, so what is the description password? Yeah, there's this file Calls, so there's RC4 script. It's a binary, but it does not have any password phrase as an argument Then you can open this one. I don't know this one is now opened in in Githra But also it also works in Ida or whatever This is the password. So nine characters Special charge. I don't know about how it coded and it's the same for all the the series of robots. So They so this So it's for example also valid for a very old one but but work 85 and so on And it's very nice to see that it's just the same password throughout all the versions and you can just decrypt all the locks and core dumps Now the question is why did I even have a core dump on my robot? So, I mean I found a core dump and wanted to decrypt it. So why did it I found a robot core dump? Yeah, Fabian was testing stuff. So my master student back then who also now supervised Janice So Fabian he did some testing with the cloud so he was just sending some arbitrary payloads and checking if anything on the robot crashes and He thought there would not be a crash because the the process is just restarted by the watchdog So he don't really notice when the service is crashing and There was this core dump which just Contained lots of ace Who would do that or lots of ones? I don't know who would do that? well It was obviously from this testing And What happens is this this nuclear cloud is sending some command to the binary bin astro So that's for the cloud connection, which is then executing something on bin robot and this bin astro is the thing that crashed And that's the point in time when I said well Johannes, can you look into this? This is interesting And now you're going to hear the second part of the story how we got control over the vacuum cleaners Yes, thank you. So we have this buffer overflow vulnerability and How can this be triggered? Well, there's this URL highlighted in blue up there and this is used to send commands from the smartphone app via the nuclear cloud to the robots and Well, this is used to set up some cleaning schedules or get the robot state or whatever and Well, it contains a serial number of the target robot so that the nuclear cloud knows to which the request will be forwarded to and Yeah, so here you can see the request and it contains an authorization header So one could think that we can only trigger the buffer overflow if we have the key To control the robot, but this is not the case and we will see in a minute. Why and And the request body contains the command Normally, there is a JSON object which holds the command, but if this body gets too long, there is the buffer overflow Okay So well now we checked what can we do with this buffer overflow vulnerability team? Therefore we send the Payload with character pattern and analyze the decal dump which was generated and The registers in the core dump reveal which register we have control over and Yeah, we can control the register our fault who are 11 directly and also the program counter Well, this is quite promising because it means that we can alter the program flow and therefore execute arbitrary code So, yeah Now we think thought okay, it would be nice to get some command execution on that robot and let's build an exploit Okay, therefore we started to analyze the root cause of the buffer overflow and the link register in the core dump revealed the last function which was called before the process crashed and The function H my context cleanup in gray was the last function call and This function belongs to open SSR the open SSL library So we thought yeah, maybe the Vulnerability is in a narrow code and not in the open SSL Library, so we checked the core craft to this function and this core craft is shown here And we analyze the function and reverse engineer the function preceding this function call and Well, this vulnerability is located in the red function very fast signature This function name is a bit misleading Because it seems that the programmers don't understand this cryptographic concepts correctly Because this function actually verifies and calculates an HMAC and during this HMAC calculation process The function on the process crushers and this is the reason why we can trigger the buffer overflow unauthenticated and arbitrary robots Yeah on the right path call graph side you can see the the handle of the requests which are Send to the robot via the nuclear cloud and on the left side The requests in the local network are handled But we cannot trigger the buffer overflow in the local network because in other binary receives these requests and forwards it to Ben Astro via IPC, but the IPC implementation of Neato Limits the maximum packet size to five hundred and twelve bytes and this is not enough to overflow the the buffer Okay, now let's have a look at this vulnerable function Here the orange line shows the vulnerable Function call it. It's called UTF-8 in code and it takes a input buffer UTF-8 encodes this string in the in this buffer and stores it in a target buffer and like you can see at the top of the slide the The source buffer is much bigger than the target buffer and because there are no length checks. Well, there's a buffer overflow Okay, so how is this string built? Well, you can see it in this third last line as in printf this function takes three strings and concatenates them first there is the Robert Sewell number then The the the data of the command of the request Sorry, and the command itself which is transmitted in the request body and this means we can control how the buffer will Will look like and well that allows us to build the exploit Okay in order to exploit this vulnerability we have to stick to some constraints first the exploit request is not Cannot contain any null bytes because this request terminate at the nuclear server and do not reach the target robot and second the request can only contain ASCII bytes because the non-ASCII part of the requests are somewhere cut off in the request processing on the target and Therefore they are not copied into this target buffer Okay, this makes the exploit development a bit harder But on the other hand the exploit mitigation of mechanisms by the operating system are not activated in QNX 6.5 the ASLR and DEP and stuff like that are not activated by default and Neato did not activate them and so they are not in place and Stack canneries and other stuff are also not also not activated. So this makes our life a lot easier Okay, so we build a return-oriented programming exploit and Yeah, our target is to execute the libc function system which allows us to well to execute arbitrary system commands and In order to do that there are three steps first We have to place the command which will be executed on the stack of the process afterwards we take the address of this command place it in our zero our zero is The register which holds the first parameter for function call in the ARM architecture And they are finally called a function system and we do this you think utilizing the corresponding rub gadgets and chain them together to a rub chain Okay, well the problem is that the address of the command string on the stack and The address of the system function Do not stick to these constraints I explained before so this means we cannot transmit them in the exploit request directly So to solve this issue we just transmit Requests which stick to these constraints and use an add gadget to add those values and to generate arbitrary addresses and This allows us to generate these addresses and call the function Yeah, but the address of the rub gadgets have to stick to the constraints because they are also transmitted in the request Okay, so what does this mean? Well the exploit allows to execute arbitrary commands on arbitrary robots unauthenticated and with root permissions so well, this is quite bad because you can do basically anything with this and You just have to need to have the serial number of the robot because the cloud needs to know to which request which target the request needs to be sent and Well, that's not so hard to get because the serial numbers are printed on the robots and on the packaging itself You can just go into a shop check out some serial numbers and attack arbitrary robots So, yeah, so what could an attacker do with this? exploit where you just can prick some robots and annoy the users or Yeah, you could could attack further devices inside the local network which would be protected by a firewall otherwise, which is quite bad and Yeah, you could use it to build a botnet and to Start some d-dosing or Bitcoin mining or whatever Or yeah, you could extract some privacy critical information from the robots like floor maps or other sensor readings Which is also Quite quite bad regarding the privacy of the users Okay, some final words to the responsible disclosure timeline Well, we started to Contact them back in March 2018 and our first responsible disclosure attempts terminated at the first level support because they just not forwarded them to the Responsible developers so at some time we just sent them the exploit in the email and well they just closed the ticket and did nothing and Yeah, so we didn't know what to do, but finally we got the contact in November 2018 and the developers were quite cooperative and communicated with us like he's cassette before and Rather fix itself was quite easy because they just had to implement a strict length restriction on the server side And they additionally implemented some mechanism to check the HMAC so that you cannot send Requests to arbitrary robots because you need to know the key the shared key So yeah, they fixed it quite good quite fast and you know the robots The the robots are safe Okay, thank you So first a question from me to you who of you has a vacuum cleaning robot no matter if smart or not Interesting so there might also some of the hands that were rising who now have questions To us about their robots. I don't know about their privacy Nobody is concerned or wants to know more Yes, Dave Just wait a second for the microphone You already asked the question yourself why they do need a software upgrade But why do they need all these cloud connections? What is the reason for this? They connect to your local Wireless LAN so you could directly control the robot not why are some Yes, so normally you could do that. That's that's true But they don't I guess that some features like drawing these no-go lines in the map are Harder to implement if you don't do it with the cloud And the next problem is so it's it just would be in your local Wi-Fi and customers want features like Oh, my friend is visiting So I started the the robot now and when I come home the it's clean So they want to do it remotely and then it's easier to implement like just one interface and then it's via cloud So it would probably not be needed especially not for the nurse of us who have their VPN to home anyway But it's very convenient. I guess for the average customer Yes, next question. Are you aware of any vacuum robots which do not come with all those this cloud crap? So what you can do is actually that you can use this Vacuum cleaner without the cloud and then it just cleans So yeah, it's just you press a button and it cleans And Also nice thing is so compared for example to show me that robot is then it seems to stop broadcasting this This Wi-Fi so you cannot take over a robot that is Unprovisioned You can just use it without the cloud, but then you could also probably buy a cheaper model At least they are very they are very good in cleaning I would say so they have a good power and are very efficient And they have this deformed to go into corners and so on so there's reasons to buy Just this special kind of robot and then just not to connect it But also might be a bit too expensive then So two questions great. Is this a follow-up question any of this? No Can you tell a bit more about the four work model? Is it like the same hardware and Proven firmware or is it a different hardware? It has rainbow colored LEDs No, I mean for real, but not only this so So first of all, there's a brush so on both models There's a brush in the front to go into the corners and it has some ultrasonic sensor to the side So that it does not bump into the wall, okay But for the four work model, it also has three ultrasonic to the front so that it's also not bumping to the front And they come with a different battery And the four work model has Better I would say more efficient Power settings so the eco mode is more eco and the power mode is more power And I feel in the needle. It's just I think it's 50 versus 60 percent of this of the maximum power and I think in in In the forework it was a bigger difference like 50 to 75 or something So you really have a powerful mode and an eco mode And I also feel that those those needle models are a bit louder due to that So when I was in the four work store and they showed it to me They were just running it an echo and that was more silent Yeah, so there is some differences. There's a reason why the four work is more expensive. I would say it also felt Better somehow it has a green brush obviously Yeah So do you think far back did their own research and just bought the IP of an existing model to make a better robot Or did they buy the whole thing and just rebranded as far back? So I think they I mean they run their own cloud. So it's the same cloud, but with different names And they acquired Nito some some years ago 2017 and before I guess they also did Development together. Okay. Thank you So the next step would be to build an alternative firmware Did you do any steps in that direction? Is it possible to disable the secure boot and put something else or? Say with jpeg or something or is this completely out of question? Yeah, so the problem is that the serial boot is not that fast and so on so we were thinking of one alternative so there's So the fun part is why we were stuck stuck in communicating with Nito There were some other heck who also did something with the Nito robots in an older version and there was a buck in In some NTP time setting command where you could inject Something during runtime and that would be better So you boot the system and then do the modification over this vulnerability and that would be Probably doable with your Raspberry Pi in a local network So that that would be a variant I think so Especially for those where you need to to open the casing and everything to attach cables. That's not really something you would do Yeah, yeah, I don't know But I think it's more convenient to have just a Raspberry Pi and a certain network config and then the robot is exploited locally But that exploit only works on all the versions also not over the cloud and so on yeah How did you get the contact of the Nito developers finally and what did they say exactly? Beside the support just closing the tickets so That at some point so when we got that call this developer Software lead he already had seen our MMR MCD talk So someone said him oh it seems to be something serious There has been a talk and he was well informed and contacted us in the end And what was the reaction afterwards? Was he happy or thankful or I was say they were overall They were a bit confused like why we chose their system and so on. I mean, it's interesting. They have interesting technology, right? and also our Like at least in Germany is something you would buy They were at least happy enough to send us one model for testing We needed more for testing and I think we are like at the border of what Amazon business allows you to buy and send back Yeah So there's Yeah, I don't know so so our secretary really hates me now So whenever I say yeah, I it might be don't don't pay it yet We might send it back. She has always this this face and then she says like vacuum cleaner Um, yeah, thank you. Thank you for listening