 Welcome to my talk on smart meters. I'm hacking infrastructure and so should you in This 25 minute talk. I'm gonna walk you through all the different things. I've been doing the last couple years to reverse engineer smart meters Particularly the ones made by Landis and gear that are used here in Texas by the power provider on core I picked them strictly because they're the one that was on the side of my house And I was interested in learning more about them So nothing in particular about those two were the focus of anything other than it's the meter I have and the power company I use You'll see all kinds of circuit board reverse engineering Protocol analysis Some software reverse engineering using binary ninja that I've kind of just begun recently There's a number of YouTube videos. I also have on this that kind of was the precursor to this talk and a wiki Called a rich s them. And so there's a lot more in-depth data and in-depth protocol stuff There's a discord channel that I have so a bunch of ways if you're interested in getting involved in this that you can As a little background, I've been into reverse engineering a long time And so I've watched a lot of talks online by different people Chris Tarnofsky reverse engineering silicon It's real fascinating to me Chris Gerlinsky and the work he did reverse engineering cable TV pay TV box That was fascinating and Travis Goodspeed obviously a ton of work out there POC or GTFO Other things like that that he's done and and a talk that he gave that was really fascinating to me that was called in praise of junk hacking and And the premise of that talk if you don't watch it I suggest you do is that you know these critical infrastructure devices they all contain off the shelf components And so you know things like the processor that's used the transceiver chip That's a CC 1020 made by Texas Instruments that's in here Those are all used in other devices like the girl tech IMME that him and Michael Osman worked on And things like that. And so his premise was if you reverse engineer Things that kind of are in products that don't matter, but they're used in products that do matter You're essentially pointing out and finding flaws in critical infrastructure or things like that without You know without the undue attention that comes from a reverse engineering critical infrastructure And and that talk is a quite a few years old at this point and at the time I think that that's you know It was a really sound way to go about it and I think also if you just don't want to attract any attention That's fine as well But the the challenge is these devices run, you know kind of our whole economy and With the way ransomware is and and things like that now You know you used to have to steal something of value and then sell something of value and and that's how you made money You know you robbed a bank you got the money you hack into a place you get the credit cards What we see now with ransomware and pipeline hacking and and even hacking corporations is that you don't have to steal anything of value anymore You can just stop someone from operating and they will pay you a bunch of money To restore their operations and and so you know That's a big motivator for me now looking into these devices and publishing things openly so that we're not under NDAs There's nothing stopping us from having a conversation about these things all of these have been purchased legally off of eBay There's a whole market for this stuff out there And it's fairly cheap to pick it up and you'll see I bought a lot of this stuff You know as we pan around my office here This is kind of just a piece of it as well like there's even more devices that I have because I've been doing it for a number of years and it just doesn't cost that much to get This hardware and it's super fascinating to try to learn every single piece about it You're gonna see that there's a whole geographic routing protocol that's used with this it's a massive mesh network where all the data hops between meters and up to routers that I Happened to look like this And so we're gonna look at all this stuff I've been publishing it all on on YouTube and and on the wiki as well and doing a bunch of protocol analysis And so that's what we're gonna look at today for the next however many minutes are left at this point And so I hope you enjoy it. Thanks a lot. I encourage you if you're interested in this stuff as well Join the discord channel check out the wiki creating account Contribute stuff that you learn. I think it's a fascinating topic to look at It's a fascinating device and these massive networks of you know Millions of devices on some kind of a 900 megahertz mesh hopping network that you probably haven't heard anything about and with the advent of Software-defined radios like this hack RF with a porta pack or usrp b200s Even RTLS DRs. You can analyze this frequency hopping data and learn more about it And so that's what you're gonna see. That's what I've been doing. I hope you enjoy the rest of this Okay, so let's set the stage with some GPS coordinates plotted on a map These are all the smart meters that are between where I live in downtown Dallas as you drive up and down the freeway with an Antenna on the car what it can receive the height above the ground is how long they've been running since a last-loss power So you can see in this there's a meter ID and it's been running for 1,767 days Now what made me think that the meters would even have GPS coordinates it comes from this paper here that I read I found on The IEEE and it was written by some of the people that work at Landis and gear and it really outlines the entire system and how it functions So it gave a really good framework as I began digging into these meters to try to understand How do they even work and this right here is the big part? Says there's a geographic routing protocol and that really struck you know that piqued my interest like a geographic routing protocol What does that even mean and so as you read it? There's a piece here I'll look over to the side and and read what it says says the RF mesh system supports peer-to-peer Communication by employing a routing scheme which utilizes the geographical coordinates latitude and longitude of the communicating nodes By nodes they mean smart meters So you can see here. It shows all the beaters. They send their messages They relay in between them so it really forms a large mesh network But then it has to get off of that mesh and it goes up to what they call a router Like the device I showed earlier. That's actually a collector We'll get into that in a little bit then it goes to what they consider kind of a traditional collector This is a device sitting inside of a substation usually with a big pole and two to four antennas on it Now I just want to give you a little bit of a framework of how the packets are constructed. These are called five five packets They're basically broadcast once a minute and they contain some kind of universal information They have what you call a WAN ID Which in this case turns out to be the GPS coordinates that I'll show in a moment here And then the LAN ID which is the ID on the front of the meter and some other information that comes across that we've been working on decoding Now hunting online a bit search in every single document. I can find which there's not a lot It's mostly on the FCC's website that you find this stuff There was some really good manuals that gave some information about how to configure the devices and what they showed was people Inputting GPS coordinates and something called a color code in the north south and this kind of information And then the output would be this other string of hex values that looks a lot like the WAN ID The more I started searching around so then the hunt was on to figure out How do I convert this input data the GPS coordinates to that output data so then I can go back the other direction So I found eight unique values and I organize them put them in a nice little list You see here and organize them by the latitudes from smallest to largest And the idea would be to try to look at the data on the right hand side the output and Understand, you know, if I'm looking at that data, does it correlate at all with the smallest to largest? You know, what what can I find about it and it stumped me for quite a while But like all things I found with these if you just keep breaking things down from hex to the individual bits And then look at the bits things start to become much clearer And so what we can see here is that I didn't know much but from reading I knew that this idea of a color code that I put a C in front of here that color code was 31 32 unique values so five bits basically and on the right hand side of this You see that everything's zero except for one bit that set there me too And so I knew probably the last five bits are going to be the color code So I can kind of strip them off So now is a matter of just looking at the rest of the data and then looking at the pictures that were from the manual To see okay, if I take a bit for the north south indicator a bit for the east west What does that leave over for the latitude and longitude and then try to convert those values to see okay? Do they come back to the latitude and longitude values that I see there? And how do I do that conversion? So it came down to a little bit of math that I boiled down to a formula right here There's a much longer video I have on YouTube about decoding the GPS coordinates So if you're interested in exactly how I figured all this stuff out Feel free to watch it But otherwise know that the GPS coordinates are embedded in the meters They're transmitted once a minute and this is how you decode it and all this data is available on Github in a block That I wrote for the new radio So this is the full GNU radio flow graph that you see here And I can't show you this without mentioning Jacob Gilbert who did a great presentation a little while back at the GNU radio Conference and he shows this whole framework this frequency hopping spread spectrum toolkit That they wrote that basically grabs all the individual transmissions Packetize them and what they called a PDU and then sends it on and so it's using this framework that I was able to grab all these Frequency hopping packets send them into this block that I wrote and then output the data That ultimately gets sent over to a Python script that decodes and dumps the GPS coordinates So what you see here is GNU radio running. It's grabbing packets receiving them I dump it on this terminal on the left That's the debug output showing the raw packets and then I start up the Python script And I basically have it point to a TCP server that's running inside of GNU radio in one of the blocks It's there and it grabs that data that's coming across any of the five five packets that contain that GPS information And then it dumps it and then ultimately I take it to a CSV file and send it into Google Earth Which is how it was visualized So let's switch gears and take a look at this collector I'd own the thing for a while, but I literally hadn't even opened it up I got it I set it on a shelf because I was so busy playing with meters and someone tweeted that inside there might be an SSD drive and You know and so let's take a look inside and see what's there I have a bit of a longer video that really dives into this and as a reminder This is one of the devices It's like one that would be mounted up on a pole that's getting all the data back And so inside you see there's a really large battery that you know The thing uses when the power goes out so that it can keep relaying messages back from the power meters They can also send messages for you know 10 15 seconds after power goes out And so as I pulled everything apart you see there's one of the radio modem boards and then underneath this panel right here Yeah, it's an SSD So at this point I'm thinking jackpot and So I pull everything else out. Yeah, I want to do a full tear down There's pictures on the wiki of all this stuff that you can really see detailed shots of the small single board computer They use it's Intel based and all the other components And this is really the backplane board that I'm taking out It has all the power supply components and also a cellular modem that's inside of this device That they used to relay the information back to the power company So I took an image of that SSD and I mounted it in Linux and what you see here is the The thing runs Windows if you can friggin believe that Windows embedded 7 the whole file systems there There's a folder called collector conveniently that has all the files and everything in it And it's all written in .net and it doesn't even appear to be obfuscated in any way I also was able to take the same image Loaded into a virtual machine and boot it up and run it and so here you can see that there's the cellular modem that it's looking for It's not connected You can also see the terminal window showing the version of Windows that that it's running now after I let it sit and run for About ten minutes. I'd walked away to you know get some coffee or something I come back and this program is running It only seems to run if there's no serial ports attached the minute it can actually talk to a serial port It just shuts down and reboots constantly trying to find probably the modem or the the cellular modem or the radio That's inside this device and it can't find either of them So let's pause on the collector here for a little bit and switch back over to the meters themselves and take a look at some of The reverse engineering that I've been doing there early on when I started looking at them I realized I needed to create some schematics of these so I have some kind of reference to understand where everything's connected And how it's connected and where I can supply a power so I don't have to run them all off of 240 volts So here you can see some of the schematics that I've created of the different processors the M16C and the Teridian If we back out what I did a bit is I bought a ton of these boards on eBay I found someone that was selling a bunch of them that just weren't in meters and I stripped off all the components on one So I'd have something to have as a reference I also took advantage of having all these different meters and boards I've purchased to dump out all of the ROM chips So that I could compare all the different files and see you know, is there any settings that we have? What's different between them? What can I learn about how these things function by comparing a bunch of different data files now? I also started looking at these integrated Wayne gave radios and There's a bunch of different versions of them But this old one is all through-hole components And so it's much easier to analyze and just literally read the firmware out And there was these jumpers on it that you could set that would put into a test mode and in that test mode I could see a menu that would show the status of the jumpers and so on the meter itself I hadn't found a way to enter this test mode but someone else had dumped a memory chip or tried and Erased it on accident and what they found was that when it booted up It would actually dump you into this same kind of test mode menu So I did the same thing and inside there I saw that it would show the status of the jumpers and so going around this chip with a resistor and trying to Set some pins low. I found the pin 89 on this m16c Would actually put it into test mode without having to erase the flash memory Now there's a bunch of other cool menu entries in here that let you do things like readout different parts of the memory and Test a bunch of things but there's one that I particularly like that lets you receive RF data From a network around it and so you have to specify the CRC seed Which I was able to figure out for the on-course system around me and some other information like pick a channel but when you do that then it receives some data and I don't care so much about the receiving of the data because I already built a new radio thing that that does that But what's more interesting is how they organize the data on the screen and even what is in the test menus Like what are the things that they care about and so that's much more interesting to me to see the different types of things They show and what they're concerned about then the actual data itself, which I can just read directly out of these chips Now also a while back I managed to dump the firmware out of one of the meters that I have and so I have this m16c Firmware I loaded into binary ninja and it was a large file and reverse engineering assembly isn't my Full-time job you could say but I also managed to get the bootloader out And the bootloader is only 4k and it's much easier to understand And so I've been reverse engineering that a bit to understand how they Load the key from memory that they use to determine whether you access the chip or not and to potentially look for flaws Inside of the bootloader itself So showing you a bit of what I've been working on up until now Let's talk a little bit about what I'm working on in the future and what would also be interesting to figure out So you've seen that they use a frequency hopping pattern in the 900 megahertz band What would be real interesting is to figure out well How are they hopping around and how did they choose what frequency to hop to next every 700 milliseconds? They hop to another frequency and I believe it's partially based on the ID of the meter and Some other information and they discover each other even though they're all hopping around on these different frequencies So how they hop around and what frequency they choose and what time would help us lower the cost of the hardware That's needed to analyze and receive all the messages on the network So far everything you've seen is me receiving data from the meters and trying to figure it out But what would be real interesting is using the ferrity cage Transmitting back to the meters trying to get them to accept packets as valid and see how they respond So not necessarily just random fuzzing but actually crafting valid packets with check sums that are correct and also the timing data and this other information and and really sending it a packet that it believes is coming from the network and See how it responds and then change things about that packet while it's still signed properly and See how does the meter react to that? Also, some of the messages that are seen on the network start with a d5 I only showed you the 5 5 packets the d5 appear to be a routing protocol for sending messages across Where the meter identifies itself and then identifies the address that it wants the message to go to across the network? what if we could craft our own packets and send them on to this network and Travel across it to another device without ever actually going to the router and to the back end You know could someone create a message and a network of devices that ride on this mesh network without actually Causing havoc or or doing anything so benignly Traversing this network without actually leaving the local area up to a router and back to the head end That's interesting as a thought experiment Also, just the process of discovery so while looking at the different Iwr radios the integrated Wang gate radios I noticed that I actually had five different or four different generations of these radios I had second third fourth and fifth generation radios and by analyzing the data at sending What I noticed is the second third and fourth generation all use the same sync word that it sends initially so that other Meters receive that message and process it but in the fifth generation They change the sync word probably so they can add additional features or to change the protocol And the rest of the message still seem to be craft the same on the meters around here But the local meters would ignore it since the sink by the sink violates the Start and stop bits that are used traditionally by the previous generations So that tells us that in the fifth generation Fundamentally something changed. They added some new additional functionality. That'll be worth digging into Also, I didn't talk about this thing much, but inside every single meter is the ability to shut off power That's right. They can send a wireless message to any single meter and shut off power to that location Could we do the same thing you can also control it through the infrared port I've seen it in the menu items that show up when I'm logged into the menu on the meter itself I haven't tried playing with that yet, but I will in the future But also on the RF side what kind of packet comes across that turns power on and off I've been capturing a lot of data. Perhaps it's already in there and I've just missed it And it's waiting to be discovered and finally like I said all this information the GNU radio block It's on github. You can see up here. It's also documented on the wiki The wiki is always a couple steps behind what I'm discovering just because Documenting everything and discovering it and creating videos all takes a fair bit of time But I do update the wiki and anyone is can have an account I encourage you to create an account and anything you find or any other meters you work on you can publish the information there as well And finally links to all these things are off of they were Chessum landing page You see it right here at the bottom. You can click to go to YouTube discord smart meter data And any other links that are an importance. I'll include here. It's an easy place to find