 Thanks for joining us here at the wireless village. My name is Barlent Sieber, I'm the director of vulnerability research at Bastille and I'd like to talk a little bit about my research into emergency warning sirens. This research is called Siren Jack and I hope you enjoy it. If anybody has any questions feel free to shout out. So as an overview I'm going to talk about these systems and sirens in general and talk about the two phases of the research finding the signal and analyzing the signal and then the disclosure process that we went through with the vendor and suggestions to other vendors and other folks engaging in this kind of security research in the future. Emergency warning systems, put your hand up if you've ever heard a siren go off that's part of some sort of, great. So as you may know then there are mass notifications to the public, they're used for tornadoes, tsunamis, flooding, et cetera and other incidents that might occur where people of a large area need to be notified where other methods like mass texting or the like fail. So the other alternatives that are in common use are sending an alert to everyone. Unfortunately we had a very bad example of how that didn't work out in Hawaii in the very recent past and also there's EAS, the emergency alert system that was originally created so that the president could address the entire population I believe within 10 minutes. But sirens are also another great way to do it, they're independent infrastructure, they can be off the grid so to speak and controlled remotely and so they're a very popular choice today. They date back to World War II, the civil defense or air raid sirens like that one in the top right but nowadays you mainly see the electronic kind which is in the top bottom left, sorry bottom right and then the center one there are the oldest sort of mechanical siren that would rotate and produce a very loud sound. And if you're all interested in going deep into the kind of technology the one stop place you need to visit online is called the siren board, air raid sirens.net. You know there's a subculture for everything, this is it for emergency warning sirens, anything you could possibly imagine is discussed on this forum. And some of the big players in this market are these folks here and this entire line of research became motivated by being in San Francisco. Hands up who knows about San Francisco siren system and the Tuesday noon test ritual. Couple of you, yeah. So I've got my laptop plugged in, hopefully we'll hear some audio, no audio. Excuse me, anybody from the wireless village, could you help me get audio out of my laptop? I plugged in the 3.5 mil thing, do I need to switch something on or do anything here? I plugged it in and I got my volume right up. Maybe, let me check if it's going through HDMI. No, it's still the right place. Oh master is on sound flower, that's the problem. There we go. That's weird. Thank you. All right, let's try that one more time. Thank you for the help. There we go. So this is the sound of the San Francisco Tuesday noon ritual where they test the entire system, this dates back to World War II. And if you move to the city and you haven't heard this before, it's quite an experience. You don't know what's going on. But then you have the reassuring voice that it's on the attest. So this took me by surprise when I moved and I was wondering how does it work and I noticed that on my rides to work there would be these poles with the sirens on top and I also noticed there were radio antennas so I thought maybe they're controlled by radio. And if you looked at one part of the city in particular, you'd see the Google map satellite sort of footprint marker of these sirens. You could see four of them sticking out there. And I would see them on the poles and they would have an antenna or a control box, the horns at the top. And I thought, well, a security researcher in me was wondering I wonder whether this system is actually secure because this kind of emergency warning system needs the public to believe that it will fire in a legitimate circumstance and not go off with a false alarm. So I took some photos. You know, the vendor left the brand sticker on the control box. So I looked them up. Interestingly, they've just revamped the website a little bit. It used to look like that. That was a nice high-res image but with the way back machine, I guess they still low-res versions. So they spruced it up recently. And I started looking at the spec sheets. And these highlighted sentences really stood out to me. So it's saying that they use FSK, DTMF or two-tone sequential data signaling. They have optional upgrades for digital and trunked radio networks. And it says there in the spec table that the signaling method is encrypted FSK, DTMF or two-tone sequential. So I was thinking, well, if they have FSK, then they're probably going to be using some protocol and if it's encrypted, then that's great. But of course, as a researcher, you want to verify that's the case. So this kind of turned into an epic fox hunt to use the ham parlance. So you have to remember that I don't have access to the system and I'm starting out from scratch. So I know nothing about it. So where do you start? Well, you start by collecting open source intelligence and also looking at the radio spectrum unfortunately once a week because the tester goes off once a week and that restricts the window and the interval at which you can look at it. So that causes things to take longer as you peel back the layers. But the system consists of these siren nodes on the poles, these two various models. They have a central controller that talks to them either directly, can talk to them directly or via a repeater network. And I also found a nice informational video online, actually on the vendor's website where they came out and did a sort of piece about the system in San Francisco. So I had some choice frames. It looked like it was being run or managed by the San Francisco Department of Emergency Management. And in the video, you know, they had these pictures of where the sirens are located in the city. They actually for a very short period in the video follow a tech there and they show you the inside of a box. So I noticed, oh look, there's a Motorola radio in the top left. Looks like a conventional radio but not quite sure what model it is. And then elsewhere you can see the kind of control units they use. And if you remember the picture of the one on the bottom, we'll see that a little bit later on. But looking at another photo that they have online, you could look a little bit closer and these were CM 200 radios. And interestingly, if you look at the spec sheet, they operate in two distinct bands for the two models. And so that sort of helps you, you know, restrict your search to still a large portion of the radio spectrum but perhaps a little bit less. The thing is that I was thinking to myself, this is a public infrastructure system, critical infrastructure. And San Francisco has existing trunked radio networks that serves the large coverage area. So I thought, well, you know, if you're going to use that, you're probably going to use an existing network to get the footprint you need. And so it might not be neither of these ranges. It might actually use the public safety networks that have already been established that are used by first responders in the city. So you can go on RadioRevents.com. They have very detailed lists of all of the networks in place there. The two public safety ones there are the Motorola Type 2 smart zone, analog trunk network as well as the P25 digital trunk network. And then you can also look up and see all the frequencies that they have allocated for all the channels there. And if you look at where the sites are, they put them in strategic locations to have a good footprint. The FCC, as you may know, has the universal licensing system. So you can perform a variety of searches there to try to find information related to this. And so I did a bit of searching and figured it was probably going to be one of more frequencies licensed to San Francisco City and County of. If you look at these records, they have the radio service field. So it was probably going to be something in the public safety category. And they also have station classes. And if you look that up on the wiki on radio reference, which is very helpful, that explains what all these station classes are. So that might give you a hint as to whether certain licenses and frequencies are maybe what we want to look out for. They also have admission designators, which are these codes that give you some form of description of the bandwidth and content of the signal. And there are a whole list of them. But again, with all this kind of open source intelligence, it quite often turns out to be inaccurate or just plain wrong. So you got to treat all this with a grain of salt. And also control points. So there are specific addresses where radio communications on these license frequencies can take place. So maybe that has something to do with this address is I think right next door at the Department of Emergency Management. So, you know, all these sort of candidates, but still a lot of combinations to look at. So again, you know, more addresses. The Department of Emergency Management has a nice public facing informational page about the system. You can download the map that shows you where all of the nodes are in the city approximately. And they've had this share of false alarms too. There was a malfunction where a couple of sirens went off in the middle of the night. And they issued a blog post about it. And then apparently looked and figured out what was wrong and fixed that up. So I said about working to find the signal on the radio spectrum, use my antennas in the then attic and hook them up to a variety of various research USRPs. SDRs have started doing captures of the radio spectrum at around about Tuesday, a midday on every Tuesday. And so this was a little over two years ago when I moved to the city and I was mulling all this over. So if you look on select slides, I've sort of got a date stamp in the top right. So this was in 2015. They did an initial capture here. And I weren't going, I'm assuming you all know how to read a waterfall, but in this case, time is top to bottom. And frequency obviously on the X axis. This is centered at 850 megahertz. And here the Motorola SmartZone trunk network, conventional analog channels are all happening on the right, plus other traffic is interleaved there. So I thought, well, maybe that's the first place to look and just get to know the radio landscape. I looked at also the P25 digital trunk network and you can see all the traffic channels happening in there. If you zoom in, it's kind of neat because you can then see the individual P25 transmissions coming out of the repeater there. And then interestingly around this time, coincidentally, the Dallas siren attack happened. Who remembered Dallas, the Dallas siren attack? It was 156 sirens went off in the middle of the night. And that was purported to be a replay attack because they were using an older system that was controlled with fixed DTMF tones. So I'm sure you're all familiar with DTMF tones on a keypad. If you have a baffling like this, you can just key up and press the buttons. So if somebody had been listening to the tests that they had been conducting, I think it was on a monthly basis there, you could either decode the tones and play them back on radio or you could just record them on a tape recorder or your laptop and then play them back over the air. And that's what people guess actually occurred. And just shortly after that, ATI, the vendor released this statement, sorry, not a statement, it was a press release entitled, is my emergency notification system safe from hacking? And in that, this sentence caught my eye, which was many older systems include few, if any, of the additional security features ATI can provide. So DTMF is obviously not secure. Sounds like they are actually doing something fancy. All ATI command packets, including those sent over the radio, that would set the sirens off, are protected by several security features, including encryption with AES. So I thought, wow, okay, well I might be able to find a signal if it's correlated in time, but if it's AES, then it'll look pretty random and I won't be able to find anything. But keep going. So did some more captures, focused on that public safety trunk network. If you zoom in there, it's kind of neat because with a good antenna and filtering on the output of the repeater, you can get really nice strong signals. But as you might imagine, recording this at a high bandwidth to RAM disk takes up a lot of space and there are also probably hundreds and hundreds of individual transmissions that you have to read through. I mean, this might only be a single packet that appears on the air. I had no idea. So that meant I had to go through every single one of them. So that took a long time and I developed some tools that would automatically detect when a frequency had a transmission on it and then isolate it and then extract a separate file. So then I just basically had hundreds of WAV files and I would just open, there's a quick look in Finder on my Mac and just scroll down and listen to the content of each one just to see. And it was just mostly on this analog system, people talking about their business. I just remember that there were times when somebody would key up on there and just talk about something completely unrelated to the siren system at midday, but I would actually hear their radio microphone picking up the sirens going off in the background as well, which is kind of funny. But every week then I would sort of keep a log and make a log of interesting signals that I heard. There was nothing that repeated week to week, nothing compelling, no obvious candidates. I turned my again to the P25 network and in a similar fashion with the individual recordings I used OP25, the open source decoder, hooked that up and had it decode all the channels that were available and listened to them as well. And I had this sort of nice plotting feature where it'll actually extract the timing of the various frames that go through a P25 transmission and then color code the waterfall so I could see what was a trunk and control channel, what was a data transmission, what was a voice transmission there. And then I focused on the 400 to 500 megahertz band. These captures here are 50 mega hertz wide so I was using a USEP B210, streaming it to RAM disk for about a minute. And as you can see there's a whole lot of traffic there. So even more signals to look at and potentially find. And again I'm trying to find something that correlates with the noon event. So if you zoom in on that and look at the individual signals they're very narrow compared to the whole bandwidth a lot of things to analyze there. Look further down in the 400 and what's an important point to make is if you really want to get serious you need to have some good RO filters. So I want you to compare this picture in which there's a lot of out-of-band interference so I think it's some LTE downlink transmissions from a nearby cell tower. This is going to be the same capture but with filters so that to that big difference. Highly encouraged and as you can see there are the strong signals being transmitted but again that was uncorrelated and not repeating with the same timing from week to week. I looked online found some more videos this was example of an older control unit. Just you know little clues to pick up about how the system might work. They have this portable box apparently San Francisco also has one so if the building gets destroyed in an earthquake they can take this portable unit and speak through it and address the population. Tell them where to go. Frames from the video you know showed dialogues in the software that you know would command the system do something and then they film the screen so you could see bits and pieces of the software and this would suggest that the different sirenodes might have different modes or status indicators. So I did some more surveying in the city and went up to some of the hills and found other poles there and I was interested in now the length of the antenna because that might give you a hint as to what frequency band to look into. So this is an omnidirectional antenna but the thing about these is that you know if you're designing antenna you know the frequency and therefore the wavelength that you want to tune to you pick some division of that and you know you might take a guess a minute about how long that is but that might be a co-linear antenna which is a number of stacked elements any number of stacked elements so it could be a completely different unrelated frequency to what you just guessed by looking at it so that was tough too but I thought well you know give it a go maybe there are catalogues online where I could compare the dimensions of that so you know you're not going to climb up a pole so you have to try and figure out creative ways to guesstimate the length so I pulled my bike up next to the pole I tried to measure the width of the shadow that the pole was casting on the ground use that to determine the width of the actual pole and relate that to the number of pixels and then go up and extrapolate that as your you know pixel to meters ratio to figure out how long that I'll mean direction like tenors in various photos the problem that you learn quickly is that camera lenses obviously have intrinsic I think lens parameters that give you distortion effects and so you know the pixel ratios relative to your measurement wherever that was in the image will be different at different you know extents of the the image so I did this a number of times it would just keep getting different results so that was frustrating and then you know various awkward moments of trying to actually film or take photos of the whole the whole whole thing and look because you want to look in inconspicuous as well right so just being a little careful about that so it's a more captures now in in the lower VHF band 155 megahertz is a strong TV carrier to the right there this in this capture I you know I had three or four computers that I was using capture with a bunch of usurps and managing all that just getting them all started in time not having you know various issues crop up meant sometimes I started early sometimes started late and you don't know how early they actually send the commander set them off for what happens afterward and this one I started you know early another week so then I looked online at email listings Amazon listings various other photos to try and find antennas that look like the ones that were deployed because they had stickers on them at one point but they'd worn off in the weather so I found some candidates and I found some candidates in this catalog but looking at the various combinations of dimensions in frequency it made it even more confusing as to which one might be on the pole so then I went absolutely nuts on Street View and went through the city and found every single one I could find from different angles to try and get some decent average measure of potential candidate links and still that resulted in ambiguous candidates so I've started mapping them out trying to remember so I wouldn't duplicate my efforts as I would go around and you know the spread all over the city and I'd seen this informational video before this this point I'm sort of really not sure what to do some not seeing anything obvious again some choice frames inside the department of emergency management you got the software they're running with the the sirens on the map close up of that and then this dialogue with what that they used to set it off I think it's as noon Tuesday there in the text box remember this was the control unit we saw earlier so presumably this is the control unit they have there and then I remember lying in bed at 2 a.m. looking at this video not knowing what the next move was going to be and I saw this frame shot for just a second it was just a very quick transition that they made and I was the first person who's going to be the first person that tells me what the clue is ready that's right it's not on the directional antenna in this one it's a Yagi antenna Yagi antenna is a great because they show you the direction of the transmitter and they also have far more visually characteristic features that can help you identify the antenna so that's there in that in the in the just a very bottom of the frame so the question was you know great they took a photo of it where is it in the city there are you know over a hundred odd sirens and I hadn't seen any on Street View yet so I kept searching and I found it and it actually happened to be one up from one of the ones that I was looking at on the on the ocean beach west side of the city and there it was so I went down with my camera and pretend to be you know tourists taking photos oh look ocean beach on a lovely day these people having fun on top of a San Jun a big tourist ship leaving the port oh look a Yagi antenna on a pole so indeed all the things you can tell here three elements you got the differing widths of the elements and on the driven element in the middle you've got that that thicker piece that comes off the the main arm and then also there's a little sticker that might tell you information about the manufacturer I couldn't couldn't quite see it was on the sticker and on the way home I took some more pictures of omnis just in case but using the compass bearing these were the rays that I thought were the likely direction and that kind of made sense because Twin Peaks is in the middle of the city so it's an ideal point to broadcast out everywhere and if you draw the lines together then it kind of you know nicely fits and this was in Google Earth when I pan the camera down it makes sense that this was the only Yagi that I had seen because there's a bit of a hill in the way between the transmitter the tops of the transmitter towers there at the bottom and ocean beach which is on the other side so they needed a Yagi antenna to give them a little bit more gain to get the signal and those two towers of the two small ones they're not you know just just offset from Sutro Tower so one of them has has the transmitter on there to repeat it so then great got a picture of the Yagi what do you do now search through every single antenna catalog you can find from manufacturers that make Yagi antennas and eventually our family found the candidates there if you put in the picture fits nicely and what's what's good is it now we have a very narrow portion of the spectrum compared to before where we have to look so this is obviously VHF now and it turns out I went back through the captures that I'd made and I'd already captured the event and I think there was just so many things that it was easy to overlook and not fine but this was here and then I went back to another week this was the one where I am one of them started early one of them started late and again this was I this one I started early and so I just caught caught you know the activity the very end of the capture but this is really zoomed in so it's masked by all the other signals there and what was interesting is if you look closer you can tell it's a repeater system from the captures because the output frequencies on the right hand side there the input frequencies on the left hand side and if you look at the transmissions they vary on the input side in signal strength because you know I'm at a static location and there'll be each of the siren nodes as it turned out transmitting from closer or further away but then the repeater that's repeating the transmissions from the central controller is always in the same location so you've got the the first transmission there from the controller the second one from the controller and then you've got one from the control and they're a much stronger response so a siren close by and then the controller another one close by another one from the controller and then one further away because it's weak another one from the controller and then I didn't even get the next one because it might be across the other side of the city or it might not be working and then neat thing about these repeaters is that if you listen for long enough by law they're required to transmit their IDENT in Morse code which is the call sign which you can then look up in the FCC database so this was great I found the frequency and then every Tuesday at noon this would be my ritual so I was also at the bow thing there and since this is a conventional narrow band FM network you can just use a handheld to pick this up so there are transmissions and then get that in the background and then I went out to actually be there under underneath one and if you're nearby it's you hear it's loud so it gets loud so the next phase analysis every Tuesday noon I do my captures and then this is what you know the the home lab would look like I had some oops some good new radio flow graph running there and this is obviously further along but once I understood the packet format it would just dump it to the screen I could watch the status of what was going on be recording and on my other machine some of you may be familiar with my new radio flow graphs if you turn the disable blocks you know that they're visibly offered looks a little bit cleaner but this was a implemented an optimal non-coherent 2FSK decoded with no clock recovery because in this case the clock drift was far slower than the total length of any particular frame which simplifies your decoded design looking at the physical layer then this is the baseband waterfall and obviously it was audio frequency shift king over FM and you'd see at the at the header of the frame there was some sort of sync or preamble a payload some post-amble or filler and then the CTCSS tone that's left over to be able to get into the repeater and then just the tail on the repeater so if you FMD mod that then you could very clearly see the two tones there this is already once of the band parcel that has been applied over those two tones and then again you have that same packet structure and interestingly if you look at the very faint line I don't know if you can see the very faint line on the left side of the waterfall that's just the CTCSS tone that's active and on the entire time during the transmission so the repeater stays keyed up now given these parameters this is what it looks like as the waterfall runs and if you use a different transform size and a bit of windowing then you can see the individual levels they're jumping back and forth in the demodulated waterfall now before I continue given the sound and what you've seen so far does this sound like maybe some other popular AFS-K protocol that lots of people like to use? APRS so this is not APRS or AX-25 but it sounds awfully similar but if you take a sample out we need to figure out the board rate and deviation if you slice around the zero point you can very clearly recover your ones and zeros whether your energy is above or below and then if you do some simple cyclostaturing analysis which is multiplying the signal by a lag version of itself and taking the FFT you can find that the transmission is at 1200 board all the things you need to know to decode so this is the Bell 202 modem standard that's also used by APRS so we have the physical layer now so once we extract the bits and do some slicing then interestingly we have some structure there but if you look carefully it's not quite lined up between transmissions there's a bit of change in alignment and you know that it's not clock recovery the issue with clock recovery because I'm not using any clock recovery but if you're playing your text editor a little bit it becomes clear that it looks like it's really just a start bit so you've got eight bits to make a byte and then you've got some number of you know idle zeros and then you have your start bit and then the next byte begins this looks like a serial line protocol and why would they have that well they probably have some time during which buffers need to get filled between the controller and the modem chip and that's just the idle time and you know it is what it is and you just have to deal with the finding that start bit I thought maybe you know it might be some other subtle level of security so I kept that information and once you turn those bits into hex then again clearly some structure comes out and this is of course a little bit worrying because if you expect an encrypted signal you expect to see a very high level of entropy seeing this kind of repeating structure was a little concerning now with the packet format I noticed that there were these long strings of ones or long strings of f's as you can see there usually that's a bad sign it's usually a sign that you need to invert your bits so when you invert your bits then things look a little bit better and these are actually the three transmissions before the siren would go off on Tuesday and what was reassuring yeah so these are the non inverted three before they go off and then when you invert them then things look better and I'm not there yet I'm jumping ahead we inverted yeah so it yeah so we've inverted things are looking better long strings of zeros you know a couple of bytes changing here in there and another week I compared the same three packets and what was even more concerning was that the packets were largely the same so the other thing I noticed is approximately every 20 minutes a packet would get sent out and so I was wondering maybe some sort of keep alive mechanism going on there some periodic announcement and what was curious here was that the periodic announcement didn't have a matching transmission on the repeater input so that meant there was some box at the repeater that was doing these announcements so with the packet format you know that happened every 20 minutes and then I sort of kept looking at a lot of data and figuring out what fields stayed the same what bytes changed for the announcements and what I call the trigger packets but things didn't still you know look like they were incrementing or anything obvious so that well hang on you can make a byte two ways you can either pack it from the left or pack it from the right so I thought well let's change the bit ordering there and voila you got those incrementing digits now in the three transmissions before the sirens go off that's pretty compelling and I was guessing this earlier some sort of a checks I'm on there because I saw the last byte changing and if you look there the last byte changes and that also increments by one so before I even noticed that I thought well I will try a revang which is a neat tool that can brute force CRCs probably CRC8 because it's a single byte no hits what do you do when you get no hits you go back to basics you add up all the bytes in the frame you mod it with 256 and voila that was what it was so looking at more data then especially the timestamp stuff I looked at various you know collections of bytes there and it became apparent how things were being encoded and this was very clearly a timestamp so you could see which column of bytes is being used to encode the minute the hour the second and the day and you just you know record a long period of time and and look at these patterns and the the system time did not match my wall clocks the system was running on its own independent time but it was still proceeding with the rate of normal time so to speak so now this appeared to be a proprietary protocol maybe unintentionally security through obscurity you've got the normal features of a frame and then no sort of Mac layer acknowledgement or anything like that and then I mentioned the the time announcements already so very clearly there's there's no real security here right if the packets look largely the same each week and you figure out the the pattern that is behind the changing bytes being that the encoding of the time that's a you know a security concern what was also interesting is that after the sirens went off the controller would then ping or request you know some status update from every single siren node and then looking at the patterns there you could figure out which field was used to address which siren was interesting is being privy to this you could gotta get a sense of the state of the overall system which nodes were working in which ones weren't so as the status would check would happen you have the the trigger packets and then the green would be pinging a node and then the blue is the response from the node I don't know what the information means but you'd see that the numbers incrementing then you'd also see retries so that one there in yellow was that the expiring retry would try four times and it would go on to the next one so that node was obviously out of action so we've got a problem here no encryption no secure authentication you can extrapolate from the last time announcement and you malicious payload that has the activation sequence and your you know the time that would be correct at that point and capturing these trigger packets every week over a long period of time some other the final little detail there was revealed with the month they being encoded in some more because if you look from top to bottom you can see the month they the month incrementing as you as you go up and my decoder wasn't perfect by any means sometimes there'd be noise and what have you and it would fail to decode the packet so there are two issues one is obviously a failing CRC and it would detect that but also because the serial line protocol you need to detect when the start bit occurs and if you have a flipped bit in that idle and then the start you might start too soon but then the next bite that you get is wrong and then every other you know filler started after that is wrong as well so I tried to do this some tree search to go down all the different paths of where the appropriate if you if you flip the bit a potentially your only flip bit back then it would try you know all these paths until it might find a valid result but putting some bounds on it so it wouldn't take all day except for some of them so it would work mostly it wouldn't but what was kind of neat too is that I took took the waterfall on the output of my decoder and then I would plot on the waterfall the result of the decoding of each transmission of each packet there so I could very clearly see how my decoder was performing relative to the IQ that I'd recorded there was an unfortunate break in the pattern actually at one Tuesday they didn't go off so my waterfall there was left quite blank you can just see the time announcements happening there and this was the unfortunate day when we lost our married Lee and they didn't didn't didn't run the tests in honor of him that day but you know it's just it's a bit of a surprise obviously when you have so true into listening to this every Tuesday so at this point we thought well you know this is an issue and so at best we have disclosure policy responsible disclosure draws on industry standard processes and the the gist of it is that we speak to the vendor and then 90 days after that we inform the public so that gives time to the vendor to create some sort of remediation and roll that out create a patch and then we inform the public so that they can take whatever actions they might need to to predict themselves and we've used this in previous disclosures too such as mouse jack and the like so the timeline looks like this where we informed the vendor on the end of January we also informed the city there because this system was obviously on city premises with a view to public disclosure on April 10th and luckily by that time the vendor had actually created a patch and provided it to at least one customer being San Francisco but as we found it the initial ramp up there can be quite slow you know getting everybody's attention and entering constructive dialogue so we first sent emails to the vendor in the city and then we tried to call the vendor and we were directed to a new email address we sent more emails to the city we tried the department technology at the city via phone we tried the man's press office and then my former colleague Matt who is right there in the rear row he and I we went to the department of emergency management we actually did a sit-in there to try and get a hold of who we thought would be the appropriate contact and actually you know physically say hi and and you know give a hard copy there but we ended up leaving and just leaving a hard copy after that though we heard from him and and they and at this point also we're trying to figure out how to you know potentially contact ATI engineers to but they're in a different time zone so that made it a little bit difficult and then we received that first email from the DM and they told us that the department of technology manages the outdoor public warning system and so our email had been forwarded to them and I also tried to contact a previously associated employee that I'd found with publicly available info and their supervisor physically via email and phone to you know because he'd been using the system maybe he could provide us with the appropriate contact then finally in February we called ATI again and they said they couldn't guarantee that our call would be returned we sent some more emails to the city and then in February we had our first call and our first real dialogue discussing the vulnerability and they said they talked to their engineers on the next day we talked to the San Francisco Department of Tech and they had said that they were in touch with the vendor which is good and then in February we had a first real dialogue with the Department of Tech and they and then we tried to call ATI but all they they just informed us that they're working on it and then we received a letter from ATI a formal one we had another call another couple calls with the Department of Technology and they were hoping for a patch in early March and then we later we found out that they're making updates to the software and it might be delivered imminently which was you know all great news and then we requested in you know middle of March because the public disclosure was coming up a vendor response first via letter and then by email and finally we received a statement and we worked together to clarify that statement and improve it and then we had the final version in early April so some relevant excerpts from the statement are that they recommended using p25 radios that provide highly secure encrypted links and they also created a patch which adds additional security features to the command packets sent over the radio so you know that that was good that was what we were after and that they were in community continuous communication with their clients and meanwhile while this is going on we were wondering well is this specific to San Francisco or does it exist elsewhere so we looked at their website this is the older version of the website and we had a look at each of these places or potential customers to figure out whether it might be a viable place to visit and look at what kind of deployment they might have this is the new version of the website but again Sedgwick County is there in Kansas and they have a list of various other customers and they spread out all over the country if you look at the link one of the LinkedIn pages there it says that it's thousands of worldwide installations of ATI equipment are in operation today so just doing a bit more research online I found this in Sedgwick County news report after some of the sirens and malfunction there as well and what was interesting is that if you play the video and listen hmm that sounds familiar and also you know cameras have pretty decent zoom lenses so that well you know sounds a little bit let let's really do a little bit of analysis here I took the video sorry I took the the audio track out of the video did a little bit DSP and what do you know the same frame format is there you can very clearly see the three repeated payloads in the filler unfortunately I couldn't decode it because when you compress a video for the web it goes through an audio codec and that will obviously destroy your FSK but this is just looking at the response of one of the the mark or the space I don't recall which Sedgwick County also had a map of their sirens if you try and fight it now it's 404 I think I was on the web the news website actually but looking up that information that's in that video it's in the ULS if you look up the address there's a big honkin antenna there so they probably have a small transmitter up there and so I flew out there and knowing where the radio antenna was I stayed at a hotel and specifically asked for room that faced that antenna so that I could optimize my SNR and sure enough they had you know a bunch of these signs around the place with the same kind of antenna and you know you always want to be inconspicuous isn't especially if they're a big warning you know videos about science sirens around and they do weekly testing here so I I was there prepared in the morning and they have a different sounding siren it's a constant tone you can hear that and then I had you know a couple of laptops doing captures for redundancy they're just in case and unfortunately it looks like it was vulnerable as well there were slight differences here but overall it was the same packet structure same you know three bytes incrementing before the the test so I thought well we should inform them I actually called the guy and amazingly when you dial the number the guys responsible for the system is actually listed there so I try to get a contact in contact with them directly but as you might imagine cold calling somebody in this manner and just saying there's a problem with your system you don't usually get very far and so they requested that we send them a hard copy and we did very very quickly and so you know they were informed and and we let it found out that they were in touch with the vendor and and we respected their security posture on that and then also we managed to get some so it's not every day you have a delivery like this at your front door but obviously needed to keep the noise down both in terms of audio volume and RF volume and since these are license frequencies I wasn't going to transmit over the air so I've got a big honking attenuator to connect between the Motorola radio and my USRP and I've got a little 8 ohm speaker I thought well maybe the who knows it's a huge amplifier but maybe it senses there's a smaller load on there or something and it'll adjust itself and finally enough I did some tests and I've had two of these speakers and I hooked the second one up to another unit and I there are buttons on there you can press to do calibration and I the magic smoke came out of a speaker and I thought well you know that's not unexpected but it never came out of the first one so for the longest time I assumed that when I actually heard things coming out of this it was coming out of the speaker so for example when I actually managed to make it do my own thing I thought you know the speakers looked up great you know I can it's got a live PA mode so you can get on there and talk and your voice or music will come out of the speaker and in fact the audio was was being generated by what I believe are the the thermal changes in the transform or the capacitor or some passive components on the board because there's so much power going through the amplifier so what the speaker had burned up a long time ago I never even realized but it's just you know the components on the board were vibrating with the the you know a mission that was supposed to go out the out through the speakers so we did an outdoor proof of concept with the full rig just to test it out at a low volume and what was interesting is that toward the tail end of our disclosure window I would keep monitoring every Tuesday and then something utterly unexpected well I mean we were hoping for it but I wasn't sure how it would manifest but this is what I heard so they were going around and they were visiting all of the sireners which is great because we had imagined that if there's an update to the protocol the firmware has to be updated and flashed on the siren nodes so they would be aware of the new protocol and completely coincidentally a maid of mine was visiting from Australia and he happened to be walking through Union Square when they had the siren in Union Square open and they had you know I guess they were flashing the firmware or doing the test and so you know these transmissions I think maybe might come from the the home base and then they would listen on the radio there to make sure that the voice would come through and in those final few weeks interesting things started to happen on the waterfall green is a packet that is validated by my decoder with what I understood from the format red is a CRC failure but I knew that in this case the signal strength was good enough to get correct bits out of my decoder so this was actually now a different frame format and if you look at the text it's not kind of in triplicate so it's blurred the raw bytes and the frames now look rather random so that's a new protocol which is great if you zoomed in though this was during a transitionary period and the siren still kept going off so they would have had to transmit both the unencrypted and the new protocol and so the green ones are still the activation the trigger frames there and the red ones are the encrypted ones and then on the tenth I believe which is was the date of public disclosure you can see it's all red so it looks like the system upgrade is complete and through timing calculations I suppose they were the three trigger ones but they were the new format that looked pretty random we also worked with ICS cert and informed them of the vulnerability so they provided an advisory for that and just recently it was nice to see in the Wichita Eagle that all 150 sirens in Sedgwick County will be tested through Friday as part of software upgrade made for quote enhanced security end quote according to an email statement from the county emergency management so looks like they were rolling out the patch as well so some suggestions RF as we all know because preaching to the crowd here RF security needs to be designed from the ground up the tools SDRs open to software the cheap and accessible so anybody you know we here as security researchers and enthusiasts but also the bad actors can have access to this critical infrastructure that has radios in it has to be scrutinized and if you actually maintain or run or or any of this falls under your responsibility you need to have an obvious and secure way to receive vulnerabilities like you know all the big networking companies and what have you obviously the radio spectrum is a shared medium so if you've been communicating on a cable over copper and then you make the jump to the air using radio modems those radio modems might not be secure and so anything you've been transmitting is now free to grab from the air security through obscurity is no longer definitely not viable anymore and if you are a researcher then you need to come up with and strictly adhere to a robust responsible disclosure process thank you for your attention if you want more information you can visit the website or the ICS advisory just quick thank you to Jay Branscom he was my boots on the ground on the east coast helping me collect some information over there and I also want to shout out to Neil Pandey and Nate Temple where are you guys up the back there they spoke only today they Nate actually did a blood sacrifice yesterday to get me a new power supply hooked up here because the one I brought had failed on me he injured himself there's a lot of blood but we made it work because I presented this a black hat yesterday so thank you guys for your help and Neil actually brought all this gear out to Vegas from San Francisco so thank you and I also want to thank you thank my colleagues at Bastille especially during the disclosure process yeah that's it any questions no questions then yes question the question is is there any indication whether the new protocol is sensibly designed I've not really looked at it it was looking different it's much better than having something sent in the clear than it was before you know hopefully they've done a decent job um that can be someone else's work if they're really truly interested yep yes uh I don't know I'm not going to test it but you know again you'd hope that that that's the case it would be a bit of an oversight if it if the old protocol still worked I'm guessing you know I yeah I mean that they would have had because they would have had that transitionary period and they were transmitting both to address both you'd hope that they did that because the new one only understood the new one so um so this is just a little the little flow flow graph I'm going to say flow graph for um somebody that understands that but this is receiving this is the interface um that I've got connected to a b200 mini here and um I've got that cabled in again not to transmit over the air on a licensed frequency through that big attenuator to the Motorola radio here and I've got the the board that's connected to the siren I've just constructed my own quote-unquote malicious payload and and sent that over the air to activate the live PA mode I don't know maybe here in the front row a moment ago you heard the the little tone come out of the radio that that was me transmitting the uh the command there and so now it's waiting for a new a new transmission so it's waiting for the squelch show open and then whatever comes out of the the radio at that point whatever comes out of the radio at that point will actually come out of the siren horn so in this way you can broadcast your own message so I'm going to pop open the other flow graph that actually transmits things which is this one and then uh let's crank the volume up a little bit so that siren jack thank you very much I'll get off stage as quick as I can thank you very much to the wireless village folks again um and I know I've totally screwed up the schedule now but uh I appreciate that and and thank you for again for turning up stick around for more exciting talks