 Hello everyone, welcome back. My name is Rubik's Woman 38 and I welcome you to the Tinkus Canary Workshop. If you have not already joined us in the Discord chat, please go to blueteamvillage.org and click on DEF CON 28 and join us in the Discord. The text chat while we go through this presentation will be in the Text Workshop's Track 1 under Flamingo Hotel. Just scroll all the way down, go to Text Workshop's Track 1 and you can post your questions there. I'd like to welcome Adrian and Bradley from THiNKST. Go ahead guys, take it away. Thank you. Hello and thank you very much for that. That was a kind intro. And we are very excited to be here. And we're going to talk a little bit about how canaries and canary tokens are now going to be integrated into opensock and give you a little bit of context and background on them so you can understand how to use them when going through opensock. So again, big thanks to opensock. We're really excited to be part of it and really glad that they thought to include us in this. All right, so what we're going to do first is just go through some slides real quick here. We'll go through some live demos as well. A lot of what you're going to see is kind of a mix of the commercial product and the free product. Really, the free stuff we have out there work the same way, same principles, same data you get in the alerts. Just for convenience and time sake, you know, you might see a little bit of the commercial product here. So some of our goals when we design canary and canary tokens was, you know, a tool that gets deployed and gets used is useful. So a lot of work went into making them super easy and simple to use. They take minutes to set up and deploy. So a lot of work goes into making them super easy to do that and little to no maintenance. So again, you know, for a tool to be useful, it has to function, you know, so any kind of maintenance, you know, we try and make nonexistent or, you know, we can do it automatically for you in the background. Even the free tool, most of that you can just generate canary tokens, deploy them, forget them, deploy open canary and pretty much forget about it until it sends you alerts. So yeah, and those alerts, that's a big goal also is, you know, how can we get this as close to zero false positives as possible. And ultimately, you know, it's the goal here is to know when it matters to know when an attacker is in your environment as soon as possible when it matters to know that. Right. So as I mentioned, our open source and free products here open canary is available. It's well documented. You can grab it from GitHub. I think the Docker container has been downloaded tens of thousands of times so we know it's getting a lot of use out there. You know, the package has been installed and the you can run your own canary token server as well with a Docker container or you can use canary tokens.org. I think we have over 100,000 people using canary tokens.org. You don't even need to create an account to use it. You can just go there right now and start creating tokens and distributing them throughout your personal devices. You know, your your production environment wherever you want to use them people use them all over. All right, so talking about the context a little bit here. What are these and why would you use them in a blue team environment or open sock. So the goal here is to let you know as soon as the attacker gets in the environment. So, you know, over the last couple of decades dwell time is really long attackers are in environments way too long and they just they go on undetected. And, and that's the problem we set out to solve here is, you know, what if we could get that dwell time down to minutes. So when you at least know that there's something suspicious going on inside the environment. So that's the goal here is the attackers already in the environment and doing suspicious things. And that's what we alert on. So that's important context to understand. And how do we do that. So I've got a quick diagram here just to show you simply what we leverage to make our product work here to make it win. And when an attacker gets in, they don't know where anything is in your environment they don't, you know, unless they land on a Visio diagram on a highly detailed Visio diagram the moment they get in they're going to have to explore the network. They're going to have to look around for what they're what they're hoping to find. And if you drop in these these canaries which are honeypots which can look like anything else in your network anything from Windows Server 2016 to a Linux server. You know the idea is it looks like a normal server but there's no there's no real reason anybody should be touching it or using it. So the attacker comes in does that same exploration, and they start setting off alerts. And furthermore, when you take canary tokens, you can set traps throughout even more of the environment you can make it even more tricky and difficult for the attacker to explore around the network without setting off even more of these alerts. So really what the attacker is doing when you've deployed these traps around in your network is they're they're painting a roadmap for you they're painting a picture for where they're going what they're doing. Even their motivations, you know what what servers are they targeting. When they open up some of these token files what are the file names that are getting them to double click that. You know the files that look like they have employee data files that look like they have customer data files that look like they just have more passwords so that they can pivot further into the environment. You know, a lot of strategies you can use when deploying both the canaries and the tokens here to understand what the attackers doing where they're going in your environment, and in paint you a nice picture of what's going on. And of course, when those alarms do get set off we want to make sure they go where you can get to them in the context of open sock this is going to be gray log that's where you're going to see any canary token alerts or any canary alerts. Of course, you know we can send them tons of other places but that's that's where you're going to see them for for open sock. So we'll take a closer look at the kind of details that you're going to get in those alerts. So, where is canaries sit on the network and let you know when suspicious stuff is going on on your network canary tokens can do it at a smaller level. Canary tokens can be a bit of code can be a file and highly complimentary to the canaries you can even have canary tokens on canaries or canary tokens within canary tokens which sit on another place that is triggered by canary tokens you can layer these really deep so that even if one or two aren't maybe triggered you know something is going to trigger if you deploy enough enough of these in your environment. And understand how these work and how you would use them or how you might see them used within open sock we're going to take a look at probably one of the most common and popular canary tokens that get used, which is just the word doc canary token. So in this scenario here, whoever's creating this canary token, you know our canary user creates a word canary token and this token can look like a real word file you can even upload an existing word file, you can copy any kind of contents you want in it. You can make it look as realistic as you want, and you're going to name it something the attacker wants so really thinking about this, you're fishing the attacker you're using the same principles attackers use against us against them. You know you're trying to trick them into opening a file into taking an action that's going to trigger a one of these trip wires that's going to trigger some of these alerts, and let you know what the attackers up to. So grab that, you know, maybe the they send it off to someone else, you know they're going to sell the data. But at this point it doesn't matter anybody that opens this, this word document this token word document is going to set off an alert. As soon as it's opened, you know before you've even seen the contents of that file it could be blank, you know it doesn't matter how clever the file is how cleverly you've made the rules. It could have just been lazy and left it completely blank. And by the time the attacker knows that it's already sent off an alert with some of their data attached to that. And the canary admin gets the alert and sees the details of who opened it where they opened it. And, and you know what's going on you know that that file has left that secure environment wherever you store that file. All right, so what can these alerts tell you at this point we're going to jump into an interactive demo here. I'm going to leave the slides behind. Right. See how big we can make this without it going off the screen here. All right, so this is my console here I've got two canaries running. One looks like a Cisco and one is set up as a Synology NAS. I've got my command line here I'm just going to I'm going to use this to run some in map scans against these hosts here just so you can see what it looks like when one of these alerts does go off. So I'm going to copy this IP address, do a quick scan against it. Hopefully that's big enough to see there, make that larger. All right, so the attacker sees this they see what they would expect to see on a Cisco router on maybe an older Cisco router it's still got telling that open MAC addresses Cisco. I'm going to go ahead and look at what the scan tracking system looks like Cisco looks like a Cisco 1920 router. And as you probably noticed there as I ran the scan we got a few alerts. So, you know, first, first thing we know somebody scans the canary. If you have port scan detection enabled, you're going to get alerts when you get scanned. And we can even tell that it was an end map scan, you know the OS flag and end map is is pretty easy to detect. It really looks like it so we know when we've been somebody's doing some OS fingerprinting with within maps specifically. So maybe the attacker tries to log into SSH, you know they saw SSH was open here, maybe they're hoping for some default credentials. And just like SSH, you know they try three times, you know it doesn't work. There is no correct username or password it's going to get you into a canary. They just exist all these services exist there for the attacker to spend their wheels waste their time trying to get in. They could try to get in through the web config there, which is just basic off nothing fancy here. But they could it could throw brute force, you know dictionary attack against it. All they're going to do is generate more and more alerts for you to see what they're doing. So already we have this picture here, you know they painted this story of what they're trying to do, you know they've scanned the network. They found something interesting they tried to log into it. That didn't work. They tried to log in a different way that didn't work. And you know you can see what this attackers trying to do. And the main thing you're going to use to pivot off of when you see an alert from one of these canaries is probably going to be the source IP. So some of these details can help you as well you can see the username and password that would that they tried. You can even see the remote SSH version that might tell you something about the host that they're trying to get in from. You know so some of these details might help you. But this IP address and the reverse DNS. These are going to be the main things that you're probably going to pivot off of looking through gray log looking through molloc. Some of these other tools that that's going to point you right direction and then you know the rest of the tools that you saw today. If you watch any of those presentations those are going to help you build that investigation and figure out what's going on. When they do hit a web server on the canary. Browsers are nice enough to hand off the user agent that tells you a little bit more about the attacker in this case obviously I'm using a Mac and that's revealed through it. Of course an attacker could you know they can change their user agent. You know they could even mask their IP address. But generally, you'd be able to tell if that's happening as well. You know there's other ways of telling that. Right so that's that's what that looks like. Just doing a scan and trying to log into a few things on the. On the Cisco honeypot there and this one the Synology NAS is also fairly interesting. We're going to do a quick scan of this one. This looks a little bit different. One of the key things that's different about it is it has a file share. So we're going to run a quick scan against it. Again, our results come back pretty quickly here. Mac address Synology services look like what you'd expect of a file share and a network storage device. They do run links and that's what comes back as so you know maybe the attacker any attacker that sees an open file share. You got to know what's on that file share right. So we're going to pop in there and take a look. And by default we're going to make it pretty easy for them to get on there. You know you could lock it down. Use some stolen credentials to get in there. But we really want to know what they're interested in. So we're going to make it easy. You know so with a guest login they can get on there. They can see the different files on there, you know, and this is where it gets interesting if you've got files that look like they have different types of data. Depending on what they go after, you know, maybe in this case they're going to go after the router config, you know, they're still hung up on that Cisco router. They really want to get in there. They want to check out this config see how it's configured. Bummer looks like the files password that's as far to go with that maybe trying on a few others here. But you get to see which files are interested in which files they go after. We see here in the alert they went right after that Cisco router config. And again, by now, you know, this is painted a pretty nice picture, you know, we know who the attacker is. We know they're scanning the network and we know they're trying to get into things. So pretty clear picture of what's going on. And that's really the goal of the product here is to paint a clear picture that, you know, yes, this is probably a malicious attack and we need to investigate it. That that early breach detections. So now you're going to you're going to take that you're going to pivot to other tools, and you're going to start to dive deeper. So I'm going to clean up these alerts real quick here. And we will talk a bit about canary tokens. So like I mentioned before, where canaries let you know when suspicious stuff is going on in the network as we saw with scans login attempts there. So in canary tokens, you can go even deeper, you can compliment that some of those files and that file share I just showed you could have been token files. And what that means. So in the case of the Word doc here. And as I mentioned before, you can even upload your own Word doc so I can take my fake pen test results here. I can just I need to give it a memo. So one of the key things here is you really want to use one token per location. So, you know, maybe, maybe I'm going to put this on my secret flash drive. I don't know why it's secret. It's just a flash drive. I only have one flash drive. Let's say that. So I'm going to put that there. I'm not going to put it in my email. I'm not going to put it on my desktop. It's only going to exist on that flash drive. So if I see any alert from this canary token, I know somebody's got my flash drive. Simple as that because that's the only way somebody could have found this token. So we download that. And remember this was an existing Word doc that I had. You know, I haven't done anything special this is a normal Word doc that you can grab you can upload. You just added a token to it. So the attacker opens it up. There's no indication that this canary token has sent out anything. There's no indication that the attackers fallen into a trap here. But in fact, it has sent out several alerts. Our Word doc canary tokens have what we call our DNS primitive, our DNS tokens embedded within it, and our web tokens embedded within it. And part of the reason we do that is because it doubles our chance of getting the alert out on some networks. Maybe HTTP port 80 isn't allowed outbound, you know, only HTTPS is allowed outbound. So this wouldn't have gotten back to the console. But as long as the person that opened that Word file had working DNS, we would still get that back. You know, so we've got a really good chance of knowing when somebody opens one of these Word docs. And you can see we get a, you know, the user agent here doesn't give as many details, but we can see what version of office they've got. But the key information here is somebody would have had to have physical access to my flash drive for this alert to trigger. And they were interested in this file out of all the files that were on my flash drive. This is, you know, the first token file that they opened. Let's say I had multiple token files on there with, you know, different file names suggesting they have different types of data in them. And they went after this one first. So that's useful to me also. And then I've got the, you know, both the DNS server that handed off that DNS token that triggered. And then the web token got out as well. So, you know, that that's my actual IP address that's assigned to me by my ISP. That's what the web server sees when this Word document opens and it reaches back to the Canary token server that lives on the console here. So couple, couple useful pieces of information there that I can, I can leverage. So yeah, that's the Word.Canary token. We do have, as you can see here that there's different tokens for all sorts of different purposes. And this is pretty similar to what canarytokens.org shows. I think we have almost every single one of these tokens there as well. The other one I'm going to show you here is the QR code token. This is a fun one because it's very versatile. There's a lot of ways that we see QR codes used in technology today. So setting up your multi-factor authentication, setting up, you know, access to a secure messaging app, enrollment, you know, maybe into your MDM, EMM product. You know, you see them out and about in physical places all the time. You know, scan here for more information about this house that's for sale, that kind of thing. So sky's the limit for how you can use these. You can use them digitally or you can use them in physical environments. So let's say I want to know if somebody gets into Adrian's lunchbox. You know, I don't want to know if somebody's just touched my lunchbox. I want to know if they've actually opened it. Because I've stuck this QR code on the inside of my lunchbox. I've printed this out, stuck it in there. You know, maybe put some text around it like recipe for Adrian's, you know, Adrian's secret recipe, you know, handed down through the generation. Something to that, that respect, you know, something somebody would be really interested in scanning. So I put this QR code there. If I ever get an alert on this, I know somebody has not only touched my lunchbox, but they've opened it up. They spotted the QR code on the inside. They read the words around it and they chose to scan it with their phone. You know, that's the only way I would get that figure. So that's that's the power there. And we've seen people put these in all kinds of locations. I think one of my favorites is somebody told me that the, they print this out on a sheet of paper, they put the words Microsoft authentic recovery around it and they just leave that sheet of paper on their desk at work and just wait. And that's great. Four people are checking out my lunchbox. They want to see that secret recipe. I know this one is me, you know, we, first of all, we just saw that source IP everybody knows this is me by now. We'll actually do a GOIP look up here. Get some details there. And, you know, we can even see what kind of phone I'm using here. You know, we, you get different details out of these user agents depending on on how they're scanned, what devices scan them. You know, so maybe there's some crazy situation where that those kinds of details are useful. You know, the key point here again is I've only put this token one place. You know, it took me 30 seconds to do it to scan it out, tape it to the inside of my lunchbox. But I can just leave it there forever. And if I ever get a trigger off of that, I know that somebody has been in that lunchbox. Maybe that lunchbox is maybe it's not a lunchbox, maybe it's a data center, maybe it's a network closet. Maybe it's underneath the battery on a laptop, you know, all kinds of different places that you could hide these. Looks like we've got somebody from Oklahoma that scanned it. Again, you know, if you're coming through VPN or something like that, that would be reflected here. But we've got an Android pixel three XL. We've got mentioned via host somebody coming from California. And a lot of these user agents will refer to the application used to open it like I think that previous one we looked at ZX ING that's a barcode scanner app. And then one more here, that would be Bradley. Bradley's in very early in the morning in South Africa and scanned it with his iPhone. So hope that helps to demonstrate the power of how you can use these tokens and how the canaries are used. And most importantly, when you see alerts from these understanding what the attacker had to do to trigger those alerts and what it tells you about the attacker, where they are, and maybe even their motivations and where they're going to go next. And help you pivot between some of the different applications here. So really that's, that's it in a nutshell, I think we still have 20 minutes. If there's any questions, we could absolutely take some of those. Bradley, I appreciate it. This was a great presentation. I did mark down a couple of questions that were asked during the presentation. Does canary token create or does the canary token created based off of the session plus protocol plus application? Is it that uniquely identified? I'm not sure I understand the question. So it might help to talk about the primitives a bit more that that might answer the question if I talk about the web and DNS tokens that are kind of sit behind this. Yeah, I think that would be the way to go Adrian is to look at how the token has been created because I think the question is related to whether it's being tripped by the protocol or the application. Okay. Yeah, so first one I'm going to create here is a web token. And I'm going to create a fake email in my inbox that makes it looks like if an attacker gets in my inbox, maybe they get all my passwords, they get access to my last pass instance or something like that. So I can take this web token and I'm going to create a link in this email that this fake email that I put my inbox and I'm going to basically fish, you know, or social engineer the attacker into clicking it. And so what we're actually calling a token is this bit right here. This is the actual token that the web server is looking for. And this address here, and if you're using canary tokens that org, you know it would be something based on that you can run your own canary token server use it on your own domain. You know run your own Docker canary token server. So this could be anything you want. This could also be anything you want. We could name that admin files, we could name this password dot DB, and that's still going to trigger an alert when we go to that. So, and it's based component. And when we just demonstrated the QR code token, that's just using an embedded web token. So super simple way of doing it. So if you're looking a web server with this token here, we know to map that back to Adrian's inbox, we know that that's the token associated with this memo with this reminder that I set for myself. I hope that I hope that answers the question of how that how the token itself works. And, and that's the same token that we have embedded in a lot of these other ones so the word file has a web token embedded in it. And the QR code does, you know, in several of these others do. The other what we call our token primitive is DNS. So, again, there's ways to get, you know, if somebody gets in my inbox, we can, we can get a DNS name to resolve. So if I was to just copy this DNS name, and again here, this bit is the token, you can tell kind of from the length of it. And really it's a C name on the front of this domain, you know that maps back to us. So if I do an NS look up on that, I get an alert. So wherever I've that DNS address, any resolves it if it gets resolved for any reason, maybe I'm just sending it in clear text on the network. Somebody captures that traffic with wire shark and they view that traffic and they tell wire shark to resolve all the DNS addresses that would trigger it, you know something as simple as that. But if if I can find files, you know, any any kind of tricks any kind of ways to get a DNS address resolved that that can trigger a token and send me an alert. Just to add on to what Edwin mentioned is that we embedding HTTP request and the DNS request inside of a document, and we relying on when the document gets opened with MS Word, for instance, MS Word reaches out with those to those URLs of the DNS. So it's MS Word the application itself that's looking up those requests that I embedded into the document and that's what's triggering the alert. Yeah, and that's important to know, you know, some of these require action from the attacker like my example of the fake email my inbox, I've got to get I've got to convince them to click that link. In other cases like Bradley's example with the Word doc, simply opening it is all they need to do you know they don't see it. It happens in the background they don't even know that those alerts have been triggered. And there's nothing else that they need to do. Well, I appreciate it. Thank you, Bradley. Thank you, Adrian. This has been a great presentation. And so this concludes the presentation for the workshop for canary and any case if you have any follow up questions. The speakers will be around in the discord for a few minutes. And again, the discord that we are chatting in is the text s dash workshop workshops dash track one in the flamingo health hotel group to scroll all the way down to the bottom. And so if you guys consider coming down, I think there's a couple of questions if you've got some from follow up that would be great. Otherwise, I appreciate ready listening in today and thank you very much for attending.