 Alright, I'm Rex and I'm going to give a talk about a series of major breaches in the cannabis compliance software industry in the year 2017. I gave this talk before, I think there was a guy here who like seen it and I was like somebody's going to have seen this talk twice and that's pretty funny to me. I gave it a few days ago at another security convention called B-Size that happened right before this one and they helped me polish it up so you guys are going to get the better version. But okay, so this is cruising the cannabis highway. Who am I? I'm a long time software developer, a web developer mainly, full stack. I've been to a handful of hacker summer camps, it's a lot of fun. Not in Infosec though I do work for some Infosec companies doing front end stuff and I'm also nosy AF. With this talk I hope to give you an overview of the cannabis industry, present a compelling narrative that touches upon disclosure. This is definitely going to be an exercise in OSNT. There won't be any elite exploits though, however there is someone who will be presenting right after me and they will be going over specific surface area attacks for point of sale stuff in cannabis. And really I just hope to foster a more general or broader discussion about Infosec and cannabis because it's sorely needed. So why should you care? A lot of times when you come to hacker summer camp you see a lot of talks about people, where people basically discuss the current state of some industry and then they say, oh this is very terrible and we all need to do a better job of securing it. And sometimes it's something like medical and medical devices and that's really important. Sometimes it's other things but I feel like cannabis is one of those similar watershed sort of industries because well there's a lot of new tech coming into it. A lot of players trying to corner this compliance software market and so a lot of people are potentially cutting corners to cut their time to market and have a greater market share but of course that means more problems, more bugs. There's also a lot of high monetary value obviously in both cannabis itself and therefore in all the ancillary tools. There's also a high number of policy touch points. So we have lobbyists and lawmakers who are active in the political space as it pertains to cannabis but then we also have software makers who are making software for dispensaries and they're also making software and very often the same companies for compliance tracking at the state level and then they're also lobbying in politics. So to me that's kind of funny, maybe potentially a conflict of interest but I'm not a lobbyist so I couldn't say for sure and potentially this is an untapped market for as much learning and drunken partying as goes on in hacker summer camp. There are also all sorts of like random 2 a.m. business deals being made in fancy suites while people get intoxicated further. So maybe this is something somebody wants to look into and maybe there's money and I don't know. So without further ado cannabis. The sweet nugs you smoke with your friends or whatever except not really. We're really talking about the cannabis industry which in 2017 was actually a $9.2 billion industry in all of North America so that makes it kind of less like my burnout stoner neighbor and more like your rich friends even more affluent uncle. In putting this village together and also doing research for this talk I've met or at least read about a number of guys who like don't smoke at all yet have extensive cannabis portfolios so that's where the future is headed I think if we don't pay attention. Not that there's anything wrong with that. We just can't let it be only that. And another thing related to that is like you know a lot of people are saying like okay well maybe I'll get into it maybe it's like the green rush or whatever but if these guys are doing this you've realistically already missed the mark pretty much. But in that light the cannabis supply chain is probably a lot more complex than you probably think. And it looks like this. The green lines are product actual cannabis the black lines are data. So to walk through it sort of quickly you start all the way on the left we have grow ops which are where the cannabis is grown obviously it could be hydroponic could be otherwise organic in dirt outdoors. That can be shipped to a processing facility to be turned into some other form that people enjoy consuming and then from there it's sent to dispensaries sometimes it's just sent directly to dispensaries and they'll make their own little pre-rolls and wrap it in a box and put branding on it but whatever and then it goes to the customer. Now at every step along that chain that I just pointed out people or these organizations are interfacing with the government in some way usually through the form of compliance tracking software. So when the grow ops starts their grow they'll often tag the plant with perhaps an NFC tag or some other sort of radio tech or if not then just a more low key whatever the plastic lock numbered equivalent is and then when they pass it off to the processing processing facility they also have to input data into the system the dispensaries have to input data into the system about where their products come from and also potentially who they're selling it to and a lot of these dispensaries have extensive profiles on their customers. We also have some sales analytics so these dispensaries are furthermore sometimes uploading this data to do number crunching to figure out which products sell the best and why. We also have grow analytics that is becoming more of a thing not as popular as the sales analytics perhaps but grow ops. I guess kind of like what Harry was saying you know entering their data into these systems to be told to figure out how to get the best yield right. If this was my best crop what did I do to get that and how can I replicate it. And then far over there we have ancillary products by which that's that's a pretty broad umbrella as I'm using it could be anything from smart nutrient dispensers that prosumer level growers are using to smart vapes I'm considering well any vape really but vapes are becoming smart and internet connected which yeah. So here are some of the major players in the cannabis point of sale space. We have metric which is a subsidiary of Franwell which is just a more general tech company like a GE or something I would say and I just cherry picked some of these stats because they just to me they give you an idea of like the scale that these organizations are operating at. So they claim to have tracked four million plants and three million parcels and also they're pretty known I think for yeah they're pretty known for their their radio tag technology. So I almost get the sense that like they while they do have software that exists at both the dispensary and the state level they also make a lot of money off of that and have a lot of market share around their radio technology. Another player is biotrack THC. I believe the original company is still around biotrack they basically made medication dispensers that go in hospitals like if anybody watched that show nurse Jackie and saw the part where she had her boyfriend sort of turned the pictures off so she could get the opioids distributed without leaving a trace or maybe you didn't see that show. But they pivoted into the medical cannabis space mainly through dispensary and compliance tracking software. And then we have MJ freeways pretty much the main topic of this talk. They are the originators of the idea of C to sale and that is what I described in the previous slide basically the tracking that happens at each stage. But what are we actually talking about here right MJ freeway they were really one of the largest point of sale providers in the industry present in many many states at the state compliance level present in probably you know an overwhelming majority of dispensaries and as such they had a lot of data to lose. So we're talking about a breach that happened on or about November 19 2016 with no data loss. That is according to MJ freeway and according to them that is what they were told by a third party security auditing company. But in actuality it was two breaches and around two months that time with data loss as they would find out some months after the fact. Except no maybe it's more like three really with a potential pivot in the government infrastructure and the definite loss of personally identifiable information and a several month outage and also in one of these states which you know this hasn't necessarily been attributed but I think the attacks are probably related that's me speculating. But there was a state where attackers modified basically some live delivery information of cannabis restocking so some pretty gnarly stuff is happening and I was kind of like all right this is crazy. And so I started reading more about it. And this is something these are some things that were said from MJ freeways marketing department that they were the victim of a vicious cyber attack. And also that they're only really coming out to say this because a lot of people are saying that this was not a hack and not an attack and it absolutely was. Which is interesting when you get hacked and a lot of data is lost and your customers say maybe your DevOps person just fucked up. And then on January 8 our clients began to experience the effects of this the MJ freeway system went offline for all our clients who also had no access to the MJ freeway site. So this is actually I went to a dispensary a few years ago here in Las Vegas during def con. And I was able to talk to the bug tinder that I met there about her experience while this was going down. And this is pretty anecdotal but I guess their boss walked in and they said hey the system's down we can't really do anything. He said you're all liars who try the most to avoid work. So I wanted more anecdotal evidence because even though objective evidence is probably better anecdotal still fun. So I went to Reddit which is the best place for subjective hot takes. But instead I actually found a lot of people who are involved in the industry just really venting their frustrations. And you don't have to read all this but to sum it up they did not do a good job of disclosing that this was happening and a lot of people found out just like I said with my friend by going into work and finding that the system was down. And then over the course of the next few weeks and months they would receive a lot of mixed signals about you know what kind of backups were available because the attackers targeted the backups as well by the way. And when they would have the backups rolled out have the system back up a lot of people jump ship. So my first reaction is like oh my God why is nobody talking about this is a big deal this company as I came to realize is so large in the cannabis community or in the cannabis industry rather. But as it turns out that's inaccurate everybody was talking about it like from national print based news publications to mom and pop blogs that focus on the cottage cannabis industry. So a lot of people were talking about it but still that made me that still gave me a funny feeling though because I'm like all right if everybody is talking about it then why didn't I hear about it. If major news outlets are talking about it why isn't everybody aware of this. And I think they just did a really good job of sort of getting in front of it and controlling the narrative. So I do give them kudos for that actually. But like I said you know looking at anecdotal stuff and just reading a lot of speculation from a lot of people. And one of the things I read about was that the source code was had to so not only was personally identifiable information leaked and put on the pirate bay by the way but also source code was uploaded to get lab. And initially I think MJ freeway came out with a statement saying that this wasn't real. I don't understand what they meant by real. Did they mean like that's fake news or more like we didn't do that. So please don't look at our code that's there. It got taken down pretty quickly though. But it was definitely posted up on the pirate bay as well. So I grabbed that and I started finding any kind of news source I could. And there were a lot of them. And I figured that the timing was going to be important. So I made a spreadsheet as one does with a timeline. And then like I said since everybody was talking about it I grabbed every news article that I could. And what was interesting was that some of them like it was interesting to see their perspective and where they were getting the information from. Some of them would just say stuff like you know they would take a very centrist approach and say outage affects major you know cannabis point of sale lots of dispensaries having trouble and then other people would be like MJ freeway fucked up really bad. But then some people even though there were these sort of different perspectives some of the authors of these articles did manage to find nuggets here and there that I couldn't really find anywhere else that really led me further down the rabbit hole. So back to Reddit. So I read the post from the people who were in the industry and directly affected. But then in that mix were these weird things where like these people were just sort of chiming in with what looked like hot takes. This one dude is like yeah I know the guy who made the software and I've been in it for 20 years and here's how I think it happened which sounds really suspicious. And apparently I wasn't the only one that thought that because MJ freeway subpoenaed Reddit and also Google for information about all those accounts that sort of had first hand knowledge I guess thinking that maybe they were affiliated with the tax amount. That's pretty logical thinking in my opinion I mean I don't know that I think that they did have something to do with it but it's worth checking. So like I said I was interested in the source code as well because a lot of the other stuff is subjective source code is pretty objective. So I went on the pirate bay I grabbed this torrent. Also I was told by the EFF that it's very important that you know that I got it from the pirate bay. But what you're looking at here took me about four months to download to get to that 46.1% and that's after fiddling with like distributed decentralized peers and all that stuff. And really just what I realized is that nobody was interested in the source code and going back to Reddit I found posts that were like who gives a shit about this leak. It's really old Drupal code that was never patched so I'm not going to learn anything from that. And I was a little bit disheartened because even if that's true I still wanted to know what was going on in there. And then somebody told me to do another search and I was like duh I didn't think of that. And as it turned out this code was reposted a number of times in the pirate bay so to me that makes an interesting narrative. You have some people saying I don't care and other people saying no please look please. So again I don't know what that means but it makes my spider sense tingle or whatever. So I grabbed this and started digging in. So okay first off just even doing this directory listing not a great start for them I mean like I think it's pretty cool that like I'm all up in their shit. So I'm not a hacker like I was saying. I have a pretty decent background in consulting by which I mean working for a company that outsources my services to another company. And so I've worked with a lot of other developers on various teams in various configurations. I've worked with offshore teams and other remote teams and things like that. And one thing I can tell you is if you look at so you see how there's up at the top it says CC leaf. And then if you read down there are other ones that like just leaf or like Washington leaf or those other things. What you're seeing there is like like if I can sort of put on my like Oracle Psychicad a little bit. This was a team that could not decide on a software development process that would allow them to share a common core and easily build a custom customer functionality on top of it. So what they chose to do instead was just split the repo five times which is a pain in the fucking ass. I actually have a tattoo on my leg of a man named Melvin Conway. He makes something called Conway's Law that says your software is going to look how the communication between your teams looks. So I would say this communication between these teams is probably not pretty good. It's probably not very good. And that will lead me to believe that you're going to find exploits when you look deeper into the code. A note on how some of this stuff is laid out right. You're quint your obligatory stack diagram. So we're most concerned with what's going on in the middle. Those are dispensary level trackers. You have MJ freeways Graham tracker. That's the one that was based on like a six or five or six year old Drupal core with like a bunch of custom plugins that basically made it so that there was no upgrade path. So really a person could just fingerprint their Drupal instance from afar. Go to any like CVE site and say what are all the exploits that have been found for Drupal since this version. And I believe that's what the attackers did pretty much. Then you have MJ freeways MJ platform and that was supposed to be their new and improved thing made from scratch. I think they got a lot of flack from people in the community because some of the stuff I saw on red it was like a guy who works in the dispensary but he also had a lot of technology maybe even some info second. So he was like hey I did a scan on my instance of MJ freeway and I didn't like what I saw. And I think some of those people were very vocal about that. So perhaps that was part of the motivation for MJ freeway to roll that out. But then also they were just sitting on Graham tracker for a while. So you can't really I mean you can be the industry leader and not really innovate and then you get surpassed by somebody else so they didn't want that to happen. And then we have other third party dispensary trackers from other companies. But below that you have MJ freeways leaf data system and that is their state level compliance tracking. Now I definitely read something from their marketing department where someone sent them a question saying isn't this kind of a conflict of interest and they were like no no that's a separate company it's okay. So that was kind of funny. But the middle tier is supposed to integrate with the lower tier. Everything the dispensaries are doing they have to feed to the government like I was showing in the previous graph based slide. Now a lot of times these companies their selling point is that you will automatically be compliant. Your data that needs to be uploaded to these compliance trackers will that will be done automatically. So to me that means that your data is probably being double dipped depending on like where you live and where you're buying your legal weed. And if it is a company with not a great track record of security that's like kind of troubling. And then if that surface area wasn't enough you have stuff like dynamic websites that are being made by your local mom and pop web developer web developers. And so they are using the API's that these trackers offer to perhaps you know show the most up to date inventory at that dispensary. You have in store menus like the ones you see above the counter at McDonald's but for weed. And those also integrate with your dispensary tracker. You have your vendor or suppliers. So maybe your processing facility is directly uploading data to you based on what they're doing. And then you have mobile apps that again are showing potentially what is available at your local dispensary. So those things can be hacked as well. It's a lot of things going on here. So one of the things that I wound up reading a lot about was MJ freeways dealings with the Washington legal cannabis industry and the sort of organizations that exist at the state level that that are required to implement that. And I would have foiled. In fact I thought like oh finally now this is my first chance to like have something good to like submit a FOIA for. And then I like seriously mismanaged my time and couldn't. But I found something that's like almost this is good. This is a sales force instance that has like PDF minutes of every meeting that the chief information officer of Washington State has. And so there are just tons of things about MJ freeway and their bids to win the contract and also how they missed deadlines with the the handoff because BioTrack at some point had that had this this state contract essentially. And at some point the Washington State decided they would rather use MJ freeway. MJ freeway missed several deadlines and so there is much information in here about that. As well as some posted videos from the Washington State Liquor and Cannabis Board to dispensary owner telling them not to worry about all the rigmarole that's been going around related to breaches in the cannabis industry. So back to the code. Again I'm not a hacker. I'm not looking at the code trying to find exploits. I mean I probably could I'm sure. But I thought it would be much more interesting to see what the metadata would say. So I harvested all the emails from Git. Git log lets you format the display in any way you want. So I pretty much made it look like a CSV and with a minimal amount of deduping and manual cleanup I wound up with a very long list that is poorly redacted. Now if you notice this is just like the A's right here. So it was a pretty extensive list and as you can notice a lot of those email addresses do not end with mgfreeway.com because they use a lot of consultants some of which are offshore or near shore or probably even working with them for an extended amount of time but some companies just like to structure themselves that way for whatever reason. Sometimes it's beneficial. Oh and then after I got these emails I fed them into recon ng which is a fucking amazing tool. I believe I was using the jigsaw puzzle to enumerate on other social media profiles and although that API key costs like a bajillion dollars if I had it I would probably buy it because I'm very nosy. So okay collected a lot of stuff there's much reading to be done I had blog entries tweets Reddit articles podcast transcripts training manuals get commits and I found a lot of interesting things. So first off like I said there are a lot of email addresses that are from contractors obviously so I can see the contractor companies because a lot of them in in those URLs but a lot of them are also Gmail addresses and some people just would like alternatively commit from their Gmail address or from their work address. I've been there I've also fixed it which is saying a lot because I can be a lazy developer too but that's the kind of thing I don't want I don't want sitting around in a get repo for other people to look at and point fingers at and then I noticed so like if you're looking at the top what does that mean I didn't bother verifying it but my thought is that that's them committing directly from the get UI. Like is that how you get that email in your in your get log does anybody know? No. Okay. One guy made eight commits in a row using just his name as a commit message which I'm sure the commits weren't all that important but again if I was a hacker and I was going to look for exploits and code I'd probably start with his code. And so again lots of remote contractors lots of various lots of varying Gmail addresses. I would probably like if I was a less principal person fudge one of those Gmail addresses like make one that's very similar and differs by one letter and start emailing like everybody on this list. And because I see all your get commits like sort of even keeping in mind that this stuff has been leaked or maybe you know that hypothetically we were talking about an open source project. I still feel like if you say some really specific stuff to somebody that's going to resonate with them they're going to be like oh well maybe this is legit. Especially it kind of especially if it looks like an email that I've interacted with before for work and why else would I have a reason to doubt such a specific message. So here's what that sweet spear fishing pretext would look like. Hey developer I've been dealing with some of your commits as of late and I wanted to point out that you do this weird thing where you commit from your vagrant environment because that's what one of those weird ending addresses was to. That makes it really hard for me to reconcile some of your other work when I rebase off master. Could you please not do that anymore. Also I think there's some stuff in your local repo that you probably didn't push to master. Would you mind sending me a patch of that. And I think that would work. I don't know. And I received a little bit of indication so when I was reading these subpoena documents that MJ Freeway submitted to Google. That was one of the email addresses. That's not the person's email address. They swap two letters. I'm not sure if the attacker tried any social engineering stuff. It might have just been an address from which they like used to do the smear campaign and I'll tell you about that in a second actually. So perhaps the most interesting is this letter. An open letter to the state of Washington cannabis industry. I'm sorry. I should be looking at you guys and connecting with you. But the icon is really small on my screen. And yeah. Well anyway. So this was written by a guy named Patrick Vo. President and CEO of Biotrack. And I don't even think I found a blog on this site or any other letters. So I thought it was very interesting that they posted this one and then even a link to download a copy of this. Click here. So there's a lot. It's like eight pages and it's very heated. He is not happy to just summarize it. He was pretty tight over how things went down in the handoff that I mentioned before from Biotrack to MJ Freeway in Washington state. And he gets at the Washington state liquor and cannabis board a little bit over their lack of accountability and how they sort of mismanaged the process as far as he was concerned. And he said a lot of things about MJ Freeway. But basically the biggest one was we really can't afford to have our reputation commingled with another company that is this insecure because it will also make us insecure. Which there's some truth to that. But like they were really what they had to do concretely was like a ship around like CSVs by FTP upload. So I'm kind of like OK how insecure is that making you really. But you know I'm not a business owner so. But it also references these data ransom attempts that the attackers made. So he was nice enough to copy and paste a message that someone received and they posted it. I believe to a Google group for dispensary owners and workers. But basically the attackers were uploading snippets of the data to these like Bitcoin lock box sites and then ransoming it and sending these emails from spoofed addresses to all these dispensary owners. And I think the price is I'm not sure if it was for a sample or for the actual dump but it was like eight bucks. And then also it's got this like foe like I want to mess up your attribution by using what I think a foreigner who doesn't speak English well would type like you know what I mean like you are by database Bitcoin 100% anonymous. And like I was going to try to see if I could like do some like lay persons threat attribution. But it's kind of obvious here who did it. I probably don't need to go into that anymore. I didn't have to make that graphic. It was just just a good fit. But jokes aside while this animosity as I said before I've worked in consulting. I've worked with some pretty asshole C level guys who when cameras were off or nobody was around they would talk shit about their their employees their customers definitely their competitors but in a public sphere even if a competitor was like messing up really bad they would just be really like gracious about it. And I think they understood on some level that engaging in drama is a Pyrrhic victory kind of right. You probably are just not going to look good even if you're you know you kind of are effectively dragging your opponent down. So but yeah again that spider senses tingling and I'm basically telling myself there's something else going on here. I don't think that just this idea of you know your reputation being commingled with someone insecure is enough for all that vitriol. So I did more digging. And as it turns out there's definitely something. So apparently both Biotrack and MJ Freeway were in the same bid to be the state level dispensary the state level compliance tracker for Puerto Rico. And because of transparency and how many sort of public records get uploaded now. You can just find information about that. It's posted online invitation for bid for C to sale inventory tracking system PR dot gov. Wow. Okay. And this is a lawsuit from MJ Freeway against the Puerto Rican Health Department and Biotrack the winner of the bid. So basically their attitude was hey wait a minute according to Puerto Rican rules a company that has a felon on the board should not be allowed to win this bid. And as it turned out Biotrack did have an ex felon on the board. Now this is something I don't know this is kind of stuck out to me. I don't even speak Spanish but I was like let me peruse this and see what happens. In conforming means unhappy. I'm not a lawyer but to me unhappy sounds like an awfully subjective word to be in a legal document but that's neither here nor there. It's definitely telling them. But so okay. More about you know the board member and the the the the the felonies or what have you. It was a Mr. Steven Seagal. Not this Steven Seagal but like it would have been really cool if it was and I wouldn't have even been surprised to be honest. And so basically back in 1999 the guy that I am talking about was involved in some male fraud and some stuff like that. And so you know I guess objectively speaking perhaps MJ freeway had a leg to stand on. But you know I don't really know like ethically speaking or morally speaking like what side I fall on here. But it was definitely entertaining to read about. And here's another suit that biotech was involved in. So I guess there's a lot of just litigation in this space in general. It's highly competitive. So okay. Drawing to the end here but here right. This is from the timeline again. This is a very dense you know snippet from the timeline. And if you look at the top I'm going to get up because I can't read what's all the way over there. But I want to speak to that. 2016. MJ freeway brings a suit in 7 2016. Steven Segal Steven Segal. He actually divests his shares. Him and another gentleman named Brian McClintock. They sell it to a holding company. So now they're not on the board anymore. So moving forward MJ biotech won't have this problem. Right. And they also put the guy who like the guy who's the CEO of the holding company I believe they put him on the board and MJ freeway launches their MJ platform at marijuana business conference here and expo in Las Vegas here. And three days later they were breached and then double tapped. I'm just speculating these things again. I'm not a hacker. I'm definitely not the person who can do threat attribution. But if I was I would definitely be looking closer at that. I don't want to say aliens but you know. But then I did install their stuff. I kept saying I'm not a hacker but I did want to dig around a little bit. But fortunately a much more experienced person in pen testing wrote me like two weeks before Defconn it was like hey I'm actually working on some of this stuff and I would love to present. So I'm just going to leave that as an exercise for him. But in conclusion who was to blame again. I don't know. I am definitely speculating about all of this really. But like in my head at some point I was like maybe this is some kind of foreign state actor thing right. Sounds implausible but I'm not really kidding like some of those consulting emails were some from some pretty interesting places. And also I would like to think that you know if I wanted to point out the sort of downsides to to one's Western capitalist bourgeois ideals probably shitting on your new found fledgling vice industry and like you know letting it fall on his face that would probably be a good way to do that. And there's been a lot of psyops going on so maybe that could be a related thing. Maybe it's like some right wing anti weed activists. They are you know alt right hackers now that's a thing. Competitors perhaps very competitive space or just maybe board teenagers like advanced persistent ones. The cannabis space has a lot of interesting concerns regulations and compliance. There's a lot of it and when it's done poorly it equals increased surface area for attack. The space is obscenely competitive so if you are weak at all your competitors will smell your fear and devour you apparently with lawsuits and maybe other things. And also another thing that's interesting is that when you are in the legal cannabis space and you get hacked you probably are not going to go to the feds. The feds are probably not going to help you and even if they would you probably would not want them to help you. And I didn't change the slide so I said come to the cannabis village and check this out but you're here. Thank you. And that's it yeah. I believe that is Mr. Lewis in the back with a question. Yeah I don't even know I was super curious. I think they might have been like second runner up and maybe MJ Freer was like both of you. Yeah and like some of the later lawsuits that I was showing was between Biotrack and other legal cannabis software companies also in Florida. So I thought that was pretty interesting also. Any other questions. OK.