 What I'm going to talk about today is a lot of my personal opinion based on years of military service, based on pursuit of a Ph.D. right now, where I'm doing a lot of study on what I believe to be a social issue, radical Islamic movements online via social media, and then it's also based on a bunch of work that we do. And so disclaimer, we do do a lot of work with the government, and so that unsuccessful approach at how we've been doing things in this space is going to inform a lot of my opinions. And I hope this is a discussion that starts with a bunch of people in this room going forward and this can snowball into something that's beyond just a talk at DEF CON. So, go ahead. So why Twitter? Why Iceland? Why Tech? The first thing, why Twitter? I think I could just as easily kind of bend the technical social media domain into five or six other spaces right now, but first of all, Twitter right now, it's a really important space for this conversation that's happening. It's important because of how prevalent it is, how free it is, everywhere it is. All you need is 140 characters and a free ISP. These two pictures I've got up here, one on the left of the screen is a target that's put on Jack Dorsey's head after Twitter starts earlier this year kind of deleting accounts and mass that are violating, you know, Twitter's terms of service and policies. The figure on the right is something that I came across literally in the last couple of days posted by a firefighter in Fairfax, Virginia, right outside of DC. So right here on our home, and I think that that's not a surprise to anybody, but if anybody can take a close look at that picture on the right and anybody point out where they see a Twitter handle? Bottom of the rifle, right? So it's important they're using these mediums to spread information and not just pictures and not just fun little images and pictures. They're using it to communicate in a very sophisticated way. So it's really, really, really important. And the next thing is it's really, really, really hard. It's really, really hard for us to figure this space out because it changes literally by the day. This was something that was in the news recently and we've got this whole selfie phenomenon. Well, somebody posted a selfie in the Middle East and then within 20 hours later because they didn't have a good feeling on privacy policies, U.S. military launched aircraft and ended up taking this individual out because of a selfie. Immediately after that and all the forms and online, what we call TTPs, so methods, how people are using these things changed immediately to the point of tell everybody if they're using Twitter to post coordinates information and we find you that's a violation of law in the Islamic State and you will be persecuted accordingly. So my graphic on the bottom, this is based on a little bit more than opinion. This is based on a lot of research. What's happening in this space right now is I'm going to go through some research projects that will give you some fancy graphics and we'll talk about different populations within this domain. But there's a population of really bad guys and then there's a huge population of kind of moderates or people who are trying to engage in this conversation. And that conversation is starting literally right in front of us on Twitter and that's the little left side of the graphic with a little Twitter icon. And then quickly after two or three messages where somebody will like or follow or do one of those really easy to measure metrics, they'll go to direct messaging and they'll start to talk to each other directly. And that's another layer of kind of anonymity and another challenge for us to have any idea what's going on. Immediately after that they're pivoting into some really, really high end encryption technology which for all intents and purposes anybody who's paying attention and wants to get engaged in this space is completely blind on. So this part of my talk today and my ask of you guys to have a conversation and paying attention to this going forward is about the first part of this graphic of the conversations that are being had right in front of us. I'm not interested in talking about all the encryption and the Snowden and all that other stuff. I'm saying that we need to pay attention to the open part of this conversation before it gets to that crazy hard part. So why ISIL? Why am I talking about ISIL at DEF CON in the Yese village? I think that this is no longer a government problem. I think it's no longer a United Nations security problem. I think this is our problem. You know, I'm uniquely spend a bunch of time in the military and do a bunch of government work, but I'm literally not even pursuing any more of this type of work in the government space because we're bad at it. And I don't think that we can solve it. I think that we need to solve it. It's a people problem. And we have examples now of how crappy we are as a big huge bureaucracy with unlimited resources and having no effects. Some examples of that. So Xiaomi witness, I'm going to talk to you about three research reports and all three of them talked about this guy is the baddest guy. He's the most influential, loudest, why? Because he's got a lot of followers and he tweets all the time. You know what he was? He was like a banker in India, just kind of having fun. The next one, happened earlier this year on US Central Command's Twitter account, Cyber Caliphate. So it was branded as the cyber arm of the Islamic State Caliphate. And that's what we thought for a couple of months. So big bad US national security, completely ineffective, spins up on, no, no, a cyber caliphate. And they have actually done a bunch of other kind of published military family members and whatnot. Turns out within the last couple weeks that we've learned that it's probably some non-state kind of group may be sponsored by Russia, who knows. But the ambiguity around this space and how other brands are now starting to jump on this Islamic State kind of sled, it's a big problem. The next one is an example in the bottom right-hand corner of how ISIL, ISIS, whatever you want to refer to it, is using this command and control space very effectively and publishing out, hey, we just did this great thing for the local communities in Mosul. And they're publishing it, so they're communicating there. This space is important and it's everybody's problem. So LiTAC, I'm going to spend a lot of time in the next half hour talking about this as a human problem and people need to be solving stuff. But this space is the volume is so big and so fast, we obviously need to use technology. So this is a super-fanty graphic put out by one of the preeminent experts in this space, J.M. Berger, he's out of Boston, he's all over the news when it comes to this. Well, what the hell is this picture? I mean, what can you do with this? What can we do with this in real time? There's a ton of smart people in here that understand network diagrams and understand kind of exploiting or understanding and looking at dynamics of people. But if we're trying to engage in a conversation in real time, it's happening 24-7, this thing's borderline useless. And these are all the things that are happening and these are the ways that people are trying to deal with this space. So this is a people problem, but we got to use tech because it's so big. So I was sitting in yesterday, anybody in yesterday for Michelle's talk? So I'm sitting there and I'm watching her. She's way smarter than I am on some of the reasons people do things. And these are resonating with me. Each one she's talking, I'm like, yep, this is one of our issues on this problem. Yep, this one is, yep, this one is. So we make the same mistakes over and over. We always take the shortcut. That's what we've been doing on this internet thing for ISIS and before ISIS al Qaeda. We just keep doing the same thing. We think if we tweet and we have somebody that has 2,000 followers that we're good at, we keep doing the same thing, the same thing. Why? Because we have a tendency to do it. As decisions increase in complexity, we don't get better at them. This has got to be one of the most historically complex issues in the history of mankind. You've got fremer religion issues. You've got freedom of speech issues. You've got anonymity. You've got privacy. You've got a Fourth Amendment. You've got international issues. I mean, it doesn't get much more complex than that. And that's, I mean, that's a reality. And we hate uncertainty. Same thing. All of those complexity issues, there's nothing certain about this environment other than that the results we're getting at our current approach are completely ineffective. And then I'm gonna come back to the fourth point that Michelle made, which is everybody likely can be nudged. So we'll talk about that later. All right, so I'm gonna talk about a couple research articles. And these are kind of some of the state of the art people in the last like 12 months that is talking about how we're engaging in this space, how effective we are. What are some models that we can look at to affect it? This one was put out in early 2014. This is the oldest one. But one of the earliest that went in and looked just at Twitter and YouTube by an organization called Voxpol out of Europe, Violent Online Extremism, funded by a grant from the European Union. A bunch of people from Dublin University. And they went in and analyzed not really understanding what they were gonna do in their research. They just went and collected a bunch of data and classified it afterwards. So this fancy graphic post-facto, they collected probably like two million tweets over two months and then took like six months to hypothesize about it and talk about it and get peer reviews and broke that data into four distinct groups. And I won't read it, but it ended up with two million or so tweets broken into 672 users, 650, 670, something like that. Users that were categorized according to these four categories. And the big takeaway that I hope kind of maybe gets stuck in your mind as I'm talking for the next couple of minutes is that big blue group, okay? That big blue group is the group or a population. And you can extrapolate this graphic into almost any demographic, whether it's a thousand or a hundred thousand. And that's a pretty accurate picture of kind of how these factions are breaking out. Meaning that there's like a 60% contingent in the middle of all these conversations. And we need to fight better to make that 60%, 80%. And really push out from the inside the moderate conversations so people are making smart decisions. Next, the Su-Fan group, preeminent group that does research in this space, published a big report late last year, so less than a year ago, I think December or November of 2014. And these got literally some of the best in the world on this topic and this problem set. And their most fancy graphic is a tweets per day. I mean, thanks, right? What are we doing five years into it? And the upper right hand corner is, oh, go delete an account. Oh, shut that account. And in the same amount of time, they get a new one out saying, hey, I just got deleted. Please share my stuff and give me a new one. And they're putting out and publishing via Twitter very, very, very high-end marketing information materials. And they're telling very, very compelling stories that you and I might not understand from our perspective, but that really resonate back with that blue blob on the last slide, that moderate population of, hey, man, I'm having a rough day. I want a job. I want to get paid. I want to go travel. I want to go experience something. I want to get married. That's all coming out in very, very fancy, fancy marketing stuff. And the only way people are learning about it is through the hashtag ISIS. Okay, the third one. This is hands down, the most informative, most advanced look at how this face is being used by J.M. Burger and Jonathan Morgan who were published in March of this year after three months of research from September to December of last year. And it's kind of cool that we're cutting down the time penalty on when we're looking at these things to when you're getting information out. So now we're down from like six to nine months and now we're only down to three, right? So looked at, now we're not looking at 650. We're looking at tens of thousands. These guys looked at like in the 17, 18 million tweets, time frame or data inputs and narrowed it in and looked at around 40,000, 40,000 Twitter handles. And actually had some pretty good metrics in how they're analyzing and how they're filtering. But very similar demographic, you know, that population output. Most advanced, three months late, same output, cool social network diagrams. They look really great on slides. They look really great when these guys go to Capitol Hill and testify before every committee and go to Google Ideas and get more money to do more kind of coloring violet is extremism research. But we're just not affecting, we're just not affecting the space in real time. So I'm really, really sick of looking at the news every three or four days and seeing this explosion here. And then I used to take in credit for it via social media, via Twitter. And so, you know the whole, when you say something with all due respect and then but and you kind of don't mean the respect, I actually really mean that with all due respect to those current researchers and with these tactics that are happening right now, I really do mean that with all due respect. But it's not good enough because we're not winning. We're not kind of moving the needle. These guys, the researchers and then these tactics right now in a really small isolated bubble are actually probably pretty good. But it's not strategic. It's very immediate. It's very near term. So upper left hand corner, Jester. Probably everybody in this audience is pretty familiar with the Jester idea. Man, this Twitter handle and person has been around for years and super passionate about doing something in space. There's probably people in this room that knows that individual. But again, pretty effective on a local level. But he's gonna be employed for the next 20 years if we don't do anything different. Bottom left hand corner, Lucky Troll Club. Again, they're gonna be employed for the next 10 years if we don't do anything different. Identifying ISIS accounts and shutting them down. Right hand corner, I don't mean the respect part of with all due respect on this one. I think that this has been a pathetic attempt at doing anything. It's a taxpayer funded initiative with the Department of State. And as Jason said before, you don't portray to be a plumber if you're not a plumber. These guys are plumbers trying to be a communication harm. In my mind. Very, very poorly affected. Probably not even just not a positive effect on the space but it's probably having a negative effect in the space. Brought current tactics, not getting results. So when we met Chris a couple years ago, and we started working with him and Michelle and pulling them into environments that they probably hadn't spent a bunch of time in doing kind of national security type stuff. And he's like, dude, don't say that. I'm not supposed to be doing that. I'm very uncomfortable. And we talked to him about social engineering and how we apply the social engineering, building rapport and influence in the social media online environment. And we would ask him to do his disc profiling and then analyze online stuff in the context of social and digital. And we learned that it's very challenging to do that. But at the end of the day, why am I talking about research? And why am I about to talk about some theories that we can approach the space in? Because it's all about influence. It's all about communication. It's all about influence at the time that those moderates are talking to the influencers. And we're not participating. We're looking at it after the fact when they move from a blue to a yellow. So here's my sleepless passion right now. I'm a couple years into a PhD program in criminal justice. I'm a technology guy at heart and I wanted a completely fresh start. So now I'm reading all this historical stuff on criminal justice and whatnot. And one of my bends is to understand why these bad guys are communicating the way they do and why they're so effective. And how can I explain that to those around me in a way that will encourage them to help? So social learning theory, that's not just because we're in the SC Village, there's a very historical contract to understand why bad guys do bad things and how they learn to be bad. Social learning theory. Routine activities theory is what I believe is a really good contract to measure that conversation in. And I'll talk about both that, but at the end of the day, I'm trying to get people to understand the cryological behaviors so that we can impact the conversation in real time, not just come up with a fancy network diagram about a three months later. So this is the boring part, but I think important. This has been around since the 70s. Elba Bandora, Stanford professor, starts to talk about why do people learn? How do they learn? Can you learn via hands-on or via association? And there's been a lot of research that says, heck yeah, you can learn through associations and heck yeah, you can learn through virtual means. So a ton of research which statistically says that you can learn bad behaviors just by watching and just by watching others. And that's how it happens most of the time. So an update on that and to kind of more modern times, Ron Akers, University of Florida, he came up with another really good way to extrapolate theory into structured equation modeling so we can throw it into a statistical program and come up with good ideas about how to measure. And it has to do with four things, definitions. So coming into a conversation, what do you think is good and bad? What is your understanding of good and bad in the world? Differential associations, who are you already kind of aligned with before you start to have conversations? Differential imitations or, I'm sorry, differential reinforcements. So via proxy, via hands away, one hand away, are you watching somebody take an action and then see them be rewarded or punished? Very simple. If they're rewarded, they're gonna think it's a good idea. If they see set individuals to get punished, they're probably gonna take a step away. And then the last thing is imitation. So after some of those things happen, they start to imitate and you don't even know it, it's subconscious learning, social learning theory. So social learning theories happen at the individual level from one to one, one to one based on other things around you. Routine activity theory is kind of taking a step out at the 40,000 foot level and looking at where crime happens. You've got a bad guy. You've got somebody who can be influenced. So somebody who's a potential bad guy or in my case, a potential not bad guy that we can talk to. And then in that third instance, those two are facts. We got bad guys and we got potential bad guys or hopefully potential not bad guys. The third thing is what we can control is lack of an authority figure if there's not a good mom tapping on the shoulder of a potential person saying, don't do this. If there's not an effective department of state conversation which is completely ineffective, participating in that conversation, we don't have a shot and this scenario plays out every time. So going forward, and this slide's pretty busy, but I just wanted to throw this up here and it is a busy slide to let you know my research tying together a lot of modern, updated things and measuring digital crime and digital personal learning. How can I measure somebody in their basement watching 12 hours of YouTube videos? How can I measure the effectiveness of that individual learning in that environment? There's some good data out there that says I can. And it's a personal learning environment. How can I measure routine activities theory in today's kind of digital age where every data is dynamic and whatnot. There's some good stuff out there. And I've put some kind of layman's terms on who the bad guys are, who the suitable targets are and what the capable guardian could be. And I'm arguing that this community could ultimately provide that capable guardian lever. So what to do about it, you know? What to do about it? Super challenging, people keep doing bad things. We keep doing the same mistakes. Let's recognize first that this is a really, really like multi-dimensional problem, very dynamic in trying to break it down into really simple before we get to maybe some of the fun stuff, which is we're trying to solve this with some tech. Go ahead, just do the build. And so we've got said Twitter users, Jane Berger says there's somewhere between 40,000 and 90,000. We've got suitable targets, which there's hundreds of thousands of them. And then we've got absolutely a lack of authority figure because we keep whack-a-moleing and not doing anything. And I'm saying that could be our intervention point at the point of influence, right in the middle of this Venn diagram. So I'm hoping this conversation starts today about acting at that point of influence. One, by finding it first, because that's a really hard thing. And then two, by having a ready response at that point of intervention based on social learning theory and influence and communication, based on doing your research and pre-texting and having an idea of what to say and when to say it. Go ahead. So how do we find that first digital point? Well, we can find it and talk about it for six months or four months afterwards. And your Twitter handle is on version 153. When you analyzed it, it was on 73. And they just keep going number to number and number and advancing. We gotta find it in real time. We gotta look at it in real time so we can respond in real time. So here's what we're talking about, the human-enabled machine assistant. I'm here to tell you that we're sending tens of millions of dollars on this problem right now. We're procuring technology that is like a great supercomputer. We're paying companies in Silicon Valley millions of dollars per year to throw really fancy social network analysis and social media analytics at it. And it's all big data approach. And we're trying to say now via smart data, via real time, via kind of decentralized approach, human-enabled machine assistant. Go ahead. So that really complex problem of radicalization, I don't claim to have all the variables. I just know that it's very complex. I know I need other people to help me out with that aspect of how to communicate and when to communicate it. In both sides of the spectrum, people have really loud, varying opinions of, hey, I can buy Palantir and throw Palantir at it and it's gonna needle in a haystack, we're good to go, serve and rule on. There's also some that say, don't do tech. We gotta build roads and we gotta build schools and we gotta do education and we gotta get jobs, just via the human aspect. And it's somewhere in between. We gotta be in the real time fight. But I would argue largely this is more of a people problem. So just like Robocop, right? Somewhere in between, Robocop. Still pretty cool. And we're gonna talk a little bit about the process in a free, freeware script. But I would argue that today, the bigger picture, the bigger takeaway is what I would call kind of revolutionary in today's time, is the fact that we could influence this space, we in this room, in this community, and everybody that leaves here that talks to 15 people and we go from one to end, that's the big deal coming out of here. And so find the accounts now and then be ready with an informed approach to have that conversation and be that voice of reason and then authority figure. So how do we do that in real time and be ready to respond? It is actually a pretty simple process. Anybody could go on your phones right now and take two seconds and start to find things. But at volume, super tedious, super tedious. So instead of a millions of dollars, instead of Capitol Hill, instead of Congress and instead of United Nations, let's look at freeware. Let's look at people. Let's look at ideas. That's what this whole community does so, so well. This is why I've got Yannis up here. Here's where it starts to get complex. And this all came to be one day in the office when I'm opining about my opinions on all this stuff and he's sitting there not saying a word, just looking at me like shut up and move on because I talk too much about it. But he starts to put math to this problem. And so this is a machine learning approach at how we classify this information and spit it out relatively simply. And this is a very simple approach at how you classify something according to other events. So the probability of something happening be dependent on a variable A. And not to get into the math of it, but the critical takeaway here is naive base is very simple, it's very adaptable, it's iterative and relatively simple to understand the results on. And that's what this process that we're about to talk through is built on. So how are we doing it? Naive base depends on establishing a baseline library of data from the same domain that we're about to go measure. So we go out and get a couple hundred and similar to those other research theories that say human, go measure this. So we do that, we go out and we build a training set. That's our baseline classifier. That's not dependent on ones and zeros and potential errors that's based on us having worked in the space for the last 10 years saying this attribute gives me a pretty good probability that this account is a nice little account. So we build our training library first step. Next step is that we process this stuff and we take it in and as we probably all know in this group that raw text that comes out of Twitter there's a lot of crap in there that's not relevant to kind of the intent and the content and what some people would call sentiment. So we recode it into something that can be measured the same way every, every time to have a pretty good probability going back to that math equation that what we think our baseline is that compared to these new variables and now I'm gonna have a pretty good guesstimate from an automated standpoint at volume of this account is said bad guy I wanna be paying attention to the conversations he's had and we're training and we're retraining that baseline classifier data because our outputs of this piece of software and script is only as good as our baseline library. So after we get our baseline library we're going out pulling accounts interpreting them according to that baseline library and retraining it and retraining it such that there's an association of set account. This is a bad guy, what's my percentage classification of likelihood so I can potentially engage in that and also maybe look like a bad guy but not with again a high probability and we're pulling this via free API, free Twitter API free inputs, not tens of millions, not any millions it's people doing work with freeware and after that is kind of where today's gonna stop there's a lot of different things that we could do to take action on an individual level when you're instead of donating some time to the local community maybe donate some time on this stuff where there's a whole bunch of ways that the social issues people can impact this space and that's part of the things I'll talk about at the end but engaging with that in a social engineering influential way. So that was the process and then we started to build Sick at Learn, it's a Python language machine learning library, a lot of historic on effective use of this build and effective use of interpreting data with high levels of probability. Slowly parsing Twitter, what is it, every six seconds we can interpret an account every six seconds, so if I extrapolate that into what some of the best have said I've got 100,000 plus or minus 50,000 that's only a couple weeks, a couple weeks of work on a local CPU where we can establish that baseline library, so we're talking even with low processing power and a moderate approach of having a good idea of what's going on in this space and I keep emphasizing this that it's free and I say this from, I got out of the military in 2008 and we wanted to, from a software guy we wanted to build an ex-fest social media analytics tool and we tried for a couple years and learned that we were miserably behind what was state of the art and whatnot and so I've come full circle on solving problems in this space is about tradecraft and it's about a creative approach and it's about using a whole bunch of different things to do it, to kind of meet your objective and so this approach, this code, this information is free, it just takes people that are passionate about it with some time. I'm not so great at some of the specifics of what the output of the script is, the honest is and we'll be around after this to talk specifics if you're interested in some of the code writing but the takeaway on this is that this freeware put together in a couple months based on a framework that we developed based on historical criminological theories is accurate somewhere between 90 and 98% and it gives you that feedback in real time so it's really enabling the human with machine to be effective at a pretty high rate and so here's some quick examples from a data set that we've just run and pulled and I said to you honestly, hey, this is gonna be cool and we maybe wanna do some live demos but I'm worried about the network and worried about getting hacked but we gotta have tangible stuff here and one kind of important kind of cool note there's obviously images associated with what we think and see on the news but we were really happy to see actually the one in the bottom left which was a non-icellar account which was ultimately a parody and that thing was able to see it because of that 90% accuracy thing so this is a data set of in the hundreds and we're pulling down handles and groups of 20 to 25 operating between 90 and 100% accuracy so coming kind of towards the end and what I'm sick of being in a position of just talking about the problems and looking at the diagram so like what could we do? All the data and examples that we've done to date have been on English language so we could start to classify stuff in foreign language we could have different classification inside of just staying at the 30,000 foot level of this is an icelar account and we could start to talk about things like hey this is a recruiter, hey this is a fundraiser, hey this is a travel and logistics handle that would better inform our communication approach going forward. We could build our library from several hundred to several thousand so accuracy is not between 90 and 100 it's between 98 and 100 with time and we could do this kind of together but we could have a decentralized response arm so that everybody's pulling from the same library and the same baseline and responding in the same way and adding to it and learning and we're not constrained by bureaucracy and government and all that other stuff and we could engage in that conversation and kind of beat that intervention point from a built upon social learning theory construct where we're communicating, understanding associations, reinforcement, definitions of what's good and bad, challenges and we could hopefully get them to imitate alternate behavior instead of that little yellow bubble on that first graphic. So, you know, when Chris and I were talking about this and I came out here and I'm one of my, you know, I don't want to spend my time talking and bothering you all about as you get to the fun part of the SC Village with Hadnaggy and the guy before me and this is a little bit of my soapbox on something I've been working for a number of years with high frustration levels of I would love to get this community involved. I know there's so many people here who want to solve problems. You hear everybody talking about I'm a good hacker and I want to do good. This to me is one of the biggest kind of pandemics that we have that is like been around so long we're almost complacent and so I'm hoping that people can see this in different fashions and you guys can talk about it a little bit and come research with us and then going back to Michelle yesterday everybody can be nudged. So if you got a couple spare jewels of energy come wasted on this problem. And we'll be around and happy to take questions and Yannis will be around to answer the hard questions on the software but I appreciate you guys listening to this topic in this venue. Again, I know it's an anomaly but I've been following on Chris's ear for like years to give me 30 minutes in the room and he finally relented. So thanks. Yeah. There's this one to rebrand the ISIL. Yeah, I have. Yeah, the question was that he's seen a movement in Japan where they rebranded some of the ISIL stuff with ISIL or ISIL Chan hashtag. And if I'm not mistaken on that it backfired pretty quickly. Oh, it's like a flop. Yeah. And so that's kind of a, it's a great question because it's one of the points I'm trying to make which is there's so many little factions of people trying to do this. They, you know, in the tactical they're probably effective for a couple of days and get trending and stuff but in the aggregate it's not, we're not doing anything. Yeah, in the back. So the question was, we talked about how we find them but when we find them what do we do about them? So I wanted to focus today on us kind of changing our mind frame of instead of looking at the research three months after the fact and figuring out what to do and when you don't even have a chance to have a conversation and my kind of last bit was here's a tool to go find them in real time and that next kind of global step of taking action according to principles that everybody in this room is interested just by definition of being an SD village, understanding that this is a sophisticated conversation of influence and so that taking action part is about understanding that people learn according to definitions, reinforcements, imitations and you have to build rapport and you have to pay attention to rapport and influence. So what do you do? Go have a conversation from your perspective that counters the only one that these folks are hearing right now which is we're awesome and we're gonna pay you a job and we're gonna pay you for the first time in five years. We're gonna give you a life. We're gonna give you a purpose. Just go have a conversation and to me the first step is it's literally as simple as that. Yeah. So we're gonna follow up on sort of that interface so let's say I'm using your algorithm I've identified an ISO recruiter, let's say. I assume you're not suggesting that you want me to just mention him like hey, buddy, stop. I'm assuming your idea is more take a look at who he's getting with. Yeah, that's exactly right. That's exactly right. Question was okay, so I've identified a recruiter. We're not suggesting that you go have a conversation and try to change the recruiter's mind. I'm suggesting that you go pay attention to who the recruiter's talking to and have that conversation in real time with him before it transitions from an open kind of bland recruitment of envision them sitting at the corner and coming by the recruiting station. As soon as they do that and they take the guy inside and have the conversation in the office, we're gone. We don't have a chance at it. I'm saying come stand at the recruiting station and be the other person on the shoulder. Yeah. Have you heard all about this? Yeah, I lost a question and that's actually a point I wanted to make. So there's, Twitter's a business, right? They're a business and there's a lot of recent things that are saying like Twitter needs to do more and Facebook needs to do more, no they don't. I mean, they get so much pressure and their stock price is going to the tubes right now and part of it could be because they get pressure on this topic right now. And so I know I haven't personally talked to Twitter but I know very influential people and our national security operas have sat down with Jack Dorsey and had this conversation and it's not a Twitter issue. It's a people issue. Twitter can't staff the volume of stuff that we're talking about here and run a business. Yes, sir. So you said require military, whatever, military? Navy, submarines. So you have a military mindset or are you taking a quite a step forward or a public center mindset? I'm not sure I understand the question. But what's happening with the military mindset that you require military? No, I think I'm- I'm a mentality because not everybody understands what it is to go out and put themselves in in a position to go head to head and communicate online is not a deterrence. Yeah, I'm not saying go head to head sir. I'm saying participate in the conversation. But do you take the military mindset in this matter? From a, from an OPSEC perspective, I think that there are very good ways to have a conversation from a social perspective, not a targeting perspective. That was my argument at the beginning of this is a social conversation. So- Political in control. No. What would be the type of political opposed to those? No, I think that this is a human rights analogous conversation that needs to be had in the online environment. And the only people that are doing this historically are those in the military and the government and we're bad at it. That's my opinion. Yes, sir. So can I just talk to you a little bit about the success of the darkness program next? Do you see anything on that? I'm not familiar with that. Thank you. Yes, sir. I'm concerned about the ramifications of what you're suggesting. The lack of the effort to do the process in terms of who becomes the authority to counteract the students' bad? And what is your definition of the type? Do you think so? Yeah, so he's saying he's concerned about my ramifications of what I'm suggesting and who becomes the authority on what's bad? In the lack of the- In the process. In the process. Great point and great question. And towards the, to me, this conversation is not to debate freedom of religion, freedom of speech. I think that there are very few people that would argue with a person that is encouraging the sharing of the heading information and attack on Western people, Western facilities. That's the definition to me in a very black and white. What's bad? Yeah, and I think that- On this, to me, this is about ISIL. I'm not trying to be the judge and jury. What if it sounds like that? Well, how does it sound like that? There's no due process to identifying, you know, what if it comes through the criminal justice system? Do you mean the lack of the good? You're extrapolating what I'm suggesting like steps two through 10. Yeah. That to look like a certain kind of case on the books you get, it kind of seems like a mob mentality. No, well, I would say that to compete with the mob mentality that we're fighting right now, I think that I'm trying to say that it's very obvious to me when I look at Shami Witness, who is an Indian blogger and he tweets about every kind of social aspect of the Islamic State from afar, and we just watched that happen. And it's very obvious to me when they're sharing the heading videos of, you know, pilot or this, that's a black and white thing. So that's the best you have to look at them, putting your videos like this, but that's not how it's going to work. Yeah, yeah. What are your thoughts on this? Like I think that this is a healthy debate, by the way. And I think that's one of the, this community really, really can participate in very effectively. So I don't mind the slingshots at all. I, in fact, appreciate them. And I think that we can get the problem solved. Yeah. How do you try to measure the difference? So, you know, with that, you know, how do I know I'm not wasting my time? Awesome question. Yeah, the question was, how would we measure effectiveness of like input into this conversation after a year's time? Have we had any effect? Great question. I think that it would be something that I, same to the technology that we have to use, I think that it would be dynamic and iterative. But I don't know. And I'm really comfortable saying that. And when I said the why I sold, I think that we've got a couple of good years of experience on, you know, these are different approaches we've taken and it hasn't worked. So maybe it's time to try something different, you know. So, I don't know, good question. Yes, in the back. Yeah. So I think I followed your question of, hey, how could I, from 5,000 miles away, be an authoritative or informed voice when it's probably a little bit more of a local problem? Is that fair? Yeah. I agree with you. I think that this needs to snowball and this can't just be a Tim Good idea. If it needs to get, you know, interpretation from other people and do this is stupid because, you know, it needs to be, it needs to be local and needs to be, I think this needs to go into something that's larger than just this room and just Twitter. And, you know, my point is is that the information component of this movement is one of the most significant and there are concerns about engaging with bad people. There are concerns about how you do it from an informed standpoint. But for us to, you know, Jason ended with, for the triumph of evil, good men can do nothing. And there are a lot of good folks in the government and normal populations doing their best here. But again, this is a snowball that hasn't stopped, hasn't stopped rolling too much in the last couple of years. Yes. So if you talk about the possibility of crowds or saying, do you all get really good talk? Yeah, I think we would be. Yeah, I think I want to talk to people about it. But yeah, I think that we're heading that direction. Yeah. Yes, sir. Excuse me? Yeah, to elaborate on a little bit about what the classifiers are. Yeah, I think, what are we as researchers using for the classifiers? I think it's some of the stuff that I was responding to this gentleman with. We're trying to be a very black and white. So there's very few opportunities to get it wrong in the wrong direction. Very black and white of, this is a published account, you know, not just interpreted by me, but by Al Hayat, it's a published account talking about published things. So from the black and white standpoint. I'm having a hard time hearing it, sir. When we built the test set, so the baseline library, that was us manually researching Twitter lists and known publishers and pronounced and pulling those attributes in class divining according to that CICCET learn, Python machine learning. Yeah. So that was a manual, that was a human thing on the first side of this, okay? Yes, sir. What's the bigger role of this? So not just from my sister, I saw this. It's the bigger role of this and with criminology in general? Yeah, government taking it to try and find. I'm not sure, I mean, I'm not suggesting that. I'm suggesting just the opposite. This is a people thing, not a government thing. Of course, like an era. Yes, but that already exists. Yeah. Sure. But again, this is a month worth of open source code and some ideas, you know? I mean, I'm sure there's a lot smarter people than me thinking about that stuff. Yes, sir. I think the first answer is yes. And again, pulling it back into, I'm not suggesting targeting, I'm not suggesting hunting and pecking. I'm suggesting pay attention to the conversation and potentially participate in it. That's all. Is that it? Thank you. Very much.