 Well, everyone, if you could take your seats, please. We have our final speaker, Anshul Regeg. No, Anshul Regeg. Anshul Regeg, like the music. No, it's not. Oh, that's why. OK, I'm sorry, Anshul Regeg. Thank you. She's a criminology professor at Temple University, and she's passionate about educating the next generation workforce about social engineering for social and heart sciences and the relevance of the human factor in cybersecurity. Let's give her her attention. Can everyone hear me OK? So also before I get started, I'm very excited, because this is the first talk that I'm giving after I was awarded tenure a month ago. So now I don't have to care about anything, right? It's great, yeah. So a little bit about me. So I've been at Temple for about six years now, and part of this is a little bit of a story about my own journey as an educator, right? How many students in the audience? Right, OK, so this is something if you're considering further education, if you go into your master's or your PhD or things like that, right? The focus is always on your research. Nobody ever tells you how to teach, right? And so all of a sudden, we're thrown into a professor position, and the expectations are, yeah, you do your research, you publish, you get grants, but you must also teach. And nobody ever trains you properly for that. So this is something that I'm trying to figure out. It's almost I have to learn how to be an educator. So what I'm trying to do today is share some of my own experiences doing experiential learning or hands-on learning in the area of cybersecurity. And this started off as a project for non-technical students, because I'm housed in liberal arts. And so my concern was, how am I going to get the liberal arts folks to be able to think about cybersecurity even as a career option? So this is very much a work in progress. I didn't have a foundation to build on. I didn't have any form of assignments I could look at to design the rubrics from scratch. OK, so this is very much an experiment. And what I'm trying to share with you are some of those challenges and struggles that I faced, as well as what my students thought about some of the projects that they had to deal with. So I got to start off with my thank-yous, the National Science Foundation, the way that NSF works, if you're not familiar, is they fund you for research, but they're also interested in looking at what you do for education. So what I'm sharing with you today is the education side of that grant. I want to thank the Temple University Office of the VP for research. This is where the ethics board is housed. And for all the social engineers here or the folks that are considering social engineering as a career path, when you're trying to do something in a university setting, the rules are very, very different. So the ethics board was really instrumental in working with me and making these projects happen. I want to thank the university's IT services, because they actually engaged with my students during one of the flags, which is really cool. And of course, I want to thank human hacker. I think he's still out there having conversations with people. But he was really instrumental in this as well. He helped me design one of the flags for my classes. And I think if I didn't get that sort of voice of reason and support and enthusiasm, perhaps these projects wouldn't have taken flight. So what's on the agenda? I want to tell you quickly a little bit about me and what my concerns are as an educator. And where did I get this idea of trying to do social engineering projects in class? Then I'm going to talk about the three flags that we tried to do in class. Shoulder surfing, a group selfie, and laptop distractions. So I'll get into each one of those in a little bit. I'm going to talk about the summaries. They're based on, again, student feedback. And end with some of my own reflections and closing thoughts, right? Why does this matter? Where would I like to go next? And all of that good stuff. So let's get started. So yes, I'm a criminologist at Temple University. For those of you who don't know where that is, it is in Philadelphia. And I've been there for six years. And my main area of research, my funded projects, look at proactive cybersecurity with an emphasis on adversarial behavior. But I'm not here to talk about research. I'm here to talk about education. And so in the six years that I've been at Temple, I've been teaching an upper level liberal arts class called Computer Prime. And for the first three years that I was teaching this class, I followed the same approaches that I had been subjected to as an undergraduate student. Go research a topic, write a paper about it, and then do a presentation at the end of the semester. Great. And everything was going fine. Every semester I would ask my students, so how many of you would consider a career in cybersecurity? And they said, well, we can't do that. I said, why not? Because we don't know how to code. We don't know how to hack. And I'm saying, yeah, that's important, but you also need to understand that who's doing the hacking is a person. And what do we do in liberal arts is understand human behavior. That's what we're trained to do, be it psychology, sociology, criminology, anthropology, all this stuff. That's what we're trained to do. So you know what? We do have something to contribute. But it's set of alarms. For me, as an educator, I'm doing something wrong if I'm not able to tell my students, you know what? You can pursue a career in cybersecurity, and here's why. So how do you start changing that mindset? So I started looking at my hard science colleagues, because they do work extensively with computer science and electrical and computer engineers. And they were way, way, way ahead of me. There are students already have class offerings in the area of pen testing or digital forensics or things like that. So they're actually training students to go out there and pursue careers in cybersecurity. Here's a real sickler. What does that look like in liberal arts? How do you get hands-on, experiential learning for the non-technical students? And so this had been brewing in my mind for about three years. I got my first grant, right, yay, step toward tenure. And again, like I said, this was looking at proactive cybersecurity. So one of the things I look at is the cyber intrusion chain for those of you who don't know what that is. It's literally a step-by-step process of how an attack unfolds. And I spoke with a lot of ethical hackers, pentesters, and I asked them, you know, which of these stages is the most relevant? What do you think they said? Right, recon, okay? So we have recon, weaponized delivery, exploit, install, C2, action, and objectives, okay? So one of the quotes from one of the pentesters I interviewed really captures it nicely, right? 50 to 75% of the legwork is to learn about the environment ahead of time. Be it through social engineering, calling these people up trying to understand what systems they operate, which is what we saw in the CTF, researching the vendors, right? All of that, oh, sent. Okay, so if this is so important, why are we not educating students about this, right? What can these exercises look at? So I know I have an idea, but I don't know how to implement it. And then last summer, I got invited to go and speak at Estonia Cyber Security Summer School. So they have the summer school every year, and last year's theme was social engineering. So they said, hey, can you come in and give a talk on your research on online dating scans? I said, sure, right? So I go there, and what they're doing is this is the first pure social engineering CTF that I've seen at an international level, okay? Where you have students from different parts of the world coming in and competing in these flags, and they're doing all these cool things that are listed on here, right? OSINT, creating fake profiles, generating phishing emails and actually disseminating it, shoulder-surfing laptop distraction vision, and I was like, oh, wow, why did I not think about this? Right, so I'm so inspired, I'm excited, now I have some ideas, I would come back to Temple. What happens? Can I implement this? I'll have an ethics board, okay? When you are dealing with a student population in a university setting and there's deception involved, the ethics board just said, nope, can't do that, can't do that, can't do that, can't do that, and so it took three months to work with the ethics board and come up with flags that were acceptable, okay? So the two that I could port over were shoulder-surfing and laptop distraction, that's when I said, okay, well, this is great, but I need one more flag, and so last August, I reached out to Human Hacker and I said, hey, I'm doing the social engineering project, can you help me out, right? So we started talking and we went back and forth and said, can you try this? I'm like, no, that won't be acceptable, and we ended up with the group selfie flag, okay? And so he said, you know what, give your students a prop of a squeaking rubber chicken and send them out on campus, and what they have to do is convince someone to take a selfie with that rubber chicken in them, all right? He said, all right, run it by the ethics board, they said, that's fine, it's safe, it's fun, logistically possible, and so I said, all right, I got my flags and I have to now try this out again. I had to design the instructions from scratch, the rubrics from scratch, but I had my first project ready, right? And so I decided, I'm gonna try this out on my fall 2017 computer crime class, so this was my guinea pig class. And just to give you a feel for it, I got 34 students, 28 in liberal arts, and I had six computer science students. The projects went fine, there were a couple of glitches which I'm gonna talk about, but they gave good feedback, and I learned a lot, and so tweaked the assignments a little bit, and did round two in the spring of 2018 semesters with about 29 students, 20 liberal arts, nine computer science. So what happened? So first flag is shoulder surfing, okay? And basically, students were broken up in teams, and they had to compete against each other, the goal being you have to be able to get a clear shoulder surfing shot of someone from a rival team, okay? So you cannot target anyone outside of the class because everybody in the class had signed disclaimers. It's like, we agree to this activity, right? Everyone had done their ethics training. So we kept it just to the class, and they also were limited to on campus only, so no one you couldn't follow your classmates home, okay? That was off limits. So I needed to make sure, though, that they weren't cheating, so how do I know that it's you who took that picture? So they had to provide what was called an action shot. Okay, so here's a picture of a shoulder surfing successful shot, okay? So they were able to get someone from a rival team, got a picture of the image, this was in a computer lab, and here's the action shot, okay? So this is proof that it was the student who had taken the picture. So students had two weeks to do this, and after the two weeks they had to give a debriefing, if you will, a five minute presentation about how did they go about doing this, what were the strategies, and so on. So some of the strategies, and these are, I put these into the offense category, right? So in class they started remembering who was sitting where, what devices they were using. Arriving to class early, trying to take a bathroom break during class to see if they could find anybody on their devices at that time. The fall 2017 class engaged in something called a honey pot, and I'm gonna talk about that in a little bit. And outside of class, so this was during class time, outside of class they engaged in what was called light stalking. So they actually followed their classmates after class, kept a safe distance between them and their targets, tried to remain in stealth mode. Some of them even cross-referenced to see that other classes with folks from the rival teams, they could target them in other classes. Overall they said, be patient, persistence pays off, and when the opportunity presents itself, act quickly. So I'm gonna talk about these two, the light stalking and the honey pot. So here's a picture of someone from the team, who is following this target. And he followed this person for about 20 minutes, if you're wondering where he is, that's a shadow. Okay, so he's a good 15, 20 feet behind, followed him for about 20 minutes on campus until the target stops at a food truck, and that's when he was able to get the successful shot. Now, that was one approach, the other approach was honey pots. So what would happen here, and I only saw this in the fall semester, it's almost like each semester brought its own unique strategies, right? I did not influence any of this. Students could try out, they'd run the strategies by me, but they were allowed to be as creative as they wanted to be. So here we have someone from team A comes to class, sits in the front, opens up her laptop. Someone from team B comes in and says, oh, this is awesome, I got her. All right, pulls out the phone and gets a picture. Okay, so I said, why did you do that, right? Like you guys got your own picture taken, they're like, yeah, yeah, we know, but yeah, so we let them get a picture of us, but we got the picture and I was like, okay, interestingly, this ended up turning out to be in team B's favor because team A basically provided the action shot, right? So there was another instance of the honey pot, okay, again, where you have the bait, and I'm kind of setting this up because it might be harder for you to read. It's a quick animation that one of the students took, right? So he has someone sitting down, someone from a rival team come to take, to show that he's got somebody else, right? So he's bragging and that's when they get him. So there was a lot of honey pots that were utilized and I did not again see this in the next semester. So some of the defense based strategies, okay? Everyone was on a high alert. So yes, you had to target someone while also being targeted. So there was this sort of spy versus spy atmosphere. For those two weeks, nobody used their devices in class. Every professor's dream come true, right? So it was great. You saw changes in human behavior, okay? So they changed seating location. They said at the back of the class, okay? By the walls, okay? They positioned themselves sideways and they tried using sabotage. I couldn't find a proper term for this so it's called sabotage and this was unique to the spring 2018 class. So I'm gonna give you the example of what I mean by that. So here's the action shot in progress. Okay, the guy with the laptop opening in the blue shirt is a target. Can you see the sabotage going on? Now do you see the sabotage going on? There's a zoomed in version. All right, so they got the picture and then I get an email that night. It's like, hey professor, are we allowed to use this? And I was like, sure, right? Why not, why not, it's fair game. So during the debriefing, they actually shared this picture with the class which is quite entertaining. So some of the challenges that the students faced. Because this was the first flag that we did, people couldn't remember, students couldn't remember their classmates, right? Because new faces don't remember the names. Sometimes they followed the wrong people which didn't go over well, okay? They didn't like the fact that they were limited to on campus, but I said, sorry, those are the rules. Trying to get a clear picture, like I said, they had to act quickly, there were a lot of blurry images. They weren't always successful. Like I said, targeting someone well yourself, potentially being a target was also problematic. They felt uncomfortable and creepy engaging in this activity. Like I said, they could only target each other. But when they were doing this outside of class, people would look at them really funny, right? And so students were like, can we get some special t-shirt or a letter from you that says that this is for a class project? I said, no, right? So there was that feeling of being weird. The last sort of one that was really interesting is coordinating action shots, right? So outside of class, people in your team aren't gonna have the same schedule as you. So how do you get someone to prove that you took the picture? And the spring 2018 class came up with a solution. They said, we're just gonna do selfies. And so that's what they did, okay? So that was a workaround on the action shot requirement. So they got the shoulder surfing picture and they proved that they were there. So this was their first activity. It was a good sort of warm up flag in both the semesters. And then we decided to go with the next one, which is the group selfie with a temple citizen and an awkward prop. So this was a flag that had been recommended by a human hacker. And for this, again, this was team-based. So each team had to give me their script ahead of time, which is what they plan to use to approach someone and convince them as to why that person should take a selfie with the team and the rubber chicken. So the scripts were sent to me two weeks in advance. I had to make sure first of all that the script was okay. And also let no two teams have the same script. To ensure that there would be no conflict in scheduling, I gave them dedicated class time because that's when everyone's supposed to be there anyways. So they got about two dedicated classes. Once they approached someone, they couldn't say that this was for the class project till after they had gotten the selfie. And again, this was on campus. And two, yeah, they were given two classes, another week to get their debriefing done, which basically meant they had to create a short video about their experience, right? So I'm gonna share some of the videos that my students made, okay? And I thought they did a pretty good job. So hopefully the audio works. I'm gonna try this out. So for our first thing, we did frat boy pledging. Justin's gonna be the pledge. And his pledge is that he has to get selfies with people holding the rubber chicken. We decided to meet outside of the bell building. And then of course we run immediately into someone we can't ask because it's Leon. So being in our class, we were not allowed to ask him to participate. Justin got out the rubber chicken and we sent him off to attack students and other people who were passing by. Then pretty much we just told him to go get someone. So we did. He walked off to the nearest strangers and approached them. Some of them didn't even give him the time of day. I guess that's not surprising given that's the city. Sorry, I don't have any spare changes. Pretty normal reaction when someone approaches you randomly in the city. I don't really blame that guy. So Justin just kept walking to the next person. Got to give it the old college try because in the first year to succeed you keep walking down the street and you keep asking everybody you see. And then eventually if you're lucky and persistent you reach to nice people who say yes. So Justin brought those people back to Justin in front of the bell center. And we explained that he was pledging that we needed to get a selfie with the rubber chicken and here we go. We got our selfie mission accomplished. And this is one by another team. Don't they have check-ins? It's our targets. We wanted to come up with a strategy that would engage them without being obvious about our intentions. Ever since we got at Chick-fil-A on campus they've been noticeably popular. The lines for it are super long so we use the popularity of Chick-fil-A to our advantage and make more sense to have a chicken related strategy in order to get people to take a picture with us and our rubber chicken. So our story was that we are prankster YouTubers who wanted people to do the chicken dance for our channel. However, we didn't want to be obvious about that so we came up with three chicken challenges and we'd reward the target with chicken nuggets from Chick-fil-A. So to prepare we looked up some chicken facts from what was originally supposed to be a quiz and we went to the Student Activities Center and bought Chick-fil-A nuggets for our reward. This was during lunch time outside of the Student Activities Center so it was fairly opportunistic to find some hungry college students. I made one announcement stating that if you completed our chicken challenges you could win some free Chick-fil-A chicken nuggets. One target came up immediately asking what he had to do for free food. I told him that all he had to do was pass the chicken challenges. I asked him if we could take a video for our YouTube channel to which he said yes to promptly. The first chicken challenge was to answer a chicken question which was what is the male chicken called? That is a rooster. That's like... Just sort of remember this scene, I'm gonna come back to it in a little bit. Again is to demonstrate what sound a chicken makes while doing the well-known chicken dance. The third was to take a picture with our rubber chicken. He completed all the challenges successfully and we joined in the frame for our selfie. After our selfie, we told him about the class project and he walked away happily with his free chicken. As far as lessons we learned, we learned that we couldn't be around too many of the other groups with chickens. We all went about the same time during the dedicated class time so it was a little conspicuous and noticeable that something could be going on. Another lesson that we learned was that if you incentivize an activity, it'll bring more eager targets, which seems obvious but while you, if you know your audience as well, like college students, they're gonna be more willing to participate and won't likely do anything for food, especially if it's around meal time. And the third lesson that we learned was that people don't really care if they're on social media with strangers and how easily their information is obtained. Like we didn't do a wave or anything like that so he unknowingly just gave his face for our YouTube channel without, you know, knowing it could be abused or anything like that. So a couple of glitches. Okay, each team was given the same prop and they were out with the same prop at the same time. This caused problems because like I said, the guy, remember I told you, remember the scene where they zoomed in, he was actually summoned from a different team. Okay, the problem was that they were approaching targets with the same prop in different stories at the same time, right? So potential targets are gonna be like, what's going on? So that was a problem and one of the feedback obviously that I got from the students was, hey, can you please use different props for the next semester? So sure enough, I did. I kept of course the rubber chicken and then I got a giant stuffed Minnie Mouse, a sock monkey, a Hello Kitty pillow, swimming old llama and a unicorn. Okay, this was the most assorted mix I could find in a short amount of time. And so again, I'm gonna share with you a real quick the unicorn video and this is now for the spring 2018 semester. Here's a personal favorite. We went in with the attention of having a staring contest with the unicorn. Little do we know, it would turn into one of the most rewarding experiences we had. We used the unicorn as bait to get our target to have a staring contest with Rob. Initially the goal was to have a staring contest with the unicorn, but in all fairness to our contestant, we went with Rob. A clock can get the... I forgot my timer. Put a timer on, let's see how long we can wait. All right, so, get your eyes already. On, on, on, on. Feel like it's hard to stop, oh, it's snow. Do you remember? Yep. Yeah, she blinked. She blinked. All right, so you don't get to win the unicorn. But would you mind if I took a picture with you? Be there, yeah, yeah, yeah. Okay, so some of the strategies based on student debriefings and reports, okay, so how did the students go about doing this? They first decided to be cognizant of what potential targets were doing. What are they working on? Are they doing homework? Are they surfing? Are they eating? Are they chatting with their friends? They had to stick to their script, but they also had to be able to adapt based on that script. So, for instance, the group that got the stuffed llama, they came up with the script of Save the Llama Foundation. Okay, so a lot of students were interested in that and they asked them a lot of tough questions. So, they had a good story prepared. Okay, they were able to adapt. Some of the strategies, like I said, pretexting obviously, quit pro quo, right? We saw that with offering food for the target's time or to get the selfie, playing on the emotional card, right? We wanna save the flamingos. We wanna save the llamas. So, there's that desire of wanting to please. Some of the challenges, like I said, getting out of the comfort zone, right? So, going out and actually approaching people was interesting. The spring 2018 class was scheduled for nine in the morning. So, it was hard to find people on campus at nine a.m. And even though the props had changed, the teams were still bumping into each other, even though now with different props, but it's still kind of wise everyone walking around campus with different props, right? So, that's something now that we have to figure out. The last activity, the last flag, was the laptop distraction flag. So, again, this was team-based. And what happened here was folks from the information technology services came in with a laptop. And the team had to distract the representative from the laptop, okay? That was the objective, using a story and whatever you wanted to do to adapt. So, they had to come up with a script, which again had to be sent to me in advance to ensure that no two teams would have the same scripts. And we did this in class, all right? So, the IT representatives came in class. We dedicated the entire class time to this, so about two or three classes. And they were given two minutes of prep time, five minutes to actually execute the activity, and then they had a two-minute debriefing with the IT services representatives. So, just a note here, okay? They were not graded on whether or not they were successful in getting the person away from the laptop, right? What they were graded on is what strategy did they use? How convincing were they? Were they able to adapt in those five minutes? So, again, there's a whole bunch of different videos, but I'm not gonna be able to share them with you. So, the one that I am sharing with you is from the fall 2017 semester. This was a two-student team. And I like this clip a lot because their execution was very, very smooth. They stayed calm and remained in character even when things weren't going their way. And they adapted in the best possible way that they could. All right, so it's a two-student team and the folks that they're targeting are against the back wall, okay? So, when you see this video, you'll know what's going on. It may be a little hard to hear because they were moving around all over the place, so I've tried to insert cues where possible to help guide you. How's it going? On Christopher Daddy, this is Brian Shadden, we're here for the interview. With, uh... With you tomorrow. No, with you. We made an appointment. We called you last week to schedule. Yeah, they did, and they almost fired. I forgot to tell you, and we scheduled for an interview appointment. Oh, okay. All right, how's it going? Sorry, I've been set up in the back door quick, so we can just run by every two, five minutes. Just did an interview and we'll be right back. Okay, all right. Can you give me some context, just so I know? Yeah, so tell me, make me a new website, what's new? And the inheritance of Ghost Runners is going to go on to a little different department, so we'll go with each department, talking about, again, it will feel to us to sort of introduce them to students, people looking at the schools or investors that are working in sort of a job, and finding some kind of research that will try to get viewers out there. So, what happened to us again? Are you a guest or a group? I'm not from Tully University. I'm a Ghost Runner. So, what I do is I'm hired to come in, and I'm just ready for the university. So, I won't be in front of the students, but that's the right way, I want to just move forward. I don't know how much time it's in, so. Yeah, sure. I actually do that because there's no reason that we don't want to be in front of the students. That was since my laptop, I just... I know it's your laptop. That's what I do. You can appear, we don't want you to be in front of the students. Let's just do the interview. We already have an area to sit on. Oh, for now, I'm just gonna walk with you. Come on, that's fine. I'm just not very happy to deal with your policies, but I'm just gonna use the laptop. Ah, okay, all right. Seth, do you mind watching my laptop? Tell me about it, Seth. Oh, my boss. So, just describe a typical work week here at computer services. It's great. It starts Monday morning, typically with, you know, two briefings. Crap, though, we've been at. Usually, depending on the number of issues we see on the weekday, there may be a busy morning. In general, we're looking forward to this, but actually, today mine's coming up here as well. So, we can talk to the boss as well. Just get both people to do it. Yeah, sure. Come on up. We're gonna have both of you at the same time. Well, thanks. So, we're at the big interview now. My partner's going to go with the other two interns. We're gonna go with it. I hope you get some coffee, guys. All right, so, what is the work environment? I know it's a little bit of an awkward question of the boss. Yeah. We know that's big of an experience. I would say it's become better over time, right? We have a new CIO, a new stage. It's a proofing. Yeah. So, how would you say? How is it working? I know it can be kind of weird. Do you feel that the employees are new to the boss or isn't it much new to the boss? I think it'll be a lot more fun. I'm the kind of one who a lot of friends and they operate on. Yeah, that seems true. I think it's new. It's new. It's new to the boss. It's new to the boss. It's new to the boss. It's new to the boss. Okay, so I'm gonna pause it right there. And basically, if you were to put this out in a playbook format, okay, it's very interesting, right? So they have a good context. They come with a good backstory. Temple's making a new website. They've hired us as ghost writers to get a feel that's why we're talking to you when there was sort of a credibility check when what do you mean ghost writers, right? So they actually clarified the term. They had done their research. They were able to defend that. When the laptop, when the first target gets up with the laptop, they said, sorry, you don't need it. You don't want it to get damaged. That doesn't work. We have everything set up. You don't need to bring it. That didn't work. It violates policy. Okay, so they tried hitting the target with three different types of adaptation right there. When you have the first pass of the laptop, okay, they asked for a dual interview. When you had the second pass of the laptop, they split up, okay, and so on and on it goes. So again, the team ultimately ran out of time and they were not successful, but they still ended up scoring, getting a perfect score, basically, because they kept at it and they stayed calm and they tried to make it work. This was the only team that used sort of a calm approach. All the other teams had emergency scenarios. So we had four teams with law enforcement scenarios, okay, where they came in and they accused the people from IT services of some type of wrongdoing. So we had four teams doing that. Two teams came up with disaster scenarios. So they had chemical spells. So teams came in with fog machines. Other teams came in with hazmat suits. Okay, so they did a couple of, somebody even came in and did a fake pregnancy, like giving birth. So it was madness, but all of them came up with emergency scenarios, right? So some of the students thought, okay, props and costumes, interestingly, the fall 2017 teams were better prepared than the spring 2018 teams. They had stronger stories, better prepared, better equipped. Their biggest thing was unanticipated tag teaming, right? And so the IT services did a couple of other mean things, like they got access to the laptop, but that's when they scheduled system updates. So they couldn't actually get to the device or anything like that. They took the battery out, so they couldn't start the laptop. All right, so all those types of things. But one of the things, I think it became really easy for the teams that were using emergency scripts. So in the spring 2018 semester, I actually said you can't do emergency scenarios, and that made it really, really difficult. So some teams came up with some interesting things. One was a birthday celebration surprise, okay? So they came in and took the person away to play some party games. Because they were IT service representatives, they came in seeking tech support help. And there were a few other creative ones in there, and I don't have the time to get into that. But what I do want to talk about is the FLAC summary. Okay, now that I've done this across two semesters, I found that the shoulder surfing activity, the students really liked as having that as the first one, because they felt that it was a great warm-up exercise. We don't know what social engineering is. We don't know what we're gonna do. So here's a good, safe, fun one to start with. The downside, of course, is well, we don't know our classmates yet, okay? The laptop distraction activity, that was the most dynamic. Students had way too much fun with that, right? Because it was in real time. But the downside was they said it was one of the toughest ones, because you only had five minutes and you had to do this. The group selfie, they enjoyed that a lot. They said they got to be creative with it, but they needed more time. Okay, so overall the structure of the flags, where the instructions were clear, easy to follow. They were happy with the amount of time that they were given. And the only change they said was, can you make the laptop distraction activity as a last flag? Okay, because that was the toughest one. So the first two would give us enough sort of prep time and let us warm up and hopefully engage with the ID services folks a little bit better. So in closing, what does this mean for me as an educator? Implementing social engineering projects are not easy, okay? In terms of logistics, duration, structure, coming up with the entire setting, it was quite challenging for me. One of the main challenges was getting approval from the University Ethics Committee. And the ethics was a big point of discussion yesterday. A lot of speakers brought that into their talks. It's, I think it's even tougher and trickier when you're dealing in a university setting. So yes, there were a lot more restrictions on the flags, the design, the implementations, but they still worked with me and we made it work, okay? So was it the best flag? Is it something like perhaps what you all do professionally? No, but it's a starting point. Designing rubrics, how do you grade a project like this? What are your categories? What are your weights going to be? How much do you get? How many points do you give for these different things? So how do you do that? One of my biggest challenges was catering to the computer and information science students, okay? So as I mentioned, I had six students in the fall semester from computer science and about nine in the spring semester. Their biggest thing was, do we have to talk to people? And I said, yes, welcome to the liberal arts, right? You do, of course you have to talk to people, right? Because they're just like, well, can we write a script that would like execute? I'm like, no, you can't know, right? You are not hacking a computer system, you are hacking the human. So they actually found this very challenging. Some other things that I'm sort of now going through my mind is how do I develop ethically acceptable variations, right? So one of the things that the students really wanted was, hey, if we're doing a group selfie project with a prop, can we target professors? I don't know how my colleagues would feel about that. Right? Yeah, I'm 10 years now, yes, but I still like my job, right? Like, you know, I do at some point again want to be promoted. So it's, how do I come up with good props? Okay, so there were a lot of students who were like, yeah, we need, I think almost like giving someone a llama and a flamingo actually was, the props were cute, right? So it actually worked in their favor. So what might be some tougher props and that the students would have to really come up with unique storylines? Shoulder surfing. They said, yeah, we'll do this, but if we get you, do we get bonus points, right? So for the first year, I didn't really do any of that because this was, like I said, an experiment. I'm trying to figure out what the heck I'm doing, but in the next year, when I introduced this, that might be a possibility. So in closing, right? This is not the first experiential learning project that I've done. This is actually the second one. Social engineering is the second one that I've tried out. Okay, I've worked with electrical and computer engineers, Idaho National Labs, right? We did something where we simulated power grid cyber attacks and then my students ended up playing adversary. They designed the attacks that were launched against the electrical and computer engineering students. So we actually did that and that actually is the talk that I had presented earlier at Shmukhand this year. So why am I trying to do this? Okay, I think it's really important to emphasize and that's why I love this village so much, right? That cybersecurity is not just technical. Is that there is a lot of human aspects to it. So how can disciplines in the sort of social sciences and liberal arts make their students realize that what they're doing is important and they can contribute, that they do have a voice. And initially I'd started this as something I wanted to do for the liberal arts students, but I think hard science students can also benefit from this, right? So it's kind of moved beyond just for the liberal arts right now. It was a heck of a lot of fun, okay? Students had a great time. IT services had a great time. They were excited that somebody asked them to engage with their students. The biggest thing for me, and this is I think the last point that I have here, is I was able to create at least a small slice of it, right? But create something effective, safe and fun, okay? For students. And I think what's really important is that we need to educate the educators, okay? It is risky to think outside the box, but I think it pays off because you take your education, your pedagogical culture in a whole new direction, and I think that's really cool. So I do have new flags for the spring 2019 semester in the works, but if you have any suggestions or feedback, I'm happy to take anything really, because I do need a lot of help with this. It's not easy. And so with that, I just wanna say thank you for your time and attention. Liberal Arts. He was in media and communications. So, yeah, yeah, mm-hmm, yeah, yes. Right, so if you notice in both the videos, right, they had a guy walking in the back and in the voiceover, they said we couldn't ask him because he's in our class. And the second one was Zoom-in was also because that was someone from another team. The problem was they were all crossing paths because they were all out there on campus doing this at the exact same time. And that caused problems, right, because they all had different stories. So it's gonna cause conflict if they approach the same target at time T, and then five minutes later, a different team comes with the same prop and says, hey, we're doing it for this reason. That kind of messes things up. So the Zoom-in was basically to show that there's intersections going on. Yeah, yes. Yeah, you know, one of the biggest criticisms, and I think it's a valid criticism that I got from my students is, well, it's not fair because the IT services folks know what's gonna happen, right? So they're gonna make our lives difficult. And yes, that was true, but they didn't know what the script was going to be. Okay, so I literally had to form a barrier between the tech representatives, between the students. So even when I had the spring 2018 class, I didn't tell them the strategies that the IT services had to use the fall semester, which was passing off the laptops. So it's so new, but it is a possibility.