 my name is Mark Orlando um and this is the abridged building a sock A team um having some uh some AV issues so while we work through those I'm just gonna kick it off and start talking um so just use your imagination the slides are amazing um so by way of introduction um again my name is Mark Orlando I've got uh about 17 years um in security operations of all flavors public sector private sector uh built 24 by 7 operations teams I've worked at managed security service providers really all across the board um I'm an 80s kid hence the uh references to the A team the true A team I don't know what that movie was a couple years ago but uh we're not gonna be talking about that um couple quick notes about this talk so um I wanted to give a talk about the people aspect of security operations um we're not gonna be talking about technology here um to some extent I I firmly believe that that's that's interchangeable uh lots of good tech out there the good people um are the ones that are uh so you gotta extend back to the other the other console the other Mac or can I put my Mac in now yeah let's do your Mac alright so um you know good people are hard defined uh they're hard to train they're really hard to retain and uh you know I think if you're gonna build a SOC A team or an A team of any kind uh the people are really kind of the foundation of what you're doing so that's what this talk is about today if I can get my slides up oh okay that's why it didn't work on this one either that's why we switched it originally there we go so whatever you did it's not going over there well it's okay so how many of you actually watched the original A team show we got some young people in the audience so I can be too too sure alright so at the beginning of the show you know if you have a problem you have to find the people that you need right um these are some of the problems that you're gonna have to worry about alright the first thing is a lack of really it says business alignment but it should be organizational alignment I think that's one of the biggest challenges in security operations is you have people defending environments where they really don't know exactly what they're defending um how do you create value how do you protect that value data and tools you know too much or too little um an outdated alert watcher model uh firmly believe that the root of a lot of this talk of talent shortages and analyst fatigue and a lot of the other issues that plague security operations comes down to uh this concept that we need to fill a room with people and point them at screens and have them do that day in and day out you know humans are not built for that kind of activity so this alert watcher kind of watch standard model um has to change you're not gonna build an A team based on that um and so I kind of mentioned this you know is there really a talent shortage um I don't believe that there is I mean just look around the events of the last couple of days there's talent everywhere right our problem is that we're having trouble meeting in the middle between what I need as a business owner and what I'm actually gonna find out there in the market and how I'm gonna find that talent and um promote that talent retain it train the people to do kind of what I need them to do um I don't think there's a shortage of people out there I think there's a shortage of reasonable expectations I think that's the real problem um so if we're gonna build our soccer team um you know really there are a couple of key things that we need to do and any of you that watch the show know that you know we have a small group of uh former U.S. soldiers that we're really kind of jacks of all trades whether it's infiltrating you know a terrorist cell or uh welding together you know a crazy like bulldozer and a barn or you know whatever it is they could kind of do it all right so that's the kind of team that I'm talking about um in cyber operations the first thing we need to do is study our mission study our business so if you're building a sock or you have a sock and you can't pick any one person out of the team and ask them to explain what it is that the organization really does whether it's manufacturing widgets and selling them or offering public services or taking care of people or whatever it is if they don't understand how the organization's creating value I would argue that they're ill equipped to make good decisions on any given day about how to protect that that enterprise um we need to be ready to experiment and iterate right so a lot of times we don't have that luxury in security operations right we have very specific things that we have to get done on any given day but I would argue if you're not able to kind of experiment and try different things if you're not able to look back on the past week and the past month and the past year and see what's working and what's not and be ready to change that up um you're you're really not going to be able to do all the things that you need to do if you're a leader or a manager in security operations you know you may not be um at the point where you can get back on the keyboard um or you know work a shift or offer up you know the technical guidance maybe you're you're past that or maybe that's just not your skill set but at the very least you know be ready to be in there with uh you know a bunch of pizzas on a holiday to support your team be ready to get into the trenches with them uh finally we're going to talk if I talk really fast we're going to talk about metrics and showing results. So security operations in my experience is a very opaque kind of activity and leadership particularly executive leadership of most organizations don't fully understand what goes on on any given day in security operations because that's not you know that's not their background that's not their skill set so if you can't communicate what you're doing in a meaningful way to that leadership um guess what your socks probably not going to last very long you're probably not going to be able to do all the cool experimentation and things that you want to do and then finally you know as Hannibal would say on the A team you got to have a plan and the plan when you're building a security operation center is not to build a security operation center that is a means to an end. So what's your plan for actually helping to make your organization, your enterprise, your agency whatever it is more secure? Okay um in the past uh and this is kind of generalizing a bit but really there are two approaches to building out sock teams right there's what I refer to as the talent centric model or if I'm being less kind the rock star model where you find one really skilled experienced person and you kind of build a team around them and the team sort of takes direction and leadership from uh that one skilled person or those few key people. The second option is more of a mission centric approach. Um you see this a lot in DOD or the federal government where it's like we have a shared mission I'm just going to bring in a bunch of resources kind of in short order maybe a new contract something like that and we're all just going to be focused on executing you know these specific functions day in and day out. The challenge with the talent centric approach um is you know even though you might have more capability faster because you're bringing in really experienced people and they're going to be able to kind of bring the rest of the team along your capacity is really going to be limited to to those few key people right and if they leave um or if they're out um or if they bring certain biases to the job which you know human nature the more we're in a role the more we're going to kind of bring our own biases into it the more we're going to start to hyper specialize so those biases and that special specialization is going to bleed into the rest of the team. Conversely with that mission centric approach you might be better aligned to the organization um but you might be a little bit less flexible um be a little bit less willing to try different things and iterate and experiment and go outside of that predefined lane um and you kind of run the risk a lot of times I'm missing the force for the trees so not making kind of more discrete observations about what you're seeing maybe new threats new trends um you know kind of issues with the process as it stands might be a little bit less likely to identify those as everyone is kind of focused on just executing a set of predefined steps so how do we kind of strike the happy medium between that talent centric approach we all want talented skillful people um but also being mission aligned and mission focused um you need to look for the right attitude and aptitude you got to have diversity of thought so that means people with different backgrounds people with different skill sets right um you want to avoid over reliance on experience and certifications I know we all um have opinions about certifications and credentials they can be really good things right but we don't want to rely on those as indicators of skill of course um and I probably don't need to say anything about egos and misrepresentation you know we kind of want to stay away from that um a few key notes and and I'm going to kind of step through these quickly um when you're working um whether it's junior staff or senior staff um there are a lot of things that I think people don't take into account um with some of these staffing models I've seen a lot of operations teams staff up really fast on very junior talent maybe they've got a good pipeline from like a university or other sort of feeder resource um and everyone's thrilled right because we're able to get some really uh talented motivated staff and you know uh compensation is usually a little bit lower and so we look as that at that and business owners especially look at that as a win right but we don't think about in the out years you know these people are going to get really smart they're going to build up that experience very quickly and so we need to plan for additional investments in that team in relatively short order so that means we need to be you know right sizing salaries to be competitive in the market we need to be making investments in training uh have a long term career growth path right if you're not willing to provide those things in years two and three in the out years um you're going to have a big problem and all the time and resources and energy you've put into those junior resources is going to walk right out the door to another security team uh on the other end of the spectrum um with senior staff and I put asterisks here because um I'm grossly over generalizing here um but by and large you know compensation is going to be higher um there's going to be probably a little bit less flexibility or at least more hyper specialization that tends to happen you know the further down the the path uh operators get and so um you know I find it's more useful sometimes to try to look within the organization to bring in those senior resources maybe it won't be in some of the specific technology or disciplines that you're looking for um but there will be a lot of um you know work experience uh technical experience you know finding those good fishing holes I'll talk about this in a second um you know talent begets talent so if you're staffing your team with more skill more experienced resources uh chances are they've also worked with other skilled experienced people right so how can you tap into those fishing holes make hires you know two three four at a time if at all possible I've had a lot of success doing that okay when we talk about training you know I think in our in our industry uh there tends to be a lot of focus on classroom training and textbook training and tool training and I think all of those things are great but I think sometimes they come at the cost um of training our our operators and our analysts our investigators and you know the right way to think uh the right way to ask questions the right investigative mindset I'll uh share some examples from Chris Sanders I don't know if any of you have read um or seen Chris talk or read of any of any of what he's written he talks a lot about analyst mindset and the right way of thinking those things are a lot harder to train someone to do so I would say whether you're recruiting or whether you're putting together a training program focus on those things first um the other stuff you know how to write queries how to use tools you know those things can can be taught relatively easily or at least there are a lot more resources out there where you can kind of outsource that you know focus more on the the kind of analytic skills right um I did want to include um before the time's up here talking about measuring operations because this is one another area I think that we don't spend a lot of time uh on but again in my experience um it really comes down to the difference between a high-powering team and a team that's kind of proven itself over and over in an organization and one that is really subject to you know changes in kind of corporate direction or what have you really subject to um external factors that that may or may not benefit the team so KPIs I think a lot of us have heard about key performance indicators those are you know how you measure kind of day to day operations um less popular and what we do is a concept of um objectives and key results so this is where we tie what we're doing day to day in the sock to larger business objectives so talking about that organizational alignment this is really kind of how we we make sure that that happens um sample KPIs uh anyone who's done sock work probably recognizes these kinds of things right what's our visibility you know how many investigations are we working on any given day what's our average close rate what's our average time to discovery time to remediate things like that those are all KPIs okay ours are more things like you know where are we trying to get to with our team right are we trying to reduce successful attacks um we're trying to reduce um you know kind of average time that our team is taking to to close out cases and there can be some overlap here um but I think it's important to keep those strategic and business goals in mind as well and be tracking those over time just as much as you're doing uh kind of the more tactical KPIs right um we also I think overlook often measuring people right so in the A team right there was no question that they were all rock stars they were all awesome right that's why the show is so entertaining that's why they always were successful by the end of every episode but you know how do we measure our people day to day um where it's you know maybe a little less glamorous right and it can be difficult to measure people because you want to run the risk of reducing um you know what is really quite complicated job to um you know numbers and ratings and things like that and and we really don't want to do that so uh I included a couple really good resources here including um an analyst baseball card Chris Crowley uh kind of came up with this concept and I think it's a good way to come up with good kind of measures of how effective uh and contributive an analyst on the team is um and we use that you know not to penalize but to kind of identify where our people can improve where they're doing well um you know things like that so I think that's uh a great resource um I've got a link to this as well okay but the bottom line is you know talent can be found in lots of different places I think most of us know that um we all come from from different backgrounds but you know understanding how to identify it how to foster it how to retain it um you know that's really the key challenge and I think um we really have to get away from that alert watcher that watch standard mindset and think about you know how we're improving over time how are we you know drawing a line in the sand and kind of moving the sock forward both as a team as a full sock capability and individually how are we helping people grow um so um I think the difference between an A team and kind of all the other teams out there uh is really cohesion and measurable results so every team that I've built or run or managed um you know they're able to point to kind of real tangible impacts in the organization thinking about how the organization um generates value and how they're protecting that and enabling that versus well you know we we do a bunch of stuff we got all these tools they generate alerts we look at those um you know I think really that's that's the wrong mindset the wrong model um I was going to include some case studies uh I'm sure these slides will be out there I don't think we have time to go through those um but you know some key lessons in here learn how to tell tell the story um if you've discovered kind of how to um tie what you're doing to that business value learn how to tell that in a narrative way and learn how to repeat it early and often to your leadership um being in a sock and running a sock is as much a sales job as anything so I've got some other resources for you you know really anything that um uh is in you know sans sec 450 on the blue team um mitre's written some really good work obviously on security operations uh the ten strategies for for good security operations is a little dated at this point but still really good resource um and just some some kind of food for thought and I think that uh we're about out of time so thank you very much appreciate it