 It's great to see everybody. My name is Andrew Ryan. I am from Metta. And I'll be talking today about building and running a diversity-focused pre-internship program for SRE. So my talk today is ultimately about helping to ensure a healthier, stronger future for our industry, as well as being a lemons-to-lemonade story about pandemic-based work. I'm going to talk about the problem that we have at Metta, how we chose to address it, and give you a blueprint for what we've done and hope that you can use too. And of course, the blueprint involves open-source software. So I'd like you to leave this talk today with an understanding of what we've done and hopefully some inspiration to maybe try something similar at your organization more on that later. So I'm a production engineer at Metta. Production engineering is essentially what we call SRE. I've been at Metta for 13 years and in this industry for over 20 years. And in that time in my industry, I've seen SRE site reliability engineering called many different things at many different places. And so I'm going to use terms like SRE and production engineer interchangeably. But your organization might call it something else. I've put some of those job titles. So there's things like DevOps or Cloud Engineer, various other types of titles that people may have. But they all involve the same core skills of reliability, monitoring, troubleshooting, incident management. And those concepts and skills transcend time. They transcend job titles and organization. It's no secret that the diversity in the tech industry, especially gender and ethnic diversity, is quite bad. This is true at Metta as well and has been true for a long time. And what has changed is the willingness of companies and society at large to start addressing this in a real way. So as you can see, this is from our officially published diversity report. At Metta, three quarters of tech employees are men. And over 90% are either white or Asian. We've established concrete, company-level goals for improving our diversity and published those goals publicly. I think five or 10 years ago we might have thrown up our hands and said, this is too big of a problem. We can't do anything about it, but not anymore. Also, we're not just chasing some arbitrary metric here. To quote from our annual diversity report, we believe that connecting the world takes people with different backgrounds and points of view to build products that work better for everyone. This means building a workforce that reflects the diversity of the people that we serve. To improve diversity in any significant way, we have to bring more diverse people into this industry early in their careers. Doing that is not easy. You might think, well, OK, go out there and send our recruiters to colleges and schools that have more diverse student populations have them recruit SRE talent from there. But it turns out that doesn't work. So why is that? Well, there's four main problems that we've found. The first one is that SRE is not well known as a career path. So for example, many colleges will have a software engineering degree. And they know they get a degree in this. They could get a job as a software engineer. But there is no site reliability engineering degrees or anything like that. And most students that are even going through a technical program just don't know that this is a field that employs thousands of people across the world. Second is that the concepts of SRE, foundational knowledge like Linux, databases, operating systems, is not taught consistently throughout schools. So even if they are, even if students are interested in this, it's very likely they don't have a chance to take a course in any of these concepts. Now the third problem is that students lack interview skills. And this is a very common thing. Now if you don't have access to a large tech company to do interviews at, you're probably never going to practice. And you can do things online. But it's not the same. And if you don't know how to do a coding interview or other types of tech interviews, you're going to have a hard time getting a job. And then finally, the issue of access, this idea that you can't be what you can't see. In most schools, no student has ever gone to work for a large tech company like Meta or Google or Amazon. And so they don't have examples. People just don't understand that this is even a possibility. That's for somebody else, and it's beyond them. So these are the kind of core problems that we're facing here, trying to increase diversity in early career hiring. So how does it work in practice? Well, at Meta, most early career hires are former summer interns, and a large number of them are from top computer science schools. And how does that work? Well, so we send our recruiters to these top CS schools. Some number of students pass the interview. They get hired on as interns at Meta. And then in turn, many of them will have successful internships. They go back to their schools. They also join Meta as full-time employees. And when they go back to their schools, now there are examples for the other students. So the students are able to see, like, oh, well, there's Andrew. We've got an internship at Meta. I could do that. And they also get tips. They're able to coach each other on how to pass the interviews and what working at these companies is like and what the expectations are. And those are extremely powerful network effects that over time create these self-reinforcing networks where these schools get a greater proportion of students work at these large tech companies. Now, I have nothing against these schools. I work with many of these graduates every day. I come from such a school myself. But they limit the diversity of the industry because we are limited by the diversity of the schools that the population of these schools that we're recruiting from. So we need to get sort of bust out of this cycle a little bit. We decided that to do that, the best leverage point was to help students get internships, ideally at Meta, but if not at some other tech company. And we decided to do this by creating a pre-internship program. Basically, we would recruit a diverse group of about 100 college students, like in the sophomore range, since usually students are juniors when they get their main internship, and prepare them for an SRE internship at Meta. The fellowship aspect of this program is important. Our interns are actual full-time Meta employees while they're there for the summer. And as employees, we can't and don't discriminate or legally favor anyone based on their gender or ethnicity. But a fellowship is legally considered a scholarship. And there are no restrictions on the criteria that you can place on who gets a scholarship. So our fellows would not be Meta employees. And we would get a partner to administer the program and disperse the fellowship funds. This gives us more flexibility in recruiting diverse participants. All right, so what would such a program look like? We decided that our program needed to combine the three major aspects of what we think it takes to become a successful SRE. The first is knowledge. So the basic foundational knowledge of the job, this includes technical building blocks of an SRE career, things like Linux, coding, databases, and troubleshooting techniques. The second is what I'm vaguely calling SRE culture, things like the ideas around release management and reliability, service level agreements, incident management, all these things that kind of are the bread and butter of how we approach our jobs. And then finally, mentorship. Going back to you can't be what you can't see, it's very important that students, when they're starting out in this new career or even considering this new career, have mentors. They have people that have done it, have been there, who can advise them, who can encourage them, who can be examples for them and give them tips and tricks on what it takes to both get into and sort of be successful in the industry once they get there. So given these three aspects, how can we combine them to make a practical program? So remember, it's early 2021, no one's been to the office for a year at this point. In fact, they're not even open, I can't even get in if I want to. So our program had to be fully remote, although from a diversity perspective, I think that that was actually an advantage since it made things easier for people who couldn't travel easily due to their circumstances. And the pandemic made, at the same time, made our company leadership more willing to accept and try out new things. Pre-pandemic, I think that there would have been more pressure to do this program on site, which would have made starting it much more complicated. So we decided on a 12-week format and to combine technical curriculum, professional mentoring and regular interview practice. Along the way, every week, we wanted to present our fellows with speakers and panelists from both inside and outside of Metta to give them more perspectives on SRE and engineering as a career. We also wanted them to prepare a final project that would allow them to tie together everything that they had learned and start to build a professional portfolio. And last but not least, we wanted the students to work together in teams. So being a successful SRE, whether you're at a big tech company or a small one, requires a lot of teamwork. And working together in a group almost always means that you'd learn more and you learn it faster. So these are the things that we wanted. But how? Right? It's February 2021. We have, we want students starting by June, but we have no program, no curriculum, no partners, and there's a pandemic raging and everyone's stuck at home. So what do we do? Okay, so you take a hard problem and you break it down and we broke this down into three levels of responsibility. The first one being the Metta team. You might think that a project like this needed a large team of people to build it or the kind of resources that only a very large company has, not true. Our core team was four people. A business program manager who ran all the scheduling and handled like things like the commercials. We have a mentor coordinator who coordinated our dozen or so mentors. A curriculum coordinator who's in charge of making sure that the curriculum was appropriate. And then the program coordinator sort of did everything else. That was me. Then we also had some other ancillary people like our executive sponsor who of course signed checks and occasionally removed roadblocks. We had recruiting partners and the dozen or so mentors from Metta. And all of us were doing this part-time. So this was not like a full-time job for us. We were all doing it as part of our regular engineering jobs as well. The second pillar was curriculum. So we didn't want to, as mentioned, we didn't have any curriculum but we didn't want to reinvent the wheel. We know there's a lot of really good information out there around Linux and databases and all these concepts. We were able to get a partnership with the Linux Foundation who were generous enough to give us access to some of their Linux sys admin curriculum on a royalty-free basis for this program. And that was a huge save. So thank you to the Linux Foundation, of course, is also the sponsor of this conference. But that was a great thing for us. It saved us from having to do all that ourselves. And then finally, the Fellowship Administration Partner. As I mentioned, we had to have a third party to do all of the administration of the program and disbursement of funds and whatnot. So we looked at a couple of different organizations and we settled on Major League Hacking, MLH as I'll call them. And they were in charge of recruiting fellows assembling the curriculum per our guidelines, doing the day-to-day running of the program and dispersing the funds to the fellows. So they're an organization that runs hackathons and other sort of programs like this all around the world for a couple of years and they have been a good partner to us. So what did the fellowship actually involve? I said it was 12 weeks. Here's the schedule that we used for last year. Now, some of our fellows had never even used basic tools like Git. So we had to start at the very, very beginning and then try to introduce them in a very whirlwind, quick ride into everything that we thought an SRE needed to know. So things like bash, databases, containers, testing. Two things I'll call out here, two weeks. Week seven was the career week. During career week, we did two things. One is we helped the students prepare their resume and LinkedIn profiles so that they would be able to present themselves better in the job market. And we also did mock interviews with Facebook or Meta, was Facebook at the time, Meta engineers and gave them the ability, gave them the opportunity to have like a real tech industry interview. And this was great for two things. One, it gave us a level, kind of a level set on where the students were and also gave the students an opportunity to have a real interview at a real tech company, which otherwise they might not be able to have had. So that was super valuable. And then the last four weeks of the program, we spent on a final project. And the final project was anything that students wanted to put together that would use all the sort of foundational pieces of technology that they've developed. The idea being that at the conclusion of the program, now they would have something checked into GitHub that would be under their name that they could point to employers to and say, look, this is what I did, and get experience with all the show, they had experience with all those software technologies. Now, since this is an open source focus conference, I think it's important to call out how essential our open source stack was for training new SREs in this program. Now, some of the stack open source SRE training stack that you see here, such as CentOS, GNU, and Python is used by Meta internally. As you may know, Meta sponsors quite a bit of open source development on these tools and uses them extensively internally. For our own proprietary software tools, for example, our system monitoring and stats graphing tools, we couldn't use those for training fellows because they aren't employees, remember, and they didn't have access to our internal systems. But fortunately, most of those tools like have open source equivalents, for example, in the time series graphing, we've got Grafana and Prometheus. And I think it's really super cool that we can train students in the discipline of SRE at a very high level, essentially like a professional level at using entirely open source software. So really good call out there for the open source world. Earlier, I mentioned the importance of teamwork and getting our fellows to work together in teams. Now, how did that work in practice? So each, if you recall, we had about 100 students in the program. Each, we had 10 pods. Each pod had about 10 fellows. And then within the pods, they sort of grouped themselves up into smaller groups. And they'd all day be chatting on Discord and whatnot. And remember, the fellows were not in the same geographic places. They were sometimes not even in the same time zones. But they would connect every day with their pod, what was called the pod leader, who was someone from MLH that would coordinate a daily standup. And they'd work through the day's assignments or the week's assignments. In addition, once a week or so, each of the fellows, either individually or in small groups, would meet with their mentor from Metta and just get a chance to ask them anything. So it could be asking them about technical stuff. It could be career stuff, interview tips. So we kind of left it open as far as what that mentoring relationship would turn into. So that's the program. What about the results? Well, first off, I'm really proud to say that our fellows worked super hard and they had a great experience. 95% graduated the program, meaning they completed all the modules and stayed until the end. And 77% reported spending 25 or more hours a week on it. So that's really great to see. And we had in our daily sessions, 95% daily attendance. I think it's a really good, all these specifics are really good at showing that people were engaged and they found value from it. But did they understand at the end what SRE was? Do they feel like they could do it? That's another important thing. And I think the answer is a resounding yes. So take a look at these numbers. Starting in week one when we surveyed the students and going to week 12 at the end of the program, you can see at the beginning of the program, most of the students didn't know what SRE was or they didn't think they had the skills to do it. They didn't even know what the skills were and they weren't sure it was a great career path. And look at how those numbers change. Basically, everyone that went through this program knows what it takes to be a successful engineer at Metta. And that's awesome. Now, do they want to be? Is that their ultimate career path? You know, we don't know that. It would be great if it was, but I think it's really powerful that we're able to kind of instill that in them and that at least they feel confident that they know what they need to do. And then lastly, kind of like the, I mean, I don't love to over rotate on jobs, but of course at the end of the day is a program for recruiting and for hiring and we need to make sure that people are getting jobs and the answer from that is yes so far. Our data here just as a caveat isn't quite as good because we had to rely on a combination of surveys and LinkedIn data, but our estimate is about 50% of the fellows from last year got jobs this summer in some kind of a tech job. 23% reported having SRE type roles where SRE was in the title or a very similar title. And over half reported using those SRE skills in their jobs. So remember going back to that first slide, this is a big tent here. You may be using these skills like CI, CD or databases or whatever, but your job title may be something else. And 90% said the fellowship gave them advantages in the interview process, which I think is really good to hear. It means that our training was working and when those fellows went into interviews, they had a leg up because they had gone through this program and had this kind of practice. And there's some of the companies, sorry, they have some of the companies that we've got confirmed SRE job placements at. So including that, so it's very exciting. So we ran the, we're running the program again this year. We're currently actually in week four with another 100 fellows, but what needed improvement? What did we find that we could do better? The number one feedback that we got from the fellows was that they didn't understand the why of what they were learning until later in the summer when they started their project. So this year, we're trying to make the project the focus of the fellowship and making the curriculum support that. So for example, at the same time, the students are learning about containerization for example, they could be dockerizing their own project so they can deploy it in the cloud. The second learning was that we needed more interview practice. So we're adding weekly, we've added weekly modules to allow the students to continue to practice mock interviewing. I don't love, be honest, I don't love to make a focus purely on interviewing. Much more, I'd rather people learn stuff but the fact is it's important and you have to do it and if you can't interview well, you can't get into this industry and if you can't get into this industry, you can't go anywhere. So we have to spend time on it. Third one is trouble spots. So with a program this large, 100 fellows, 10 pods, numerous mentors and pod leaders, it was really easy to have a couple of people here and there slip through the cracks or indeed sometimes like an entire pod would just kind of be lagging or not getting a certain concept. That is one downfall of the sort of group project. No one of 10 people can kind of gets it then they all kind of just stuck. So this year we're instituting weekly check-ins and weekly surveys that we'll make sure that if we catch these earlier. So if there's kind of trouble spots with individuals or pods, we're able to address them and no one's feeling left behind. And then lastly, we're gonna be much more careful this year to provide follow-up support and tracking through the recruiting process, especially at Metta. So this is a little bit embarrassing but we didn't have a good integration between our program and our internal recruiting. So when it came time last year at the end of the program for all the fellows to apply for jobs, we were sort of scrambling to find out who they were and locate them through our applicant tracking system and whatnot. So we're gonna do a better job on that this year and also hopefully for the people that don't end up working at Metta, we'll have a much better handle on where they ended up and what they're doing. Because at the end of the day, this is a recruiting program. We do need to see results. We wanna see results and we want to know that it's working. If it's not, we wanna make it better. All right, so as engineers, we're always thinking to ourselves when we build something, how do I scale this? How do I scale at 10X? How do I scale at 100X, right? We're really proud of our work here but this is still only 100 students a year. If we're successful again this year, which we hope we are, we'd like to grow this program further inside of Metta, hopefully to Europe and elsewhere but we can't grow it to 10X the size or 100X the size. So what opportunities do we have to scale this? Well, for the first opportunity to scale, I'd like to refer back to the diagram that I used earlier about how we have these schools with their self-reinforcing network effects that bring us qualified candidates year after year. One of my hopes is that we can use this program to seed new schools and effectively be able to create schools that didn't have these network effects. Now, if we can get a couple people in and get them as interns and get them as full-time employees, we can start to seed new schools and we know this is possible. It's just challenging. And secondly, we'd like to run this program or something like it at other organizations. So I'm hoping there are people out there watching this talk right now, whether in person or virtually, that have authority over hiring and budgets and have similar diversity goals at your organizations. So we'd like to talk to you. Come chat with me in the hall during, after the conference, after the talk. On the last slide, we'll have some contact info where you can reach us. There's also, I will point out, a couple of folks from MLH in the room as well. You can go talk to them. And finally, we're thinking about trying to kind of open source this program in the same way that, as I meant, showed you that open source SRE stack. There's no secrets. I mean, you can go out already and find most of this content somewhere. Like there's plenty of content on how to use Linux and how to use databases and all this stuff. But bring this program all together and adding the mentorship piece in and adding the interview training and stuff like that. That's what really the magic is. This is an industry problem. It's not just Meta's problem. It's a problem everywhere. The logistics might be tricky, but we know that for industry to change and improve in the long term, we have to work together. And I am happy to say I can't give any, I can't say anything officially yet, but I am happy to say that we've got some real strong interests from other companies that is definitely happening, and you will see something like this at at least one other company in the next year and hopefully multiple. So things are happening, but we can always use more. Now, how can you help if you're interested if this excites you? First off, hire our fellows. So I know most people watching this talk are looking for people and they have open positions for full-time engineers and many of you hire interns as well. So if you see the MLH Production Engineering Fellowship sponsored by Facebook or sponsored by Meta on a resume or LinkedIn, give them a reach out. They've had some good training, I would like to think. And secondly, as I mentioned on our last slide, you can support a similar project at your company. Lots of things appear impossible until they've been done. Well, we did it and you can too. We're giving away the blueprint, we will work with you, we will tell you what we did, we'll give you every sort of contact we have. This is nothing, we're not trying to steal all the good people here, we just wanna bring more diverse people into this industry. So thank you. We are, as I mentioned, we are running the program again this year. If you wanna get some like whatever collateral on it, you can follow Major League Hacking. These are some quotes from some people that have been through the program last year and I'll be happy to take questions either in the room or virtually. So anyway, thank you very much and hope you have a great week in Austin. Does anyone in here have any questions? Okay, perfect. Sure, I did mean to go first, but yeah, I just really appreciate, just wanna say I appreciate that last sentiment that you know, about bringing diversity, making this inclusive, bringing diversity and having all of us be thinking about as an industry-wide problem. So thank you, that's pretty awesome. I did, one of my questions I was wondering was, when I looked at the curriculum on the face of it, I didn't see a lot of details about teaching open collaboration, I think open source way kind of principle stuff, which I know some of that, sometimes we just assume people know, but maybe they don't. Is that, do you have that kind of embedded in there or how has that been thought about? Yeah, that's embedded from day one. Day one, they get GitHub accounts and they learn what a pull request is and they get, from the beginning, they get like, here's how to write, like a commit message or here's how to write a pull request or here's how to do a bug report. So how and why? Yeah, yeah, so the how and why. So that's built in from the beginning and then that collaboration model that we use throughout where the students are working in groups enforces those concepts throughout. So this is very different from like a course or something where students go off and they do a bunch of assignments on Linux and then they bring them in, like from every day from the day one, they're working as they would sort of in a company. Yeah, it's with an immersive open collaboration. That's the idea, yeah. And I think we've been successful in that, but you know, can always do better. Yeah, hi there. I just wondered from when you decide to implement this to the first course, how much time did it take and how much resource in terms of like people and money did you have to devote to it? Yeah, so that is an excellent question. I did have a slide on that. So there's basically about four people working part-time inside of Meta. Maybe one, probably our business program manager was working more than part-time on this. That was pretty busy for them. But not a huge lift and then in terms of money, I mean, obviously I can't like go into like the exact commercials, but what I can tell you is it's not very expensive. It's certainly not expensive relative to interns. And it's a, I would say like very affordable. I mean. And how many sort of days, weeks from sort of conception of an idea until? So we started from zero in February 2021 and we had students in the seats in June. This year it was way easier because we had a lot of the scaffolding done. So and at MLH, for example, they do all the hiring. So we sort of write them a check and then like students appear at this point. That's good. Yeah, I think you kind of answered this question. You're like, you wrote MLH a check and then students appear. I was actually going to ask the question, like how do you convince students to do an entire SRE thing because, you know, like I'm a DevOps engineer and when I talk to, when I try to encourage, you know, like people that come up to me, you know, like women, you know, people of color that are interested in what I do, they seem to be really intimidated because as like a college student, they're used to hearing back in engineering, front-end engineering and they're like, oh, this DevOps thing is just so like high level and it's super intimidating. So I'm curious, like did you, like what, because this is like the pre internship 12 week program, like did you do like a pre, like, I don't know program, I don't know, presentations and like what are the most convincing factors that made them want to join it? Yeah, you bring up, I mean, you hit the nail on the head, right, like this, you know, and we talk about that, I talked about that in the slide, in my slides, where this is something that doesn't have much awareness in the industry or in schools. This is a field that people just don't know about. So I can give you some partial answers. The thing you should walk away from is this is still a work in progress. How to sell sort of this idea of like SRE in DevOps as a career. We're still trying to like figure that out and trying to figure out how to make that more inclusive. And I think even for this year, there's more things that we can do to kind of, you know, explain, which we'll do in 2023 to explain, like and make it just seem less threatening or scary, you know, probably in our case, the meta name helps a lot, right? Just because people are like, well, I don't know what, maybe I don't know what this is, but like, gosh, a job at meta or a chance for a job at meta sounds pretty good. So maybe I'll just, you know, take a, I think that's what a lot of the students in our first session did. We also had some informational sessions throughout the last three months through MLH. So we'd like have these open sessions where we'd have somebody from meta, like talk about the team and talk about the program. And I don't know, maybe we've got some videos and other stuff, but that's an area that I'm very, I'm very passionate about and I wanna get better at because, yeah, I mean, like you have to look back even in the funnel before the people that apply into the program of, you know, how many diverse people are looking at this application page and saying, I can't do that and they're not even applying, right? And that's, yeah, so we wanna do, we wanna do better. It's just gonna be a continual process of improvement there. You just, a question just, they came up on that last one. On the job role description, have you had that done, brought through a diversity edit, editing team of people with diverse backgrounds and skills to look for things that might get people to go, oh no, and walk away from it? For our full-time jobs, do you mean? Or for the fellowship? Oh, for the fellowship. I don't know that we did that specifically. I mean, so, I mean, I know that I reviewed it and all of our core team reviewed it, but yeah, I don't know that we had it specifically reviewed by a diversity or inclusion expert, but that's a good, that is a good idea and that's definitely, like I said, something that we'll be looking at for next year to see how we can improve, get more applicants, get more, you know, a broader base of people. And honestly, part of that is also things like this where we publicize the program and get the word out so that more people know about it, it becomes, eventually it becomes a thing and people just know it's there and then they're excited about it. Okay, any other questions? Okay, well, there's nothing online, so I guess that's it, but thank you for speaking. Thank you all for coming. Thank you.