 Cool. So hello. I'm really glad to be here today. This is my first conference in three years. It is something else to be here, so thanks so much for coming to my little talk. This is the title that Matt kind of landed on based on a previous presentation I've done, which, you know, it kind of sounds interesting. But as I started to get into the kind of the meat or the core of what I was trying to get, what I was going to present today, I realized it's really about how do you use metrics to get people to do the stuff that your organization, or your project, or your foundation, all of those user groups that Gay or Glycid earlier, because they saw a metric. And I think this is an area of work, at least for me, that I'm continually interested in, is how do you take those metrics that you care about that you've built around security, diversity and inclusion, value risk, and kind of push it down to the organizations and get people to care about those and track them with you. So that's what my talk's about today. It is largely based on experimentation. I don't have any super great answers for you, but I do hope to inspire your conversation later. So yeah, so hi, I'm Emma. I know most of you, I'm really excited to see new people joining the Chaos Project. I've been a part of the project peripherally and involved over a number of years. I can't say what my favorite open source project, honestly, is very inviting. There's an opportunity for everyone to get involved and build something. I work on Microsoft's open source programs office, so Ospo. And then I spend my volunteer time working on Canada's Open Government Stakeholders Forum, where I help, I'm part of a group that helps the government push their commitments to being an open and transparent government. So that's the thing I do as a contributor. And just another way that I feel like open source is a great place to be. I live on Sook, so on Vancouver Island, I just took the ferry over yesterday on the land of the Sook First Nation. So my slides are being mixed up, that's a good start. I'm more nervous, I haven't stood on the stage in a long time, it's so weird. I just need my little camera and stuff, so thanks if my voice sounds jittery, it's new to me as well. And I think this is the slide that I really want to start with. I want to acknowledge that when we're talking about the motivations of people, we need to acknowledge where we all are right now. And from my perspective, this picture really captured how I feel have felt in the last three years. So through a pandemic, kind of on the outside with my mask or my helmet, whatever, looking at the world, going by, looking at people through cameras and being restricted by a number of things, increasingly more complex, things like the economy, the meteors in this picture kind of represent to me what's felt with the layoffs in the tech system recently, like big ones, small ones, but kind of randomly falling around us all the time. And this is why I think as we start to talk about recognition and value and just helping people accomplish things, we need to recognize that we're in a different place. And the old ways that we might have thought about recognition and value are maybe not entirely different, but they're definitely changed. So I think that's the challenge. And it's not just my experience, so this is my first conference in three years, but it's something that I've learned from working in Microsoft's OSPO and surveying folks whose job in some way involves open source. So in our annual survey this year, we saw that four times as many people responded to a question around recognition. Do they feel recognized for their work in open source? Do they feel like it lines up with their business plan? And four times as many people said, I'm not sure. I'm worried. I might not mention it in my review. So that's four times as many people, and this made sense to us because lots of layoffs, people are trying to be really sure that what they're working on is what will be recognized. So it's a feeling and it's a reality for a lot of people. So some of the hypothesis and experiments that we've been working on and that I'm going to share with you are around these kind of three areas. Creating curiosity hooks, moving away from asking people to do things or at least make curiosity the first contact between you and the metrics. Between the people you're trying to engage in the metrics. Consider designing personalized motivational stories with data. So I've been to lots of great talks on creating data stories but making that something for individuals in organizations and communities that will motivate them around the things that you need them to do. And then finally this will be more of a call to action at the end but creating communities of practice. How do we do that? Chaos is one. But to even go further and build those within organizations. So building curiosity hooks. This is what I'm calling it anyways. I think curiosity has been a great tool for me personally especially learning about diversity and inclusion really taking time to learn and listen. And I'll give an example of how this hasn't worked for me first. So inside of Microsoft we have a working group of five really smart people who are absolute experts in the areas of diversity and inclusion recognized outside of the company and anyone would be lucky to have these five people offer to take a look at their open source project and offer to make changes. We thought the pitch of only two hours investment on your side plus the list of experts names would be, you know, we've had a lot of people to choose from. Literally no one responded to our call to action for our call to help them build more inclusive projects. Now is this because maintainers at Microsoft don't care about diversity and inclusion? Of course not. In fact at Microsoft diversity is one of the pillars that everyone is called to review as part of the review cycle. It's like everyone has to have a set of goals for diversity and inclusion in the organization as part of their job. So that wasn't it. And then I started to think like, and talking to the wonderful group, you know, what are we not doing? Like why are people not interested in this? And then we started asking ourselves what are maintainers most, what are their pain points right now? Well, it's time, right? They've got lots of PRs. They're trying to get their work done. It's maybe on the side of their desk for something else. And so that sounded maybe a lot like work. Let us give you two hours of work and it might help you in some way. So instead we decided let's think about who it is we want to work with and how we might phrase the opportunity to increase the inclusion of their project using words that matter to those folks. So we queried our GitHub information. We found the top projects who were relatively new, increasingly seeing a lot of pull requests, issues, contributors. You kind of see it going faster than the speed of what a maintainer might be able to keep up with. And we've started to reach out to these folks with this opportunity. Let us help you. Here's looking at the data. Here's a list of potential contributors that might take on a triage role, which doesn't come with a lot of privilege, but is very helpful. And let us help produce your workload. So the first three projects I've reached out to have all responded like, yes, that sounds great. Tell me how. Let's set this up. Let's talk about it. So the first two steps of creating a curiosity hook is really define people who are most likely to benefit from your metric. Just broadcasting. This is important to the company. There really needs to be something that has that cross-pollination. And again, thinking about the time we're in and the stress people are under. And then pitch that out in terms that hit their immediate pain points. It's okay that we're not using the specific words diversity and inclusion. Our group will make sure when we work with them that the onboarding and all those types of things that we're experts in will be there. And they can talk about that later, but initially it started the conversation. Here's another statement that I've made in multiple places before because I know how important it is that people know what to do if they're maintaining a project or they're working in the open that they know what to do if they experience a violation. If you say to someone in a decision-making role who's less familiar with open source, we need to make this mandatory. Well, this is kind of the reaction that I've had over the years because it's not understood. The value is not understood. You're suggesting that we create more mandatory training for all these people across the company who are already taking training on a lot of things involving privacy and safety and diversity and inclusion. So just kind of maybe go away. This works though. Finding people like HR legal, policymakers, standards people and saying we have over 42,000 employees who are members of our GitHub repository and organization and these are recent numbers. We've contributed to over 12,000 repositories that are governed by our company, by people who may or may not have codes of conduct and by other teams. But don't worry, we have a course that reduces risk for safety and privacy of those employees. This is the hook and this is being the response and I can't say how much better this response has been for me because now we have a cross-company team that involves HR legal, involves standards folks, involves open source people, working on different open source across the company slowly. It's not mandatory yet, but starting to share this opportunity of learning and making it so. But again, the curiosity, without that we wouldn't have got that far at all. And so finally the last hook there is using data in that pitch as I did with the one here. We don't have to wait for a second conversation. We can open conversations with data in metrics. A third one, a third and final one around kind of like asking questions in the wrong way is going to a team or maintain or asking, are you following the security best practices and here's a bunch of ways you could do that. This is kind of the eyes and I've heard this a lot from people who work trying to enforce security standards is people are doing their best out there. They really are. They're trying, they're responding to the compliance flags, they're working to the best of their ability to do their job and to be as secure as possible. Please just, can I just get back to work? Can I just please keep going? A better question that prompts curiosity is to give someone a number like this. What does your 7.4 open SSF scorecard tell you about your project security and preparedness? When we show this to people, we've done this in a few different ways, which I'll get into, there's definitely curiosity. Both from the perspective of, oh, what is that made up of and that can't be right. So, and I think Geyer was talking about vulnerabilities early. This is a, these types of questions can prompt that early discovery of vulnerabilities instead of it being something that just kind of happens later on. So, yeah, that curiosity has definitely happened with that question. Another way that we've tried to, oh, sorry, another way that we've recently tried to bring curiosity hooks to people is through an email. We sent all 36,000 maintainers at Microsoft an email and, you know, Microsoft is actually a very email-y company already, so we did not do this lightly. But the goal was again to get to the motivations of the people who we need to take action on the things that they care about. So, community connection. We hear this all the time. It's one of the most requested things of us, but I want to connect with other people. I want to hear what they're working on. I want to share what I'm doing and, you know, build on each other's work. The second is engineering excellence, building on the work of others and diversity and inclusion. Those last three are all in the review cycle. So, people seem to pay attention, at least that was our hypothesis, when they saw these categories. This is just part of one of the emails. I'll show you a couple of sections, but if you catch me later, I'll show you one of the whole emails so you can kind of get a sense of it. This is one of the more interesting sections to people. We showed them what their contributions to an upstream or non-Microsoft repository brought back to the company. So, this is in the category of building on the work of others, inviting others to build with you. This is something they can talk about with their managers and their review cycles. That's otherwise just sounds like, oh, I made contributions to open source. Under the category of do stuff, we really need people to have two owners for GitHub repositories. And so, at the end of the email, was this ask, please make sure if this GitHub repository is not one that you maintain or own, please just update the owners quickly. Just take a moment to do this. So, that was actually really successful in that people did take action. People don't really want to own things or have responsibility that's not part of their job. And so, going into building community of practices around metrics, this is something that I'm always thinking about and always kind of have a lot of goals to get better at. How many people, if you 3D print, which is fun, have been to like the Thingiverse website or the Prusa website? Yeah, good. More of you should do this. But when you go, you can see I want to print a Star Wars a lightsaber. My youngest daughter did this the other day. And you'll go and you'll look at the different prints and you'll see how many people have made them. You can see pictures of people have made and left comments and people make improvements. And I'd really like to see that community of practice exist inside of our company but also through chaos, like something that we could build together. That makes it easy and is clear. We started a little bit with an experiment on this. Again, around the open SSF scorecard, we had a little hackathon inside of the company and added the scorecard for every single repository inside of Microsoft. So when they go to their repo, they'll see the score. And when they click on it, they'll see the breakdown in ways they can improve. So we've had different types of feedback here where people were really interested and reported that they made changes, which was huge, that's the do stuff. Some people actually made changes in their work flows. We saw a lot of people react in different ways, which is exactly the type of learning we were hoping to accomplish. We're not there yet. Not everyone looks at this page. It's hard to get eyeballs, but it's definitely an early insight that's encouraging. And we asked people, sort of in that idea of the Prusa website or the think averse, share what did you do? What did you learn? And I did try and do some bribery, I'll tell your manager with the kudos and people did self-report that as well. So that's just an early experiment. Seeing people respond at all was very encouraging. And finally, this is a mock-up. I wish this was real, but something that I'd like to see is how do we build these communities of practices with metrics that matter to our specific organizations in a given time. So these are categories that we've come up with just based on a lot of conversations in our metrics group internally. There's a grouping of metrics that we want people to review regularly. So we really want people to pay attention to their security alert response time, their scorecard value, and to take the code of conduct training. We want people to think about their operational efficiencies, sorry, that bar is collapsed there, but under there's things like pull request, responsiveness, and then community health metrics. The way that this is imagined is, again, to have those categories that indicate where those metrics fall in importance and relevance to people's workflow. Maybe it shows them what their average is compared to others at Microsoft. We're thinking about the motivations there that you're reposed behind the average, but I know that that can get complex. Somewhere to ask a question. This is something we hear all the time. If people will get a metric and the scorecard was one of them, what is this this way? Being able to send people to a working group of people that work in security all the time or work in diversity inclusion all the time or community experts feels like a real opportunity to build that community of practice. And then, what actions do I take? So we find this is a or I find this is a challenge as well, like, so what do I do? Like you've given me this, what am I supposed to do with this? So clicking people through to a page that allows them to improve their response time, make some change, improve your code of conduct training, take the course. Some buttons are very easy, some buttons are more complex, but just that there's this very simple interface is something that we continue to think about. So these are some hypothesis and experiments in creating motivation and impact stuff through metrics. Create those curiosity hooks, not asks, even if it sounds like you're offering, reread it. That's something that I've definitely learned. Design personalized motivational stories with data. We're going to be continuing this experiment in the fall. We'll be sending a customized email to all Microsofties who have their GitHub linked with Microsoft, so it's an opt-in kind of thing. And then establish communities of practice. I really feel like that's where we're headed next and I'd really love to talk to anyone who's interested in thinking about that. So getting more to talking about getting people to care and do stuff because they saw a metric. I came up with a few possible questions, but of course I loved in the previous session how people also just kind of went off on the things that felt like touch points for them. What are the differences now between recognizing, between recognition, empowerment, and reward in open source that wasn't through, true three years ago? So for me, isolation less opportunity to talk to people, the layoffs, there's just a lot of concern. I want to focus on what I need, but how is that true for others? How is that true for you? Maybe it's not true. How do you think that turns up in contributors who are lending their time? I wasn't surprised really to see metrics where people were leaving communities because of time because it's become more precious than ever. How might we get better at creating curiosity hooks? What is a good way to challenge ourselves to do that better? I'm still learning by trial and error. And what story can we tell people data that gets them to take those actions? And how might we have this community of practice within our organizations? I mean, I'd certainly love to keep sharing with you the things that we try, the things that and I'd love to hear what you try. So that's maybe one of the first places we can think about sharing. Thank you.