 Cool. So just by show of hands, how many of you are in product management roles already? Cool. OK, great. And how many of you guys are in startups first? OK. And bigger companies? Cool. OK, so it's a good mix of folks. Great. Well, we're going to spend about 20 minutes or so. I'm going to talk a little bit about my journey at Facebook and one specific thing I've learned about product management there. And then we'll just open up for questions and hopefully learn a little bit from each other as well. Cool. So let me switch here. So what I want to talk about is kind of a product management approach of what we call people problems first. So at Facebook, we actually don't call our users users, we call them people. So we don't have monthly active users. We have monthly active people. And just that one change in how you talk about the end user changes how you think about the product you're building. At least that was kind of the big change for me on how I was pursuing this is not just a user. I'm trying to get to do a thing on my website or on my app. But it's actually a person behind the product that is trying to accomplish a task. And so that's what we're going to talk about today. A little bit about myself, I'm a product lead for a profile and identity team on Facebook. So it is everything that has to do with your profile when you land on there, with profiles of other people when you get there. It's a second most visited surface on Facebook. It's huge. Everyone has it. Prior to joining Facebook, I had my own startup for about three to four years. It was in travel and local discovery space. I'm also a huge USC fan. That's going to be relevant a little later. Any Trojans in the house? It's a bummer. It's a big game tomorrow. Cool, OK, so let's dive in. When we talk about product ideas, there's always this question like everyone has a different idea of where do product ideas come from? What do you guys think? How do we PMs, product managers, come up with product ideas? Use of problems? Use of problems? I would say the customers. Customers? These are needs. These are needs? Cool. The engineer. Engineers, yes. Competitors. Competitors. It's a good one. How come and do the group? Yeah. Cool, so a lot of different ways to come up with ideas. When I first started working on products, I thought that you have this strike of a genius, come up with idea. And I think a lot of times, as the first time start-up founder, which is how I enter into product management, you have this vision, or you have this idea, and you want to come up with a way to make the world a better place. I also thought that you just get a bunch of people in the room, potentially, just brainstorm a bunch of ideas, and you come up with something revolutionary and great. Of course, both of these methods do lead to some good ideas. But as many of you probably know, Paul Graham, he said something really stuck with me even in my start-up days, which was, why do so many founders build things no one wants? Because they begin by trying to think of start-up ideas. That MO is doubly dangerous. It doesn't merely yield few good ideas. It yields bad ideas that sound plausible enough to fool you into working on them. And when I read that, first I was like, ah, I should have. I wish I read this four years ago. This is like three, four years into working on my start-up. Second, it just resonated so much with me that it's not about coming up with an idea or a start-up idea. It is about thinking about the people problems. And so when I joined Facebook, what I learned very quickly and very early on, and we're lucky to have a huge research team, but what I learned is that it is all about customers. It is all about understanding what are people doing, what problems they have, and how do we help them just achieve a task and be able to have, whether it's a utility or some other way to entertain themselves, but it can actually help people do the thing they're already doing in a better way. So what does that exactly mean? I joined Facebook, and this is actually an image we used in some of our onboarding. This is literally what I felt like as a product manager on the first day. I was like, I have no idea. I had a start-up where I was like the CEO and the legal and everything else you do. And as a PM and a big company, it was my first time in November 2014, which is when I joined Facebook. And I joined the profile team, so I was working on the product that was very different from what I was working on before that was used by billions of people. And the question that one of the things I had to figure out early on was, well, what is the big next kind of strategic idea we should tackle? Where is the opportunity as a team, as a product, that we should go in? And so remembering Paul Graham's quote, I thought, OK, I should not be coming up with an idea. And probably the first step I should take is figure out what's going on with our users today. And so in this kind of journey of people problems first, the first step is always to look at research. And if you're lucky enough to work at a big company or a well-funded start-up, you probably have some research. You have some capacity to look into how are users using your product today. And so this is both look at the past research if you have any, but also doing interviews, looking. Actually, it's interesting that people say, how do you make decisions based on just what 10 people said? The reality is that after you listen to seven or eight, you literally hear the same patterns, like top two or three things come up. You're just like, wow, incredible that you have such a usually diverse sample of people, and they all encounter the same problems. And we try to do it. We have a very international user base. And depending on your customer base, try to get an interview from customer base that represents people who are using your product. So for us, it's not just folks here in the Bay Area. It is someone on a very slow connection in the merchant market. It is someone in maybe Germany on a very different network and very different cultural points of view about the product, how they interact. And so we do research internationally. We do a lot of remote research, as well as bring people into and talk to them. So we've done all of this. And this was kind of like the first couple of months on my team before, like even diving into building products. I was just trying to understand what's happening. And what I saw is that we saw a bunch of different patterns. But one of the things that was very interesting, over and over again, from small towns in Kansas to Cairo to New York, we saw people use profile pictures to show support for things they care about most. And this was fascinating because there's a lot of different themes. But this was just like super international. Almost everywhere we talked to people, we saw examples of people using this space to express what they care about. And it kind of made sense because in real life, people paint their face with their favorite sports team or wear a t-shirt to support their cause. And they were using their profile pictures as a way to kind of express that in their online identity. And it kind of made sense to me because I'm a USA fan. I wear a lot for the games. I wear the hoodie and sometimes paint my face. Probably going to do that tomorrow. And you go to the game. And so this is like online. This is how you represent that same kind of passion. We see the same thing with causes. People wear the same kind of shirts and logos to show support for different causes that they care about. So when we look at all this research, one of the things we saw and we kind of had a hunch around is that this could be big. There's something really interesting here. These people really care about something, about their personality. And it's something that helps people feel like they belong to a bigger tribe, just something bigger than themselves. And something that's also really associated with their core identity, which is what we on the profile team really care about because profile is a place where you can represent yourself in the Facebook community. It's kind of like when you wake up in the morning, you dress up and you go outside and you go to work. How do you dress up and how do you want to represent yourself? We enable you to be able to do that in the online community. And so this was one of the things that we saw as an interest opportunity. And Facebook is used by over 1.3 billion people. So we can enable them to help them express what they care about. Then the second step comes in. It's like, oh, OK, cool. This is so exciting. With all these people do this, let's figure out something here. But the second big question is data. How big is this? Can this be really big? Can we really? Is this something small where we come up with a couple ideas and that's it? Or is this be really big and exciting? We didn't have perfect information, but we knew it was big enough already. And with the right ideas, we could make it really meaningful at the Facebook scale. So we looked at a bunch of data and I highly encourage you, no matter how big or small your user base, always think about it. If you see the pattern in research, how can this translate into bigger numbers? Whether it's your entire user base or maybe the potential of how it's going to grow, how it's happening in the real world. And could you potentially grow your user base because this is already happening in the real world and they could do more of this online on using your product. So step two is data. And then that leads me to step three and another program quote. Let's look at something that people are trying to do and figure out how to do it in a way that doesn't suck. It's very blunt, but it's very, very true. Because it's much easier to help people do what they're already doing easier than trying to get them to do something they're not doing at all. So when we looked at our product and we looked at this, this is a real person's profile. I heard like, whoa, this is like a lot of work to do, right? And so we were like, well, what are the current pain points of this person trying to represent their really go blue? They're all about that. It's like, well, they have to go out. Stephanie, she's like, well, I go online. I download an image. Then I download another image. Then I upload them and I try to crop them so they fit the certain layout. And just really, they spend a lot of time doing it. And it's not viral. So they put it up and then they're all their friends, like, how did you do this? Then they try to do it. They may not have Photoshop. So she ends up being the only one with this awesome profile. So we looked at this and said, OK, cool. How do we help her do this and make it not suck when she's doing this? And so we brainstormed a bunch of ideas. And we came up with a few. And some of you may have seen these. But we came up with a really, really, really scoped-down MVP. And I'm talking about scoped-down, super scoped-down. One engineer and me and a designer just iterating on this. Took us about four weeks from brainstorming to a public test. And what we had is just a few. So we said, OK, we can't really boil the ocean. We can't help everyone do this. We can't come up with all the. It's like, you can go abroad. You can say all the causes and all the sports teams. How do we make this really awesome? I said, no, let's just pick one or two. And let's just see if we can make this really work. And this was August, I think. So college football, as you can already tell. I'm really into college football. College football is coming up. So we reached out to a couple of teams or partners that we had. One of them was Gators and there's Alabama and a couple of Oregon Ducks, I think. And we got an MVP out. It was OK. Just put a frame, what we really built was just the call to action at the bottom. And let's see if people use this thing. Within the time that the pages posted this to the game, which was, I think, a day or two, we had close to a million uses. And this is like, you could say it's a Facebook scale. So of course, you'll have a lot of users. However, if you think about how many people really care about Fluridogators, maybe not a lot, but not at the Facebook scale. So the denominator wasn't that large. And was that set still? People were doing this and they were doing it at a scale that pointed us in the direction that we are going for something really exciting and big. So the step four is always to figure out what is the minimal viable product within one user base or some group of people where you can test and see very quickly whether the thing is going to work or not. And there are so many things we cut out of this product. The list of things we cut out was probably 10 times longer than what we actually built. And it's so much, actually at a bigger scale, it's harder to cut things out because you have the resource and you can't be like, well, we could do this and we could do that and wouldn't it be nice? And we had this focus on let's prove that this thing is actually going to work first. Once we knew it's going to work, and we got a bunch of press actually out of it, which was hilarious. And this is when we were like, okay, cool. This is going to be big. Of course, TechCrunch wrote about us. This is awesome. Then step five is actually figuring out the V1 of the product. So we're not talking about going from zero to 10. It's not like let's take all the lists of things that we decided not to do for MVP and now build it. Let's take the next two or three things that were like next P1s, priority ones, and let's build those. And then collect a lot of feedback. So what we decided to do here is empower more organizations to connect with people in a whole new way to join movements around the world. And so in October, 2015, this is about two months later, we launched what we call the beta platform for pro-phobic frames that come from verified pages. So we didn't go to all the pages, we only went to verified pages and only subset of them. And we said, hey guys, we build this platform, you can create your own frame and you can put it out to your fans. And we got this overwhelming just positive response from partners, everyone from all the sports teams, I mean, FC Barcelona, Real Madrid, each of those pages have over 80 million fans, different movie studios, other sports teams, NFL, NBA, et cetera. So we knew that there is excitement on their end. And then one of the first frames that we also launched was a French flag overlay that came directly from the Facebook's main page in support, this was in November of 2015, the support of the victims of the attacks in Paris. And we just were absolutely overwhelmed by the adoption of this frame. There were over 120 million people who used it in just two days. And this was just a moment of solidarity that everyone around the 120 million people were coming from all over the world. And it was just like, for us, it's still a very small team. It was like truly moving to see so many people coming together in this really dark moment in the real world to connect to our platform and to really show support. And so this really overwhelming adoption showed us that we need to create a scalable solution that for any organization to connect with their fans and to start movements at a global scale because this platform could really power that. And we also knew that we can't possibly then create frames or art for every occasion, a partner or organization to support all the billions of people. And this was a time, this was a moment where we knew we had to build the one to 10 product. We really needed to build this, the fully crowdsourced platform that does support all the causes and enables people to show what they care about. And so the step six of this was building a fully-scaled product. This took us a while. So we're talking from November to a few months later where we build a fully-scaled platform that allowed anyone in the world to create a frame, whether you're a person or a page, to submit it into the system, it would get reviewed and they would become available. And so actually all of you can do it today if you want to. And we have frames for all types of things, like small high schools in the middle of nowhere, for their girls' teams, a chess tournament, just all types of things. And at this point we knew that it was a time to, we actually had a product market fit, it was a time to go that one to 10 step. And this is just a sample of some of the frames that currently live in what we call the crowdsourced store. All of them are crowdsourced and they change all the time based on what's happening in the world, what people care about, et cetera. And now this product is one of the more adapted products. People use it all the time both to create frames and also to just, for Christmas coming up, a lot of people are putting up their holiday mood and expressing that, which is really exciting for us as a team. And so this is kind of one example of many different ways and there are many different examples of how you can build a product that really thinks and focuses on the people problems first. So what we talked about just to recap, first starting with research, what are people doing today? I would have not, for a life of me, came up with that idea just sitting randomly and thinking like, what should profile team do? Research would have really powered that. It was big enough with looking at data. We saw the pain points in the current flows. We build an MVP very, very quickly and this is so important. You have to be able to get out something. The step four is very important. You have to get something very quickly, get that feedback. And then only then you build a more scalable version but you don't do the feature creep, like still focus on what is important, what are the hypotheses you're trying to prove and disprove. And only then you're iterating, getting more feedback and then the last step is shipping a fully-scaled product when you know there is a product market fit. And that's how you can survive also the startup life and be nimble and not get into a feature creep. So this is an example of how I did Facebook, how I learned about kind of this people-first approach and I'm happy to now open up to questions. Thank you, Chris. Like you said, like you're having a situation in TechSport and how they come up with it. Yeah. Why the teams? The teams were pretty random. Those were more like where we had... It was like numbers and like the data kind of back to... We looked at, well, the numbers did play a little bit of a role where we looked at which teams had a decent amount of fan base in terms of like the page has at least over 100,000 fans. So we know there's some foundation. The Facebook page is right. So when we partnered with them, we would have like the Florida Gators page would actually make a post, was the frame attached to a post and then that post had that CTA that try it button at the bottom. And so like when the fans would see it, like they could update their profile picture and then when they updated then their friends would see it. And that's kind of like that was like the key component that we wanted to build is a viral factor that once you seed it into the system and then people can start updating would that create kind of the network effect and get more people to do it. And actually we also saw a lot more people ended up following these pages as a result of this because they would see their friends do that. Oh, cool. This is such a awesome and yeah, yeah. Can you maybe describe a little bit how did it work back if you lost you or if a person lost you? Well, we didn't have a platform at a time. And this happened and we felt like this was a moment where our platform could really help unite people and help people express solidarity. And it was also something that was like very timely. So we had to get something out quickly. The flag overlay, we already did something similar before it was pride. So this kind of followed that pattern. And then the Facebook page made the post was the flag and then we had a little over 150 million followers of the page, I think. So anyone who saw it, they adapted it and then their friends could see that kind of update and then it went viral. Yeah. That's a great question. There's a lot of feedback. So we actually have like a system where you can file bugs. So actually like our end users can report bugs or can report their feedback. And we have, so we get those. We also look at, obviously we do a lot of research. So we bring people in house. We do remote research and interviews. And then also data. So just looking at patterns, what's happening across the board and kind of like combining those three things together is how we prioritize. We, it's kind of was a combination of both. So like we heard a couple of times in research enough to say like, wow, this is interesting. And then we looked at data. It was because like we don't classify if you just upload a profile picture that is like, let's say a, you know, go blue. We don't know if it's go blue or if it's just a regular profile picture in this early. So once we knew that there was something going on there then we looked at data to see if there's actually a pattern. We had a hunch from the beginning that that's where we need to go eventually. But probably within the first months or two, once we, you know, once you do go blue or gators, all these other schools, right? I say like, well, what about me? I want to do one cause. And it's like, what about all these other causes? That's when we knew there's no way we can design for every cause. There's no way we can even guess what every single person really cares about. And so pretty quickly we knew this is a direction we need to go into. There's a question in the back. We would probably look at, we looked at kind of how many people did it versus how many people were exposed to it or had an opportunity to do it. It depends product by product. I mean, in this case, we had over a million people do it within the first week. So it was pretty clear that this could be pretty big. And if it was a couple of thousands, it really depends at that point how much we felt like we want, you know, maybe, one thing in that case, when you get kind of results that are lukewarm at the early stage is to go back and say, did we actually, did we execute perfectly? Did we pick the right audience? Were there any biases? Were there any bugs? Were there any other issues? Before kind of like giving up, I want to just make some numbers. From this six step process, what would you say is the average length? What would you say is the section that you spend the most time? I mean, there's no average length, to be honest. I would say the step from once you have the research and data to actually building the MVP should not be more than two to six weeks. Because you really want to get something out quickly and get that feedback. The research and data part may take, actually I often encourage people to take your time, make sure you really understand. It's so easy to jump to assumptions once you hear something that potentially is exciting to you and the idea you're like, oh yeah, I'm absolutely a genius. I really thought about this. It's very tempting to jump in and start building, but that's really actually the stage where you should kind of avoid that urge and make sure that you really understand and validate it. So the step one and two could take longer than actually step three of building, for example. And then from that to building out a fully kind of full product, the one to 10 product, that could take a while. That actually, the last step probably took the longest for us, but it's because we wanted to, we iterated a bunch, we were looking at data. Where are the drop-offs? What's happening? Doing a little bit more research understanding. What were the pain points? Where are the products sucked now that we build it? And really getting kind of the right flow. I have a question there. Yeah, that's what it says, we did that failed. Let me think about that. I'll get back to you. I'm gonna think about it a little second. Yeah. If we do this and then what are the behaviors that drive our revenue, how do you fold things back? What is the process? Yeah, I mean, as a PM, you are, you're responsible not just for the success of your product, but for the success of the overall health of the app and the product in the system. So we always have counter metrics and top level metrics that we look at. And if we're significantly negative, your products are negatively impacting something. The first step is to figure out why, right? What is happening? What is, how are you changing user behavior that is so negatively impacting a product or a metric? And then if it truly is like, this is just a trade-off, then you have to think about like, what is your recommendation? Like a lot of times as a PM, even if it's a successfully free product, you actually like, no, this is not the right thing for the system because of the trade-off. So we gotta pull this back. But sometimes you may feel pretty strongly that this is the right trade-off to make and you have that discussion with the product managers or the leadership and then you figure out kind of what to do next. Depends. Yeah. It depends. It depends on the button. There is a question in the back, so. Yeah. Like, are there people changing their profile to quality, a lot of pictures to quality? To quantify that or like, is it just depends when you're in there? Inside time, photo and all, so that's the word. Yeah. I mean, I had a startup, so I can totally relate to lack of data or bad data. So it's a combination. When I say like the step two of data, it doesn't necessarily have to be like, you have all these walls of data as like, your users are doing this thing. A lot of times it is looking at the competition or even how they're doing it in real world. Could this be really, really big? And it doesn't necessarily have to be happening on your platform today. So a lot of times it's a combination of kind of understanding the market landscape and also intuition. And this is where like, big part of this was actually intuition from our side where we felt pretty strongly that like, people really, really care about these things. And like, if they go out of their way to paint their face in the real world, like they are likely to click a button to show how they feel on our platform. And so it's kind of a combination. It was a lack of data, it's coming from intuition and then any data you can find to validate it. Going back to your question. So, I was thinking in the back of my mind. Well, I mean, I definitely can talk about my startup, which I didn't really cover much, where my product management experience didn't go so well. And so one of the ideas we had, the original idea was around building a platform to connect travelers with like-minded locals. And so the hypothesis we had is that like, imagine like when you travel, instead of just going on the top 10 things to do, you can actually meet someone who is similar age, has similar interests, and you could kind of almost have like a local friend anywhere you go. And the hypothetical, when we pitched this, everyone loved it. And I was like, yeah, like I would totally use this. So great, I'm such a worldly person. And we built the first version of it and we had a bunch of actually local sign up, which we thought was gonna be the bigger problem is getting locals interested. A bunch of local signers were like, yeah, we were so excited, like we're here in like Jakarta. We want all these Westerners coming here and like to get to meet them and practice our English. And then we're trying to market it to travelers. And they're just like, yeah, I mean, they would sign up. And then when they go there, like, they're actually want to do the top 10 things to do. They don't want to go on the beaten path, even though they tell us that, even in research, right? They would be like, yeah, like I'm totally worldly and I want like that hole in the wall place. Actually, no. And so this is one of the, like, and as a small startup, it's like terrifying when you've realized you spent all this time building a product and there's no product market fit. And you have to like write these like updates to your investors and be like, this thing is not working. We're gonna have to pivot. And that was pretty hard. But the numbers were so clear. And like when we did that, that we knew that we had to pivot. I guess it makes me feel better. Thank you. It's a good point. It's a great question. How fleshed out were those ideas? There were, what was fleshed out was the key hypotheses we had. Like we knew what were the questions we need to answer to actually like know if we can even convince to put more people on this thing and like grow this product. The actual ideas changed a bunch. I mean, we had the early, like the early, early idea that's how it's gonna work. And then we put it out in front of people and they like did not understand. Even like everything from the copy, like what the copy said. And even to this day, like a bunch of people think this frame, this art, there's just a ton of art in the system. A lot of people think that Facebook makes them, right? And we've done everything we could to try to communicate that this is done by the Gator's page and this is done by FC Barcelona, et cetera. And that is still an ongoing kind of product iteration on our side. So a lot of things have changed from like the early days. But what's really important is to know like one or two things you absolutely have to prove are working in order to kind of move on to the next stage. Before you get to that stage. Yeah. Yeah. So we actually have this really awesome thing called hackathons. So I pitched this to like, I just need two or three people in my team who probably would be interested. And student engineers were super interested in helping out. And so this actually started as a hackathon where we have like what you basically have 24 hours to build whatever you want. And then you, everyone gets to present it in front of this kind of like a group of judges that are picked by the committee, the hackathon committee. And then from there they pick the top projects and you get to pitch them to Mark. So that's how we got to, and we also internally like show them to all our VP like here's the stuff that we came up with in our hackathon. And that's how we got the early kind of pitch. In terms of like pitching engineers and designers, I, a lot of it was like trying to identify who was interested in like causes or sports. And just being like, wouldn't you want to FC Barcelona for a profile? And then like helping them kind of envision, get it, get it, get it as excited as you are. And pitching it kind of from their angle and then taking it from there. So that's how like actually, yeah, like the first two guys I pitched the, they were, they just bought into the sports idea and we took it from there. I didn't know when you were done with it. Like how do you know when you need to like sort of like kind of get to the point where you're off of it and just be doing it. That's a great question. I mean, I think like once you get to that stage of like maturity of the product, so you definitely pass the product market fit and we felt like a lot of times it's not like, there's never a stage where you're just like completely done, done where you're like, you're always iterating, always getting feedback. But I think when you get to a place where you feel like this, like people, it's working on the zone. People are using it. We got into a place where we feel like actually internally proud of the product we've done. We feel like this is, like this is the baby we're really proud of. That's usually the station and there's also like, well, what are the opportunity costs, right? What else we could be working on, what are the priorities? That's how we shuffle things around. You mentioned that this one came out of a hackathon. It sprints a lot, like a design sprints or something. It takes quite a lot of time for it to leak. We do use design sprints cycles. We did not use it for in this particular case, but yeah, it depends on the scale of the project. I think like usually something medium scale, we would use like a design sprint, something smaller, like with hackathon ideas. A lot of times you don't even have the resources to run a design sprint. Like five day thing was like fifth day being the in lab research. A lot of times you just kind of hack something together and get it in front of the employees, which is enough usually to get feedback. Let me take it from there. Oh, in this case, we did not have it and mostly because we didn't have the resources. We would use a design sprint in the cases where we already have, for example, a buy-in from leadership or from our management to make a change or a bigger change or there's initiative or something that we really need to go tackle. Then you have a bigger group of people because design sprint requires five days and you need cross-functional partners and kind of a whole spectrum of things. Thank you so much. Yeah, of course. Pretty like, that's the most awesome part about Facebook is just super move fast. Anyone can push code. You have to get your code reviewed by a peer, but usually, you probably wouldn't be able to push it 100% of the world, but a small test is pretty like, depending on what team you're on, you may need a buy-in from, for example, if you're on a profile team, you may need a buy-in for profile leads like me and my edge counterpart, but you may not need a buy-in from all of the leadership. That's something bigger. It depends if you wanna go 100% of a country or 100% of the world or it's a bigger change that can impact other teams or the product overall, then yeah, depending on the complexity, you need more and more buy-in. Yeah. Yeah, it's less about the frame itself and it was more like how, like so for example, the Gator's thing, like only US, we were only open to users in US, for example, but if we were to roll out globally, it would be something that would need more cross-functional buy-in, or like more leadership buy-in. Thank you for a lot of questions. How many different product teams are there at Facebook that you know? A lot. Actually, I don't even know how many there are. Depends also how you find product team. I mean, for profile, we're one product team, like a profile product team, but there are many, many dozens of product teams. There's Messenger, there's News Feed, there's Instagram, groups, there's like so many. French groups? French groups, like. I mean, I personally haven't, so no, I'm sure there's something happening. You have that, like a bunch of hypothesis that you want, and then another is you also have like a pipeline of ideas that they, before you say, okay, let's get Tinder and users on this one. Or like platform users. Yeah, yeah. We do have a bunch of hypothesis. We have always rolling research. There's always some kind of new information coming in, and we want to make sure we capture all of it. And yeah, and the same with kind of ideas. And it's all gets prioritized based on a number of different things, like yeah.