 Awesome. Hi guys, welcome to another product school talk. I'm Cassandra from product school. We teach product management coding and data at our 14 campuses. Before I introduce our guests, I want to make sure you guys can see and hear me. So let's go ahead and type in book to get a free copy of the product book. And if you guys already have a copy, feel free to say hello or tag a friend that you think would enjoy the talk as well. I can see and hear me. So awesome. So it looks like we are we're all set. Good. So today's guest works at a really cool company. Many of us are know pretty well. I'd like to welcome Fawn Q, product manager at Spotify. Hi Fawn, how are you doing today? Hi everyone, I'm Fawn and as Cassandra mentioned, I'm currently a product manager at Spotify. Awesome. Well, thanks for being here today. I know you have a presentation set up for us. So I'll give you a couple seconds to screen share. While she's setting that up guys, as you know, after the presentation, we're going to take questions so you can type those in the comments section right after her talk. So it looks like we're all set. I'll let you take it from here. Sure. Thanks. Today I'll talk about how to launch a new product or feature. I'll first start with a little bit about my background and kind of my journey to the product manager role. And then I'll go over a few of the new product I've developed while I was working at Sesame Street and then Amazon. And then talk a little bit about my current work at Spotify, which is mainly around new feature development. So getting started my kind of background and product management. I've started kind of as a research assistant because of my that was kind of my first exposure to product management through working on this online web product called Scratch, which is a visual learning tool for coding. So basically you can go on to this website and then learn how to code. But it's all using visual blocks and some circuits for like children or teenagers for learning code. So when I was working there, I didn't really know that what I was doing was somewhat of a project, what the product manager would do, which is kind of managing the timeline of the product, also conducting a lot of user research, gathering feedback and sending that to kind of the PhD students and the researchers and professors at MIT. And so they kind of kind of distill those feedback into the product. Then naturally I, after I graduated, I then started working at Sesame Street in their digital media department where they focused a lot on developing a new product for toddlers in the educational sector. And then I first started as a business development, in business development department and then gradually from one of the project I was working on kind of changed role into a more of a product development role. And then kind of saw the launch of this product called Sesame Street Go from ideation all the way to launch. It's a video and game subscription service on iOS and Android and web. And it's actually now renamed to be Joe Sesame Street. Then after that I was really interested in education from these past experiences. Heard about a great opportunity at Amazon. They were trying to launch a new product called Amazon English. It's an English learning tool. English is also my second language so that had like a lot of personal connection to me. So I wanted to create an application that helped people to learn English and specifically the target was first in Japan and then rest of the Asia after. And after that I moved on to my current role as Spotify. I am part of a team called App Integration in the San Francisco office and we mainly focus on sharing experiences on third-party platforms and integration with other applications. Basically how you view Spotify or how you outside of Spotify. So I'll kind of get started by talking about my first product real product experience at Sesame Street. When I was first looking at this even thinking about like why should we even develop this application called Sesame Street Go before it even became a product idea we saw a few customer problems. First was that we saw our customers who were parents and kids viewing Sesame Street just had a change of habit when the iPhone iPad launched and linear TV simply did not really meet parents need and then college also didn't have as convenient safe and ad-free way to consume Sesame Street content and this was kind of before YouTube Kids or a lot of like the kids specific video streaming service were also available. At the same time for Sesame Street there was a lack of data in terms of like what types of content that user likes, what is our user behavior, has their taste changed over time and a lot of times we were creating content based on use kind of user research but or more intuition and just expertise but less on data and we really wanted to be able to kind of have a better gauge and access to our data and user viewing behavior. So one idea we had which later became this application is called Sesame Street Go and we decided to make episodes available to parents and kids on the go through an app and also kind of create a sustainable business model that would allow us to continue to engage and retain users on a one-to-one basis versus like through television or through channels like Netflix or Hulu. So we first launched this product, wanted to kind of test like if this concept even would work and if users even want a standalone streaming application for just Sesame Street content. So we launched a beta website that was you know pretty pretty like lean in terms of features. We had a gate where we describe where the product is and then it's what you see on the screen here and then after user signing they kind of get access to hundreds of episodes like about a hundred episodes of Sesame Street content but then even though this product was pretty lean kind of gave us an idea of like just gauge interest level and what some pain points user have while user, while they use this product. We saw a lot of like funnel drop off when we first launched this website especially from this screen to actually consuming the content. So we did a lot of AP tests kind of changing the call to action, changing the copy and also we virtually even added a video describing where the product is but then even though we got like additional increase in conversion the I think the the number of the increase was still pretty incremental versus like what our goal was. So we saw the biggest drop-off happened on this landing page and also registration page and we saw the easiest way to kind of remove this gate is to actually remove the landing page and registration page and then create an experience where the user can see the content right away. So on this screen except the screen before where you can really experience the full product here you can actually this will become the new landing page where user can just go and then click on a video and be able to play right away versus all the steps you have to do before to get to this step. Also as we launched this beta test we also started kind of experimenting with user testing customer feedback and then just looking at our engagement data it really kind of helped us understand you know what features we should do about next and also how we should kind of like surface these features to our users. So for example we had originally had a hypothesis of that when user are using our app they're actually having a co-viewing experience where the parents and the child were watching at the same time and then through kind of just customer feedback and user testing we realized that's not true so we kind of created a separate parent section and we also noticed that kind of season and title based browsing experience which did not really serve the needs of kids so we kind of changed our browse experience to completely just character based and then also as we were looking at our engagement data the average session time was about 15 minutes while each episode of our content was around 50 minutes. So then we started actually dissecting a lot of our episodes into more short form content to kind of really adjust to the viewing habit of our users and a lot of new features like recently watched and favorite were also things that we observed from just user testing and then customer feedback. And also another thing I want to talk about is that the new features that we will build and how you maybe prioritize really depending on the goal that you set for your product at a particular time. For example when we first launched this product our goal was really to increase our acquisition number and increase our free user, paid user, and new user account. Then in order to do that we kind of created this free user experience where you don't even have to have an account to watch a video but then once you reach once we noticed that you watched five video or six videos then we kind of still start to upsell you on the paid experience. And then if similarly if our goal was to engage our user we want them to stay on our platform longer, come back more frequently, watch a video longer. We started adding new content types such as games and music videos especially games we saw a lot more longer engagement time with kids and music video just adds like the new content type that user can engage versus for example instead of going watching an episode here and then watching a music video on YouTube they can just stay in our app longer and watch the music video there. And then as we probably becomes more mature and as we think about monetization we think about how do we convert our free user to paid user and actually one of the most successful things we did that in converting for each paid user was to decrease number of free videos. And then we really had to experiment like how does like reducing one video affect our free user experience that it's still good but also give a boost to that free-to-paid conversion. And when it comes to retention it's really about giving user when that offers or keeping a newsletter to keep parents engaged to keep topics relevant to our users and always updating the content. And then here I share a little bit about kind of just the timeline we started with a beta web launch which is pretty common to kind of set user expectations on you know this is still a beta website there's a lot of changes there might be bugs and then once everything is fixed and we feel confident that this is a good product then we can launch have a full launch on iOS and through the web and then eventually we also launch on digital platforms such as Smart TV and Chromecast. And then when I went to Amazon we I mentioned a little bit the product is an English learning product for Japanese users and the customer problem we're trying to solve was really to help users develop confidence in English but also here English content in a very native format a lot of times when Japanese users are learning English in Japan there's often absent or you're not really you can't really ensure the quality of content that you're getting or English learning that you're getting. Also at the same time at Amazon there's this product called Kindle there's also a product called Audible and there's this technology that we developed where user can listen to an audiobook at the same time while they're reading and then the app would highlight the entire content for users and we found this to be a really incredible tool for people who are learning English so we wanted to kind of leverage this for a new audience and then we product the product we created was basically a browse experience of a lot of the content that we curated specifically for English language learners and then also a listening experience where user can follow along to a book they selected reading it but also listening to how it's pronounced at the same time there's a lot of educational tool we created such as the ability to tap on more to get the definition and then also serve user a quiz after so they could test their understanding comprehension of the content. I think the most challenging aspect of this project was really figuring out the level of content to serve to our users so we started with a top of funnel kind of user survey where we're asking what their English level is how they feel but also getting their test score that almost most of our user will take and then based on that test score we serve content that is right at their English level and then we also have a filter section where user can filter content based on their level of interest and then as well as the the English speaking level that they feel comfortable at. And then I'm going to quickly skip over this slide but then going over to a little bit of our product launch timeline we we took a little bit different approach than what I did at Sesame Street so because this was a new market for us we first tested this product at with all Amazon Japan's employees so we get an employee only beta got some feedback may some changes and then we had an invite only launch specific to a group of user we identify as they're very interesting learning English and we and we thought like by kind of identifying these group of users and sending this product to them we can kind of get more learning from them not just from the internal employees who might be a little bit biased because they work for a company that the product is created for and then I'll jump to a little bit about my current work at Spotify so Spotify has been around for many years now so a lot of work we do are testing new features or making changes and improving new features so I'll talk a little bit about two features I've worked on recently we this year we kind of revamped our share flow we had an old share flow where a little bit confusing copy like when you want to share a song it says post to or send to and then based on what you tap on it kind of leads you to different sharing destinations so we wanted to kind of reduce the friction and confusion on that experience so we came up with like two options that we wanted to kind of replace this old experience and then the new one is version one is where you can see most of the destinations line by line and then you can surface all of them but then each of the destination is a little bit smaller and the other version which is a version two has kind of more prominent buttons and more prominent sharing call to action so we thought maybe you know both have different type like both have different elements that with UI that we think would drive users to share more but then we really wasn't sure which one would be better so we actually ran an AB test test both like version one and version two that's like the second screenshot of third screenshots that you see on this page and then based on those results we decided to kind of go with the version one which led to more conversion for our same terms of successful sharing rate and then another features we launched recently was kind of screenshot share so a lot of users love to take screenshot screenshots and then share to their social media channels and we have no tracking on one's users share that screenshot to another platform and no tracking of that and their user have also have no way to come come back to our platform so by kind of implementing this feature it allows user to kind of easily first like reduce their friction to share a screenshot to social channels but also kind of on the last screenshot you see when users share a screenshot they also get a link back so whoever they share with can come back to Spotify and listen to that full song versus the image the user would have used to share and then that just really wouldn't have that kind of loop experience that we we're hoping we can provide to users who share and their audience who views that share and then with this feature specifically because we know that users previously did not have any way to come back to our platform and also there wasn't like a better way for users to share screenshots we just launched this feature with a hundred percent of our users right away without kind of doing a beta to see if it's going to kind of hurt any of our other metrics and then I'm going to quickly go over some like common strategies for launching product features so I kind of mentioned a little bit employee only rollout so a low risk way to collect data get feedback from employees but sometimes the feedback might be biased there's an invite only rollout where you can kind of really control the size of users who are using your product and also you can maybe do some selection of the type of user you want to get very similar to like limited user rollout where you can kind of restrict rollout to a specific user type for example geography if you only launch this product in the US or if you want to only launch this product for iOS 11 you can choose to do that too there's a beta version beta rollout where it really helps set user expectation well maybe the user who actually end up trying this are more willing to give you feedback there's AB tests talked a little bit about that and I think that's very commonly used nowadays there's gradual rollout rolling out to five percent of users 80 percent 99 percent just so that there's a period of like maybe it helps with like bug discovery helps with if you on if you there might be up maybe chances where things might go wrong so it gives you a chance to not affect your entire user base and then there's also the full rollout which is actually very common for product or experiences that requires all of their users to have that consistent experience for example like if you want to launch like a chat in your in your app you really have to launch it for all the users so that they can communicate with each other and then also other things to remember is to always set goals and track metrics and then communicate with your customers if it's a huge rollout and also create a long checklist of all your stakeholders including legal customer service and public relations are ready for when the roll happens and any aftermath that might occur cool so i'm done with my presentation i'm happy to take any questions awesome thanks Vaughn for a great presentation too guys you can go ahead and start typing in your questions too so we can we can get those going um here hey let's see we had a couple come in through um Slack as well too um can you share some of the yeah can you share some of the books or resources that have been your favorite throughout your experience yeah um so yeah when i was um interviewing i really uh liked cracking the pm interview i thought that was like a very good outline of um that pm interview process and i think it does depending on like which i really like the fact that they separated it out i think uh what a pm means in each company so i think for a lot of people who are maybe looking to the pm positions it really i think what the pm rule is it is a little bit different company by company um so i think that really kind of outlines like maybe what you could expect at your work a little bit but then more like broadly if you're if i'm just thinking about um product in general um i really like the book um inspired it's kind of the first book people um read if they are interested in product management and understanding the difference between product managers product marketing managers and project managers i also really like um the blog put together by intercom they have a lot of very interesting kind of just very up-to-date best practices of what they use in their company but it's very it's like short and sweet and also very up-to-date so i really like just reading those blogs um and recently i think i was recommended by a podcast called clearly product and it's actually when you ask me like which book i like it's actually a really good podcast because each podcast talks about a helpful product books that you can read but also they summarize it in 30 minutes so if you're like short on time yeah a lot of books listening to that podcast it's called clearly product okay awesome um thanks and we had quite a few questions come in now so i'm gonna check here and see um pick one of these um here so this one is from Keith um what were some dates related to your timelines between each milestone and how did you come up with them yeah i think the dates isn't really interesting because sometimes it's dictated by maybe in your within your company you have some deadline that you want to reach um for example um with when i was at Sesame Street there is um certain like like right before Christmas we see a huge um like boosting like number of apps downloaded game spot a lot of um parents or like grandparents or caretakers they like to give apps as gifts now um so we're really trying to kind of hit that date for um our like all of our ipad iphone android basically device launches um so sometimes it's dictated by that sometimes dictated by um partners like when we were working with um promcast to um expand that uh Sesame Street product to promcast um we were trying to hit their big announcement so that we get a lot of promotional opportunities through that um so sometimes it's it's dictated by a timeline by company or by your opportunities and then you had to work backwards a little bit maybe you had to slim down your product a lot um but always i think um we should always build a minimum viable product but then i think when you have a deadline here you will even more pressure to do so um other times it's about hitting certain um metrics for example when we were launching amazon english um we did the um invite only beta and then we sent um this invite to x number of users and we said until we get um x number of subscriptions we're not going to roll this out to um the broader country um so and if we don't hit that number we understand like maybe there's just isn't a product market fit so we need to make additional uh feature improvements to in order to um uh hit that number and then we feel comfortable to kind of roll this product out to the broader country so it's a little hit or um like a little different based on i guess situation and then when you do a lot of tests sometimes the test result will inform you um whether this product will all make sense or if you should revisit and go back to the iterative product development process to kind of start over um and then make some changes yeah awesome um this uh this next question is from john um tori so what factors prompted Spotify to decide in investing in its sharing feature like was it user feedback or leadership if you can share that and then furthermore um how do you and your team develop ideas and decide on which ones to start building and testing against yeah um sharing feature i think um yeah i think it just i think even since Spotify like before i joined Spotify um i think just sharing music um like when the age when the people are like spamming on facebook like a lot of like Zynga like those games i think that's that's kind of like when Spotify also started um in united states and um through a lot of these like sharing and music updates we got a lot of new users um so there is has always been like a history of how sharing on social platforms has led to a lot of um users to uh Spotify so there's always already like this history of like investing and share but then definitely we're doing a lot more these days um sharing music sharing and thinking about creative ways where we can share music we see it as like a really great acquisition channel but also engagement channel and also we don't have a lot of like social features on our own platform that you can on the desktop you can kind of see your friends feed what they're listening to but we want to we know music is kind of a experience where people love to share with their friends and family so we want to make it capable available for users on the platforms that they're most uh that they use most whether that's like facebook or twitter or something else yeah and then um another the second question was like how do we decide which one to invest in um or to build um yeah i think it's always um like challenging when you have a lot of different ideas or especially for our team there are so many partners and apps that we can integrate with um i think most of the time now we're really considering which um platform will bring us um the most um the most users and how big is our platform so maybe before we were very interested in trying um like things out with like smaller smaller smaller apps and smaller partners but now we just see the acquisition channel and the kind of the loop is much bigger and much meaningful to us as a company when the partner has a lot of existing um and they use um and i think it's kind of translates into that funnel where if you start really big um then the user that it brings to you um is a big number two awesome um we have time for just uh one last question um this is from tambi could you speak to what product teams look like at spotify and how do you influence your team to see uh your vision or align with your product roadmap yeah so as spotify the product team um um it's it's actually quite similar to um what i will envision like my experience in other other companies um it's pretty flat um it's very agile i think one great thing about spotify product team is that they try to keep everyone local so that's like really big bonus that your engineers or designers um are all with you and we work in kind of a squad formation so we are like one team every one we have a dedicated um well not always sometimes we share resource too but most of the time if you're working on a project you have a dedicated designer you have dedicated engineers and then um you kind of aim to kind of launch a product um and then everyone is fairly involved um from the ideation to kind of forming a user experience to kind of building I think everyone's pretty involved so that's like I really think it's a really good thing just so that you are not kind of in this uh you don't get tunnel vision into like you know ideas um and um and uh I think another great thing is um it sometimes most company nowadays use like agile um but um I think spotify also uses agile but we are pretty um quick to change the way we work if we notice something doesn't work um for example my team we went by scrum for like a good number of month when it first joined but then um the team just really had a feedback that they felt like the meeting was too heavy and then we actually changed to Kanban which has a little bit of less um planning meetings and like um a lot of the process and procedure that are very common just from um so we went to that but then um I was talking to other product managers and we thought like well actually that became a little bit hard for us to have transparency in how they work um so we're now suggesting maybe some kind of in between method where we do a little bit more planning but also keep keep it like light so that um they don't feel the team doesn't feel restricted by that um yeah and I say it's it's pretty flat um and we have locations like New York and San Francisco and Stockholm and I think one great things about Spotify is that we really encourage the team to travel and learn from other teams um so I had an opportunity to go to New York and Stockholm I think that's always been helpful to you um feel like more connected to right right yeah this is like awesome um well thank you again for your time um really appreciate the presentation and taking the time to answer some questions um before we let you go could you share your final advice for aspiring product managers out there and those that are trying to break into the field yeah I say um try to figure out what your strength are in terms of being a product manager because a lot of people say product manager is is part of engineering part of design part of business um I say try to find out like I think it's impossible to for someone to say I'm good at all three um trying to find out like what is your strength are for me I've um just through working I know that I have a very good product like design sense I have a I really enjoy like talking to users and understanding their user behavior um so that's what I enjoy to do enjoy doing and then um and then throughout the process you can maybe figure out like if you're good at it or how you can improve on it and then try to get into a role maybe that will play to your strength so for example if you are really good with numbers um and you're very technical maybe start a role in like data or engineering first if you can't get into product and then you can maybe slowly switch into that role like I mentioned for me that I started in business development first because I had a business background um and then by being part of that and then also like kind of show that I have a lot of good product sense I was able to kind of transition into that role so I think um a lot of times like starting unless you get into like an internship or apm role it's really hard to break into product um I think getting into a role where you can play to your strength and then transition um is a good strategy if going into a product um doesn't yield to the success you want right away yeah awesome um well thank you so much again I appreciate you being here with us thank you thank you guys awesome and thanks to and thanks to everybody out there that was watching um if you want more information about us you can find it at productschool.com um hope you enjoy the rest of your day bye guys