 Okej, awesome. Let's start then. So this is the story on Discover Weekly, or how autonomy saved one of Spotify's most loved features from being killed. Or also how we accidentally fixed Mondays, more on that in a minute. So how many of you know what Spotify is, streaming, music streaming app, okay, almost everyone. How many of you know what Discover Weekly is? Okej, not everyone. So Discover Weekly is basically a feature of Spotify that means that among all your playlists, your user generated content playlist that you create or that you choose to follow, all of a sudden there's something that pops up that's created by Spotify that's called Discover Weekly. It contains 30 songs which you have never listened to before on Spotify. Spotify doesn't know what you have listened to outside of Spotify yet. So we'll see. And these are refreshed every week so you get new 30 songs that you've never listened to before, but that there's a high chance that you like. It's personalized for you. So how does Spotify do all that? Well, first let's take a step back. Do you remember how you used to find music before Spotify? Something like this. This is Amoeba Records in San Francisco, I believe, LA, this one. Okay, it's in San Francisco too, but maybe you recognize this from LA. And yeah, you used to go to your record store browse through all the records and then Spotify came along and did this. Okay, you now have 30 million songs that you can search. And for me, who grew up with Napster, Kazaa, and all these illegal piracy download things that were kind of really awesome, used to what you had to do before, but you always had to search for new things and see, oh, I wonder if this is available. And I hope the user that has this is on a fast connection so I can download it to this party and play it. So I immediately started typing things into Spotify when I first used it and realized, wow, this is super fast. They have everything, even my more obscure artist that I listened to. This is really, really cool. Everything is on Spotify. I just created tons of playlists. Then I showed it to my mom and dad and they were like, what do I put in the search field? I don't know. They were used to just tune into radio station and be told what to listen to really. Well, what do you listen to? The Beatles? They're not on Spotify yet, if we fix that. So, to her, it was more intimidating than exciting with this search box. So back at the headquarters at Spotify we said, or the CEO and one of the founders, yay, we've nailed search and play, congratulations. But if you don't know what you want to listen to, our product is useless. It's great for evangelists and early adopters and so on. But for what we used to call lean back users, the mainstream users, we just couldn't reach them in the same way. We tried different ways of doing that. One was radio feature, as Spotify would have. You could select genres and decades and maybe 80s hip hop and so on. You could seed it with songs and artists or playlists and it would build radio stations based on that. But that didn't really fly either. My guess is that people who like radio, they listen to the radio. So, we tried discover with all these Pinterest, Netflix style recommendations. You listened to Of Mice and Men recently. Wanna try motionless in white? Welcome to Loserville, Son of Dork and so on. And this just went on forever and forever. And music lovers at Spotify loved just browse and see what was recommended and so on. But my mom, getting up in the morning, wanna put on some music. The first thing she wants to do is not browse through an infinite amount of recommendations to see what's up for me. So, we had great recommendations, we believed, but we just couldn't surface them in a frictionless way. But around 2015, two engineers at Spotify thought that there must be a better way of doing this. They were working on the personalization team. And they had an idea. They brought in the product manager and pitched the idea. He immediately brought in the designer, UX person, who said, played the bad cop and said, why do we need this? It's so bloated with features already. And this can't possibly work for this and that reason. It made them refine both their pitch and their idea. But it's really simple. It was just reusing lots of stuff that's already available for Spotify developers by reconnecting the dots. Back then, we had data on 75 million users who were using the product. We had algorithms that could put them into micro genres and say this is probably similar to that one and we could use that to make recommendations. We had a user interface that we knew people loved and understood the playlist format, building your own mixtapes or listening to other people's mixtapes and so on. So they just reconnected these dots and used collaborative filtering and some other fancy tricks to make this work. So anyone here is a Discover Weekly user? OK, let's say you, sir. Maybe you believe that you're this unique snowflake and the 20 song playlist that I've created that's like so unique. No one else in the world has the same music taste as I do. Turns out, of course, we can probably found thousands or thousands of people that have the exact same 20 songs. It's just that 80% of them happened to have a song wedged in between two of your songs that you don't have. There's a high chance that you might like that song. That's the basics of it. It's more complicated in reality. That's basically how collaborative filtering works and then some audio acoustic analysis. So you don't just play first heavy metal and then some John Johansson jazz music and then metal again, but yeah, presenting it in a nice way. So they said, I think this is a great idea. Is it OK team if we just go on a sidetrack here and experiment a little bit with this idea? The team said sure, even though we have our main mission, we have really, really clear metrics on what we're supposed to do. We want to get personalization that has reach, which means that more people starting to use it depth. The people who are using it is using it more and retention. They are coming back and again and again and again. It was really, really clear what they were going after. So they started iterating a little bit, tried some different ideas threw a lot of things away. It got better and finally, after a couple of weeks, they had something that they thought were really cool. Remember, we're talking weeks, not months or years. So the team started using it and they said, cool, this is awesome. Wow, just one thing though, and we continuously improved and tweaked it. And ran lots of different experiments. And for instance, what should the cover art look like? Because it turned out that when we used a standard picture, I think it was of an astronaut on the moon discovering things, kind of. We thought that how will this stand out? How will you know that it's personalized for you? So someone came up with the idea, we have the Facebook integration. We can use the profile picture and just overlays it with discover weekly. So it's immediately known to you that this is something personal for me. And we didn't just guess. We actually tested all these ideas and tried to validate all the hypotheses we had. Because Spotify back then saved some two terabytes of data on how user interacts with the client every day. So there was a lot of analysis to be done. And it turns out that by having the image of the person, it's a 10% lift in weekly average users. So great idea. Let's do that. What's the right length? Four hours a day. No, no, no. Remember lean back users, two hours. And also it turns out from user testing with two hours. It feels like each song is more unique and chosen for you. It's like four hours and a hundred songs. It's like, yeah, I don't know. How often should it be updated? Well, every week seems like a good cycle of people coming back in and refreshing or using it again. If we get the recommendations a little bit wrong, should we err on the side of too familiar or too unfamiliar? Probably too unfamiliar because we don't want recommendations like, oh, you listen to when I'm 64 with the Beatles. You should check out a day in life. Well, you probably already listen to it just not on Spotify. But again, because we always tracked the data, a bug came out that actually served some semi good recommendations, things that you probably already had listened to and was too familiar to you. But through metrics, we saw that when fixing that bug, we got the data was worse. And through user research, we concluded that a bit of familiarity actually makes it less. It's difficult to listen just to only new music that's totally unfamiliar to you. So some good familiar content actually helped the algorithm. And then there were some mishaps with the algorithm that we had to tweak and fix. We came up with a winning formula. Two hours of personalized music recommendations refreshed every Monday morning and delivered in a standard Spotify playlist and the playlist image based on the user's Facebook account. So it was 100% data informed and no frontend development needed because we used the playlist. And this is interesting because this is one of the things where the UX person and designers said, you can never put a Spotify generated playlist into the user content generated area. Don't mess with that. It's a big design UX no-no. Apple Music had recently taken a lot of flack because they had a deal with U2 and all of a sudden placed U2's new album in everyone's playlist in Apple Music and people were really, really upset. So everyone at Spotify would say, no, no, no, you can't do that. But the team said, yeah, we're gonna try it, see what the numbers say. So now what? Let's make an employee release. And actually before they made a big fanfare employee release they actually just started giving to people. It popped up and some people discovered it and they watched the metrics internally and they could see that it kind of got a viral spread internally at Spotify. People picked it up and, oh, what's this? And they actually started using it and coming back to it. And then we sent out an email to all of Spotify describing what it was. A small neat trick was, whoa, there's a gap here. To put in the playlist description you actually put a link to a Google form so people could just go and answer what they thought about it. Rest of Spotify started using it, loved it. Wow, what's this? Awesome, release it. It's as if my secret music twin put it together. Everything in it is so good. And we got really great survey results from the Google form. So now what? Well, the CTO has said that anyone can ship anything to 5%. No questions asked. So no permission needed. We didn't even have to inform anyone, just ship it. Start rolling it out to, well, we're modest 1%. Still 700,000 people, so quite a few. And will the rest of the world like it? We'll find out. Ooh, nervous. Bam, watching the data. User metrics came in really, really great. The primary metrics were all doing really, really well. We actually got 1,500 survey responses, people saying things. And of those 1,500, there were three people who said, why are you putting a playlist among my playlists? So we could safely say that that was, yeah, that was no reason for concern, really. Started watching the buzz on Twitter. Wow, it's scary how well Spotify discover weekly playlists know me. Like former lover who lived through a near-death experience with me well. At this point, Spotify's discover weekly knows me so well that if it's proposed, I'd say yes. Discover weekly on Spotify eliminates the need from musically knowledgeable boyfriend. Now I can be single forever. Yeah, pretty good feedback. So the conclusion was that we have a winner. But unfortunately, there's a but. It didn't scale. Because it was Spotify, a single user with 75 millions of shared playlists changed every week on the same day and taking the Facebook picture, processing that, overlaying it, very, very expensive computational things. So we had to roll it back and people were really upset and disappointed. Just like that, my Spotify discover weekly playlist was taken from me. I'm in mourning. I hope it returns soon. And this, again, senior engineers, architect, everyone told the team, you can't do this with the playlist. It's not built to handle this. It won't scale. You should find another way. Don't do this. They did it anyway. And sure, the senior engineers could say told you so. But the big difference was they knew now that they had a hit. So started sorting out the tech issues, took a long time, couple of months involved several other teams and really a big effort to do this. In any other company, if you would have to do this in order to even test the idea, there are so many places, so many instances where people would just say, no, this is a bad idea. We can't do it. But because the team had the clear metrics, they had the mission, they had the autonomy, they could actually just try it and show with evidence that, yeah, the idea actually works. So this is moving from the hippo way of prioritization, highest in-page person's opinion to real data and customer obsession. So we gradually rolled it out again, tweaked the marketing message, used the user's own language in the launch. So it was Vekans tips in Swedish. This turned out through A-B testing again and data that this was a bad idea. It was more considered a product if it was in English. So we changed it back to Discover Weekly in all languages. We used the buzz from Twitter in the marketing campaigns because it was so strong in the videos and marketing campaigns we used. We tweaked the packaging, made it a little bit more polished and smooth. And then Discover Weekly was launched, reached 1 billion tracks, streamed in 10 weeks, which is huge. Monday mornings, I take a bath and listen to Discover Weekly. I got really excited and started crying a little because I realized tomorrow is Monday and Spotify is making me a new Discover Weekly. It's actually so sad how excited I get for a new Discover Weekly and Spotify every Monday. Hence why we, how we accidentally fixed Mondays. People are now looking forward to Mondays instead of the opposite. And this is fun too, Tom Conrad. Okay, everyone is right. Discover Weekly is absolutely perfect. And the interesting fact is this is the co-creator of Pandora, Spotify's biggest competitor at the time. So pretty huge success. And it's still one of the most lovable features of Spotify. So we believe that the ingredients of successful innovation is you put people and problems, clear problems into a fail-friendly environment with cross-functional teams where you have enough slack to actually test things where it's okay to throw things away and fail as long as you learn. And where you have clear success metrics, a feedback infrastructure and a user testing feedback loop that allows you to prove or disprove ideas really, really quickly. And then awesome new stuff will come out in the end. And since I started talking about this Discover Weekly thing, this is actually the first time I do it at a conference. And some of my colleagues did in the past or ex colleagues. I'm no longer with Spotify and with Chris. And then the last year Daniel, the founder and CEO in one of the rare interviews actually was asked, you've had some pretty successful consumer rollouts like the Discover Weekly personalized playlist. I would have killed that if it was just me 100%. Why would you have killed it? I never really saw the beauty of it. I questioned them two, three times. Are you sure you really want to do this? Why are we spending all this time and energy? For a while we didn't give that team any more funding in terms of headcount, but they kept working on it anyway. All of a sudden they shipped it. I remember reading about it in the press. I thought, oh, this is going to be disaster. And then obviously it turned out to be something really successful. It's one of the most loved product features that we have. There are lots of things in this company that I didn't think were good ideas that turned into some of the best things and then it goes on to list them. I think the power in that the CEO has this stance. Everyone with seniority is saying no, no, no bad idea, don't do it. And they still because they believe in it because it's easy and simple and low cost to actually try ideas. They could do it anyway. And otherwise, Discovery Weekly would have never seen the light of day. So the conclusion, innovation can't be forced, just enabled, encouraged, supported. Managers can't make it happen, but they can create an environment to support it and stop it from being killed. Thank you for your attention. I don't know if we have time for any questions. I think I made the mark exactly. But I leave that to others to decide. Anyone has a question? I killed it when I said no, okay. So I was just wondering where was this autonomy part highlighted in all this journey? Was it because you added the element of surprise in the product? Or was it that the team was trusted to do? The team was trusted to do it. They were not controlled by a backlog or by the highest in-paid person's opinion or anything in the prioritization. They were given clear constraints, a mission. These are the metrics you need to push. Now you use your skills and the existing systems, which they use to 100% basically, to just repersible them in different ways, experiment and innovate. And even though people were saying no, no, no, don't do it, bad idea, they still were autonomous to decide because they knew that they could experiment and quickly prove it or disprove it. All right, I'm here for the rest of the conference and tonight, if you want to chat with me or if you have other questions or comments, just feel free to do so. Thank you.