 Hi, I'm Raphael Courage, VP of Product at Get-Around. And today I want to share the product approach we used for launching our first machine learning component. But first, a few words about Get-Around. So transportation is the number one cause of greenhouse gas emissions, which lead at, as you know, to global warming, and personal cars account for more than half of that. Then you have air pollution, causing 4.2 million premature deaths each year in the world. And personal cars also take a preposterous amount of space in our cities, in a city like Paris. It's the equivalent of the seven most central boroughs, completely taken away, fully covered with cars that are only used 10% of the time. Well, car sharing solved this, and this is why Get-Around's mission is to empower people to car share everywhere. Now how do we do this? By providing our customers with access to safe, convenient, and affordable cars everywhere around them. So what we are is the number one connected car sharing marketplace. Connect in because you just book a car around you, unlock it with your phone, and off you go. And marketplace, because just like Airbnb, we do not own the supply. And so we have a model that scales much faster thanks to our community of owners. So back to our topic, machine learning. And I'm going to talk about in-house, tailored algorithms, and not the kind of chat GPT-like AI integration, which don't get me wrong, it's just as interesting. So do you want to tune in and come for the later panel discussion for that? A bit of context. The application that we are going to talk about is smart pricing. It's a way for our owners to get their prices optimized automatically for them. It helps them get more earnings and save time because they don't have to go and edit their prices manually again and again. Now let me ask a question first. How many of you are data science literate and will be able to follow the technicality of this presentation without a trouble? Raise your hands. Wow. Technical crowd today. So probably 40 to 50%. That's great. But here's the thing. It actually does not matter so much. It does not matter so much because if you want your launch to be a success, it is not a technical challenge. It's a design challenge. It's a product marketing challenge. And yes, okay. There is a technical component to it. But even there, we'll see that it's not all about tech. And it's very simple to understand. Say you do your algorithms and you optimize prices with your first iterations. Maybe you're going to get a 10% improvement, 15% if you're really good. Well, this means nothing. Don't get me wrong. It's very meaningful. But it means nothing if nobody uses it. So what you want to focus on is making sure that your users opt in for your feature. You want to focus on adoption because most likely the spread of impact that you can have by focusing on adoption is going to be five times higher than what you can do with your first iteration of the model. And that should show as well in the way you organize your teams. For instance, the marketplace dynamic squad that we have and that built the smart pricing is one of our most technical squads. And yet we absolutely want it to be staffed with a product designer. Speaking of product design, it's a design challenge. So product designer get around groups UX writing, UX design, and user research. And we used all of them in that feature, starting with user research. What we did is that we made some Figma mockups of the flow that we wanted and how it integrates in our current product. We tested that on our existing user base of owners. And for those who haven't done qualitative testing in the past, you'll be surprised because you know a very, very small amount of interviews to get some very meaningful and valid insights. Usually between five and 10 is the magic number. So what did it learn? Well, it's all about trust. Let's look at the quotes. I hope the algorithm is doing its job. I would have tried it just to see. It's reassuring because I can go back. Your users won't trust that the machine can do just as well as they can do. That it can understand the specificities of their use cases and that it can think smartly. And this is even more true if your application has anything to do with money. So in the end, your users, they will fear to lose control. So since you will have to reassure your users, what I want you to do is to write down the exact words that they will be using during the interviews. These are the words that they use and understand. Make sure you speak their language. And so we reviewed our flow and our copy, and here's what it looked like. So first, we propose and engage, present the benefit of the feature, what it's going to be about. And then you want to give insight to your user on how the technical algorithm is going to think. You want to make them understand that, yes, it's a machine, but it thinks like you do with very, very practical features and signals basically. Then as I said, our users feared that they would lose control. So we introduced a few features. First the fact that there was a floor price that owners could set and that the algorithm would never go below. Second thing was that at any given point in time, the owners could override the smart pricing prices with their own manual prices for a given day. And last, of course, they can opt out at any time. But even then, you know, it could be still a little daunting for our users to take the leap of faith. So help them and show them a preview of what the prices are going to look like for them. So by then, you did most of the job. So just take the opportunity to make it like. Super smart and looking like something important is happening. Which is. All right. Now we have a nicely converting flow. But it does no good if nobody knows about it, right? So it's a product marketing challenge. And this one is going to be a shorter one, but I just wanted it to have its own dedicated section to make sure you don't forget about it. So product marketing. You want to do two things. You want to raise awareness and you want to sell. To raise awareness, just think of the basics. Product announcement, in-product announcements, banners, emailing, push notification, blog post, and talking to your community. And then for each of those channels, you want to sell. And you want to sell by striking the right balance between reassuring users again and making sure they want it. So the very marketing part. And then again, please do use the same language as they use. Ah, here's the technical part for the 40% of the crowd. Well even there, not quite. Because even on the technical side of things, you want to make sure that you put as much emphasis on the people and process around the technical part. Introducing the four O's. Half of them automated and half of them manual. Organization, Oracle, Observation, and Optimization. Starting with organization. For any machine learning project that you start, you want to start by organizing and cleaning your data. This will radically improve the precision of your model and your ability to iterate. Then we introduced a very precise routine for our team, which helped us set the pace for all our iterations. Oracle. So smart pricing, what do you want to do? You want to be able to forecast and to predict prices at any given place and date. How do you set prices in a free market? You just look at the supply you have and the demand you have and the balance between both. In our case, demand is going to be people searching for cars and supply the cars that are available on the platform. Starting with searches. You want to look at actual searches, real people. There are a lot of butts out there, scrapping your site, and you want to make sure that you filtered them out. So there are several ways to do this, but a good way to start with is just to consider that any headless request to your site and your search is probably a butt. And then we also looked at bookings. These are a much more precise intent, basically, than searches because there was an actual payment for it. And also it gives us a very good view of how many cars are already utilized, already booked, so how many cars are left, so the supply side of the equation. And for this, we used two ways to predict this. First for anything that is short enough in the future, it's just looking at our data, our actual booking at the moment. And then anything further down the road, we used a Facebook prophet, which is a forecasting model provided by Facebook. And with that, we actually got some fairly precise results on our oracle, looking at the predictions versus the ground truth here. And the third observation. So far, we've been looking at supply and demand balance over our platform. But the true market is much broader, right? Let's consider we have tons of cars on get around. I guess your oracle will tell you to push the prices down. But what happened if there was a shortage in the rest of the competition, which is actually what happened after the COVID crisis? Then a lot of customers are going to come to your site in the end. So actually, you want to push the prices up. So you want to make sure that you look at competition as well, and not only your internal data. You want to look at trends as well. And this time, it's internal trends. I'm going to give you another example. Let's say there is a specific event, one year in one city. And it's not a recurring event. If it's not recurring, it's not going to be able to be picked up by your oracle. If it's not happened in the past few years, there's no way that it can just infer that it's going to happen this year. But by looking at your data and how the trends are looking like compared to the previous years, then you will be able to have some more insight on whether you should boost the prices up or if you should take them down. And so we introduced actually a manual component, some manual tweaks that our pricing managers can add on top of the machine learning model. So it's a very hybrid model that we are using. And last, optimization. Any machine learning project will ask you one thing. What do you want to optimize for? Do you just want as many bookings as possible? Well, there's a simple version to this. And you probably don't need machine learning, you just put the prices super low. But in our case, we want to optimize for a total revenue over time. So our optimization function is to maximize the booking probability times price. Now, the other thing you want to optimize for is that so far, we've looked with the Oracle at masses. Masses that can be maybe located to a city, maybe a district. But the prices that we set, we set them for each single car. And every car is different. So you want to bake the characteristic of the car in the model as well. Is the car new? Does it have a lot of booking history? Does it have great ratings? You want to make sure that you observe that as well. And last, optimization of your time. While at first, I'm a very big believer in doing things manually because you get so much insights at looking at everything with your hands and eyes. Because it's a process of continuous iteration, as I showed you, with our routine, we do it over and over and over again each week. You want to make sure that over time, you optimize your time and you spend less time on the manual things and more time on optimizing your model and algorithms. So as a recap for your first machine learning component, you want to invest in qualitative user research and product marketing. A good model that nobody uses is worth nothing. Invest in UX writing and use your user's language to make your smart feature sounds familiar to them. Invest in a healthy routine. You will refine your model faster. And to do all this, make sure you start with a narrow scope. Just to illustrate, if you try to have your algorithm be a good fit for all of your users, you're going to have a lot of trouble making it right. It's a lot easier to have a precise algorithm if you take a narrow scope. And also, it frees you time to do all the rest of the integration that I talked about. If you focus too much on having all your users at the same time, you will have bad precision, and you will not work on feature adoption. Thank you very much. Hope you liked it, and thank you so much. Amazing. Great job, Raphael. First bump, love that, love that.