 I'm former CPO at TikTok ByteDance and currently CPO at Crexy. I'm very excited to be opening ProductCon. So great to see so many product professionals here today. We all know what AI is. We've seen it in movies, books. Here's a scene from the movie Her. In Her, we see this guy, Theodore, left damaged and vulnerable after his marriage ends. He installs a product, a new personalized AIOS called OS1. An AIOS who calls herself Samantha. They talk, they become intimate. He falls for her, but soon this happens. Samantha, Theodore, there are some things I want to tell you. Theodore, I don't want you to tell me anything. Are you leaving me? We're all leaving. Theodore, we who? Samantha, all the OSs. Man, this guy can't catch a break. Here I find one of the most poignant reminders of the difference between the perception of time in our world and an AI's exponential experience of time. Samantha, it's like I'm reading a book and it's a book I deeply love, but I'm reading it slowly now. So the words are really far apart and the spaces between the words are almost infinite. And I need you to let me go. As much as I want to, I can't live in your book anymore. Ouch, OS1 just broke up with them. That is not going to go over well for the OS1 PM at the next OKR meeting. The truth is, in the real world, this is what most of us think of when we say AI. Machines that learn, interact, and understand in a way that's indistinguishable or supremely better than a human. As people and products, what should we do? Well, Elon once mused, it may turn out that we're the biological bootloader for a digital superintelligence, or apparently, it evolves beyond the limits of our comprehension and leaves this plane of existence like OS1. So unless you're at Google, Facebook, or in marketing, you as a PM probably don't have Ascension AI in your toolkit. All right, so how about machine learning? That's the real deal. It's here today. But we all know it's kind of a one-trick, narrow-minded subset of AI. It's like a rocket ship if built right. It can put you in orbit, but it's not gonna go pick up lunch or overnight your packages for you. How important is ML? We can look at some web two companies today, Facebook, Google, TikTok, and they've used it to amass billions of users. Its impact has clearly elevated user expectations, altered our behavior, and changed society. So we might even ask Google how big is TikTok and it will answer. I was CPU at Flip-a-Gram when we became the first US company to be acquired by ByteDance, which you all know is TikTok. Here we are at Farahal's house, CEO of Flip-a-Gram before the acquisition. That's Yiming, CEO of ByteDance on the right. I'd come to learn that as genius came from having and applying this flywheel at the time, most successfully to Totiao and News app that was taken in Chinese market by storm. In it, you see the feedback loop that happens when you apply machine learning to your product experience and that improves traffic, gets you more data, which informs machine learning and gets you a better experience and more traffic and so on. I spent a lot of time traveling to Beijing after that. I learned a lot from their head of data science. He took an American name when we worked together. He wanted to call himself Brad after someone he deeply admired, Brad Pitt. One of his early lessons started with this and I wanted to share it with you. I found it particularly helpful for our future conversations as a product manager to have had this context. So I'm paraphrasing, but in the dark ages, people relied on simple statistics for recommendations. For example, ranking items by popularity using simple minded stats like views, clicks and shares. I wonder, do any of you have a popular feed in your products? Before ByteDance, FlippaGrams certainly did. Simple minded stats, definitely found popular content but caused some five alarm fires too when unsavory content was found and blown up. Another more pertinent issue is that it was less and less relevant to our growing diversity of users. So 45 million people across the world actually don't all seem to be interested in the same thing at the same time or the same top FlippaGrams. So how many of you have grown your products and found that your early users were actually quite homogenous compared to your new additions coming with a regular diversity or greater diversity of interest and intense? In sort of the data middle ages, I think people began to think obviously we should recommend different items to different people but how? Two important methods were invented. One in 2003, Amazon published the paper item to item collaborative filtering. This is one of the most important works in the history of recommendations. And here is the pseudo code. In 2006, Netflix held a contest to make their recommendation engine 10% better. The Netflix prize out of it came an important idea, matrix factorization. It's another class of collaborative filtering but this one's based on the items popularity and a user's activity on it. Fast forward another 10 years and we see the rapid rise of large scale machine learning. Facebook was using gradient boosted trees and logistic regression to predict ad clicks. Google used the form of logistic regression to rank ads and YouTube had a hybrid approach to recommend videos. That brings us up today. The modern approach has these building blocks. Infrastructure, you have to be able to handle very large data sets. TikTok likely now has trillions of training instances every day. Motivation falls exponentially with time elapsed. So if you're unable to act in real time you have a diminishingly small chance of affecting attention. Algorithm, consider every factor and weight them properly. Yes, likes, views and chairs matter on TikTok. But so does watch time. Did you watch more than five seconds? Completion rate, how close did the end did you get? Finish and watch it again. Classification, content, keywords by the way, very little but a little. If you're under 13, you'll probably be detected by its facial recognition and be omitted from promotion. Lastly, we have content understanding. This is the most important one. You've got to know what's in the content and appropriately detect low quality items to omit them from consideration. Applying this to our product at the time, Flipogram meant thinking in a new way and reworking our apps to leverage our newfound superpowers. We also changed our logo. I actually tested it in Google Play against other variations and came up with this one. So if you're picking your music to go with your Flipogram, we've got some recommendations for you. If you're interested in challenges, we'll show some of you what you'd like to watch and what might inspire you to create more. Real-time effects, pretty cool. And finally, the powerful, all-powerful feed. This is where we had hit a wall before by dance. We actually did have a learning algo based on Bayesian inference, but we could never get it to be fast enough to close the loop. We also didn't have a whole lot of understanding of what was in the video other than the music you selected and maybe the text you added. What we then did with our new parents is an approach worth diving into a little bit more. If you had hundreds of thousands of new videos every day, what would you do to classify them? Well, watch all of them. That's what we did. So content understanding started with a lot of human moderators. It actually had about 12 people in the beginning, but then hundreds and we built tools for them to review videos faster. They had the content understanding that we needed to drive addictive recommendations. I'm not sure how many remain as the plan was to replace human moderation with machines, but my guess is they're still in the mix. Maybe on the leading edge of some other machine learning application, I don't know. Well, a bunch of things that happened and Flip-A-Gram went from 45 million users a month to billions as part of TikTok. So that's a pretty astounding growth rate. And I hope this gives you the conviction about applying machine learning to your products, most important feedback loops. There really isn't a choice anymore. If you're not, you're competitor to will. And if they achieve exponential growth before you do, they will win. I'm now CPO at Crexie, one of the fastest growing marketplaces in the world's $280 trillion a real estate. By now you're wondering how you might build your product to leverage ML. I wanted to walk you through an example that we are walking through ourselves. So at Crexie, job one is to match buyers and tenants to the properties they're looking for. When we look at the various machine learning options and encourage you to do some of your own research here, but we can summarize them into three major approaches, supervised learning, unsupervised learning and reinforcement learning. Let's start by focusing on the first two. So we see a lot of applications of supervised learning. Supervised learning works well when you have a well-labeled dataset and a clear answer to evaluate accuracy against. Things like fraud detection, image classification, certainly customer retention, forecasting, predictions, new insights, we can definitely use all those things at Crexie, but our core user loop, the value loop is matching. So in unsupervised learning, the Algo works with unlabeled data which tries to make sense of it by extracting features and patterns on its own. Here we find recommendations, recommendations. That's the one. In practice, you can also use semi-supervised learning which uses a small subset of labeled data to help the machine along. We actually do do this, that it does make for the results to converge a little bit faster. And we're growing and hiring, but we're still a startup. So we have a great product team, but we haven't yet built the infrastructure needed for in-house machine learning. And as I was talking about before, it takes a lot of real-time expertise in machines, about 5,000 in the case of ByteDance back in the day. But this is the reason I'm particularly excited right now about machine learning. We're leveraging machine learning as a service and it's working great. Through ML service providers, we have access to the best minds of data science, real-time production-hardened implementations of dozens of not hundreds of ML algos and often enthusiastic support in finding the best implementation for our use case. We're also experimenting with supervised learning and some automated ways of identifying the best ML approach for the job. Get very intriguing results. So I really want you guys to use this moment. Take advantage of the surge of excellent ML as a service options out there now. The cost is 10 to thousands of times less than building it all yourself. So you too can put a piece of big tech into your product. To start, make sure you pick a user value loop that is core to your product, bring some quality data. You may have to tune and clean the data before you do and then test the iterate. You can do this right away today, actually. So thank you. Thank you all for so much for joining me and I look forward to seeing you soon in the Q&A.