 Hello everyone and welcome to today's webinar which is hosted by the product school. My name is Alina. I'm a machine learning product manager in booking.com and I'm going to talk to you about the ML products and how to build them in the ever-evolving generative AI era. So a bit about me. My name is Alina and I'm currently a bit over one year in booking.com as a machine learning product manager overlooking different kind of content ML models. We're building models around NLP, computer vision and serving them across the business according to user needs and business impact. Previously to that I was also a product manager in eBay and also in RapNet overlooking similar products in the ML area around computer vision and also some search and platform capabilities in RapNet. I have a BA in visual communication. Yeah so let's look at today's agenda. So I'm going to kind of walk you through the generative AI landscape. A bit about my experience as a PM in my current company but also the previous ones and yeah with all the hype, all the trends around AI, generative AI, how can you as a PM adapt to it and I want to give you my tips around it and a few things about kind of what you can do with it and what's next in this area. So with that let's start with a simple definition. So generative AI is generally speaking a type of artificial intelligence and AI technology that can produce various types of content as an output and we can look at some examples right now. So we have outputs possible outputs like an image. Why not you know asking AI to have their own interpretation into a very famous painting such as the Mona Lisa, how it can look like beyond the you know the face that we all familiar with. I'm sure you've seen a lot of AI art around the internet in the past couple of months but it's truly interesting and fascinating. Text is an obvious output again. I'm sure you've seen many many reddits around the possible outputs of HRGPT or generative AI. So could be many a text output about let's say a poem, a story, a recommendation, conversation and yeah answering basically any question that you might ask it. Here's an example of a poem about a cat and a mouse and there's much more. So it's really evolving all the time and they're releasing new versions all the time and there's possible outputs around code. So generating pieces of code, optimizing your code, completing your code, reviewing your code, music, videos, yeah we've seen a lot of those and also around the web so it's truly truly an interesting era as you can see. So yeah with that when did I kind of notice that it's really becoming a wide public spread topic and everyone is talking about so I got a message from my mom asking me if I heard about this like latest thing chat GPT and that she's been using it. She's been asking it questions, recommendations, recipes, travel recommendations or just some basic questions and you know she played around with it. So I think that was truly when I recognized that it's becoming a really massive thing you know beyond hearing it from my colleagues and from the industry but yeah the point where my mom tried to use it and she wasn't scared of it and she knew how to use it. I think that was kind of the turning point for me understanding that it's really everywhere and it's here to stay. I think the really kind of important shift that happened is really chat GPT which I'll touch upon later but it became really accessible to everyone and everyone like it's free to use everyone can use it you can ask a question the interface is quite simple and I guess that's how it got to my mom as well. It's quite helpful it gives you meaningful answers to your problems or just like any just FAQs that you had and it's a very human like interaction so yeah like you get it can be funny, sarcastic, scary but overall it's something that is very familiar to us and I think it makes it easier for us to interact with the chat GPT. Yeah so with that I think we've all noticed and I'm going to cover the bits that there's has been an era or a stage between some of the tech giants and that I think we can call as the race for AI at this point and just going to play you a short video before it I deep into that AI AI AI generative AI generative AI generative AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI yeah so that was Sundial Beach Eye Google's CEO and one of the recent recent conferences they had kind of introducing they're the product they're AI AI and Genetic AI product. AI AI AI AI. Yeah, so let me just give you some maybe overview of who is in the race of AI or who are the main players in it and just to give you an idea where we stand. So obviously we have open AI and they are the creators of chat GPT and Dali too. And and they have been heavily invested by Microsoft. So there's been like, I think around 30 million billion investment by Microsoft in that company. Their founders are Elon Musk. And I think this one, this is the most famous one. And they've been recognized as the fastest growing consumer up in history. So they reached to 100 million users within a month, a month and a half or so. So truly, this is kind of the first interface that got out to the market. And they just developing since new versions of their models, and, you know, adding new capabilities. Next, we have Google or Alpha Beta where they have the bar AI model that they also recently released. Basically, it's a conversational chat bot as well, similar to to chat GPT. It's also powered by a large language model LLM Lambda, which is basically part of this like foundational models umbrella that you will hear about. Basically, they're trained on very massive data sets. And they their goal is to interpret a natural language and understanding and processing. And this is the basis for for all of these companies and their AI race. We also have a few kind of smaller players, you can say, but we have Meta with their Lama model, which is also similar to a kind of a chat bot. So they have question understanding and answering sorry natural language understanding. It's an open source model. And Amazon released their bedrock kind of service where you could build your generative AI applications and using their platform ML platform and AI platform services. So I would say that from my interpretation and the interest interpretation kind of open AI Microsoft versus Google, those are the two major ones. And we should keep an eye on those out of everyone that's out there. So now that we kind of I think realize that it's really in the tech giants company priorities, it's becoming their kind of full time investment. We in resources investment wise and also, you know, they really identify that there's a big business impact that they can do with those models. And it's inevitable that, you know, your role or your your your roles at PM or your company, at some point, probably looked into that a new information trends, you probably been even affected by maybe changing your roadmap or a company priorities. So I really want to touch base on that. And what is from my perspective, how do I see the PM role in this AI new landscape, generative AI landscape? So I would say that the things I would list here are probably generic also when there is no AI special time, but I would give specific points on why I think it's it is unique to, you know, the AI era. So as a PM, you always need to kind of stay up to date to some technologies, to your competition and product developments. But here, it's really even more important, like because the trends are shifting so fast, there's new releases every few weeks, days, month, and, you know, more and more companies are joining the race. And I would say you really have to stay up to the technology and understand what's out there, what's being released, who are the main competing competitors and leaders, understand some basic kind of terms, as in generative AI, large language model, foundational models, prompt engineering, those are some of the basic terms you will have to familiar yourself with. That would basically allow you to evaluate and assess the options and the technologies that you have in front of you. So you would have to, you know, select the best technology that you would like to use, according to your use case, you will need to evaluate your data, your current data source, determine the quality of your data, how those models perform, and, and yeah, basically going to evaluation process, which, by the way, also includes costs. So there's a lot of cost calculations that you have to do with those, with those new technology. So make sure you're on top of that as well. And just to touch upon the last point, which is the ethical implications. So not every company has its own kind of an ethical AI, let's say, a group that oversees all the, all the models and make sure there's no biases and, you know, really building the right thing. So really make sure that, you know, there's no issues in that area. As we know, the models have some, have some drawbacks, like hallucinations, where they can make up things, and, and yeah, sharing correct information. So really, and keep an eye on that as well, your evaluation process. Yeah. And the last two points is you'll really try to identify your potential use cases. Is it, you know, okay, there's a trend with generative AI, but can you, does it apply to your use cases? Can you, is it relevant for you? Is it relevant for your users? Are there user problems and scale that really needs to use, you know, this technology? So I think it's also part of the assessment that you need to do. And of course, product up and test quickly. So once you identify the use case, you evaluated the technology, try to kind of measure it and assess it with real users, it could be internal employees, some beta users. But yeah, try to gather feedback as quickly as possible and test your hypothesis with your use case. So yeah, like I said, it's kind of generic tips that you can use. But I do want to also dive deeper into some practical tips that you can do with this new technology. So again, this is like based on my experience, but you can adjust it to your kind of your work, the way you work and your company process. So I would advise you to either create or join an AI task force. So if there's nothing at the moment, maybe you can come up with one, it could be even in the level of, you know, sending out like weekly updates or having a weekly kind of team meeting between all the relevant colleagues that are interested in this and leverage some, you know, a company communication channels, it could really, I think, add to transparency between all the, all the different teams that are trying to assess or work on the same thing as it's really new kind of technology, I would advise you to try and create a special kind of task force or join one. And as I mentioned, the new technology, especially if you're a PM, a machine learning PM, I think it's really inevitable that you would look again at your roadmap and assuming that you've built a yearly roadmap or even if it's like a quarterly or half year roadmap, I would really advise you to look at it, revisit it, review your existing capabilities, planning capabilities, your objectives and try to understand how you can, you know, leverage the new technologies, the new gen AI technologies, come up with workshops with all your stakeholders and see whether you need to change your roadmap based on that. The last two points are about, yeah, taking the small step approach. So, yeah, if you want to test out the actual new technology, the model, you can start with really a small data sample. You don't have to acquire any new technologies. You can use an open source LLM and try to evaluate it. You know, ChagYPT is really accessible. It's a front end application. You can really try it out by yourself, even, but yeah, make sure that you know, follow the company guidelines around that and everything, but you really can test it out even without any dependency in other teams. Just to try out the output, see how it works, whether you can really leverage it to your use case. And, yeah, focus on the high impact use cases. I would emphasize that again, try and find maybe a generic problem, a generic area in your company where really you can leverage the gen AI capabilities. Try to solve for real user problem and answer user needs. So, yeah, it can be in any area like customer service, a content creation, marketing department really depends on, you know, what's your area and your product. I do want to cover maybe a few like specific use cases from my perspective and from, you know, things that have been following and maybe just leave you with some ideas on what could be possible opportunities in use cases when you're working with gen AI. So, I think from the customer perspective, I already mentioned that it could really help make your any customer interaction you have across your product or platform more human like, which really adds value and interest from the customer perspective. Think about like your customer support areas, any incoming messages from your customers or between your customers. It could really kind of ramp up that experience and make it faster and more accurate can help you prioritize some of the incoming requests or tickets from your customers. So think that area is really something that can be useful. And yeah, of course, like if you're dealing with content, you can really create a personalized experience for your users. You can ramp up your product recommendations. If it's e-commerce or you know, anything else. I truly think that this is the area to invest in as well if you're a product in that area. From the product and ML or like data scientists and development engineering teams perspective. I think that like as a product, again, it really helps you to test your ideas. You can simply use charge APT and try out your idea and you know, maybe create an MVP based on the outputs and even get ideas from charge APT, you know, or any other kind of interface for your product roadmap or ideas. As a developer, I think that yeah, it would truly also help with some creating like generating some code templates. You could create test cases or improve your the quality of your code and it could help you like translate your code or autocomplete it and so on. And for the ML part or data scientists, you could really use GPT or other LAN outputs and to fine tune maybe like internal models or existing capabilities that you already have. Yeah, so those are a few ideas like scattered ideas around the areas where I think you know, maybe it could give you some direction where to start. So maybe if you're a PM in marketing department, think about how you can create some marketing ads using GenRT AI for customers and for users could be anything around Q&A, FAQs, creation, generation. Product descriptions, you can improve them, you can personalize them. So it really falls also under content personalization, product recommendations. So yeah, if you're looking to build a commendation engine or based on a search input, you could really leverage that experience with GenAI. And of course, the customer support area, which I mentioned, where you can really also enhance that experience and make it more human like, fast and more accurate for your users. So to recap, I'll leave you with a few points. So as you might notice that GenAI and the race AI is here to stay, I would advise you to join a relevant forum, create one, create a team around it, really to stay up to date and have everyone updated on recent developments. I really believe strongly that it's a good opportunity to revisit your roadmap, revisit your priorities and it's a new opportunity to identify high impact use cases. And you, I think you should keep on taking the small step approach regardless of AI, but I think you can really test out your ideas and using even chat GPT, as I mentioned, and try to go for a small MVP. And based on that. And as you've seen, the diversity of the GenAI capabilities is quite vast. So really identify the one opportunity that matches your users, your objectives and priorities and work around that. So I think that's all. Yeah, and I'm leaving you with a few resources. Some YouTube channels and news, PM newsletters I think are interesting and entertaining. And a few podcasts which I think you can mostly find on Spotify. So thank you everyone for joining and thank you for product school, for hosting me. And if you want to contact me, ask any questions, give any feedback about this talk, please do reach out on LinkedIn. And yeah, thank you very much everyone. Goodbye.