 Hello everyone and welcome to another session of Product Management in Age of AI. My name is Kushbu Goyal. I'm the Head of Product Marketing at Crucio Cloud. Prior to that, I was an ex-product lead at Meta for internal products. Less about my job, fun fact. While I focus on my day job from 8 to 6, during the rest of the time, I'm either an aerobatic pilot, you'll find me up in the sky, or you'll find me under the water doing scuba diving. I just came back from an expedition from California to Alaska where I flew for more than 37 hours. With that said, I'm super excited to introduce our panel and I'll start with Prashant. He is a lead for our WhatsApp business platform for SMB. He's passionate in helping the small-medium businesses to determine to succeed by democratizing their enterprise capabilities for small businesses. Fun fact, his mom and his sister both are SMBs in India and have built their businesses on Instagram and WhatsApp. He has a personal interest in here, guys. With that, I'll go on Omkar, who also comes with a very interesting fun fact. But before that, let's talk about his professional experience. He is a PM with experience in working in different industries such as e-commerce, banking, and startup. He is recently working as a product manager at Shopify, where he leads the marketplace product platform, connecting merchandise with a social e-commerce destination such as Facebook, Google, and TikTok. So anytime you see any advertisement, you need to know whom to blame, it's Omkar. But fun fact about him, Omkar can polyglot that can speak five languages and is actively learning to speak more. He can be a good travel buddy, guys. Siddharth is a lead product manager at Meta, building avatars. So before joining Meta, he founded StoreZop, a B2B e-commerce platform, worked at Amazon and few other startups. Great experience Siddharth brings to the table. Fun fact, Siddharth once completed 1,000 kilometers. Wow, cycling trek in just one week. And he stayed in camping tents and lived off the gear grid during this thrilling adventure. Imagine no cell phone connection for one week. I don't know how will I survive that. Even at Alaska, I had connection all the time. And last but not the least, Shantanu, who is a director, PM at Microsoft, with one and a half decades of experience in building products and running business functions for cloud and e-commerce space. Fun fact, despite his youthful spirit, he is still in old school, also getting lost in magical world of books and the charms of vintage comedy show is his big thing. And I think we should all spend a little bit more time in going back to our traditional way of doing things. With a great introduction, I would like to kick off for what all you guys are here on understanding what does a product management means in the age of AI. So Prashant, I'll actually kick off with you. So let's get started with some housekeeping questions here. AI and product manager world. Can you define AI for a product manager's lens and maybe quickly walk us through what are the different categories of AI and which are the most relevant to product management today? Of course, thank you. Thanks to me to everyone. Thanks, Kashmuru. So AI is a broad term. So the most simplistic definition of AI is tools and technology that makes computer systems smarter and may make our human intelligence. We are product managers. We want to look at AI from the lens of applications. So let's take that lens. So if you think of lens of applications, there are three broad categories, which I see for AI. One is machine learning. Machine learning is the most relevant categories for product managers. It's already well-established. It's a subset of AI where you use technologies and machines to adapt for data patterns and give you recommendations. Some examples here, you're using statistical methods. Some examples are your YouTube recommendations, your Instagram feeds, and your search. Some applications are also in finding patterns such as fraud detection in a finance organization. That's the number one. Number two is deep learning. Deep learning is basically mimicking what your brain does. It's inspired by the functioning of your brain. And it really is a part of ML, but it requires more data to train and requires much more compute power. What is a common application for product managers? Number one is generative AI. We all know chat GBD. That's based on a deep learning model. There are others in gaming. Imagine you're in a game and you have a non-player, a non-playing player. You call NPCs and creating those out of a deep learning model. And then there are applications in healthcare where you could actually detect a lot of patterns and make huge advances in the field of genetic studies. That's your second field. Third is robotics process automation. So think about your simple tasks such as filing taxes or finance tasks which could be automated. And in this field, a typical example is UI path is a great company automation. Anywhere is a great company. Here you take a set of paths and then you automate them. So while their AI is a broad field, but if you look at for product manager lens, this application-based lens helps us to think through different categories and different applications that has enough for a product manager specifically. Thank you Prashan. That is such an amazing breakdown of AI. Maybe Siddharth, considering there are so many different spaces and it's such a wide space, what skills and knowledge should a product manager possess in this age of AI where there's so many different applications from generated AI to robotics to training and inferences and how can a product manager up skill to stay relevant in this field? So again, as Prashan said, it's a wider thing and AI is so broad that you are, you cannot know everything about it. But to begin with, I think you should start with having the awareness of the market and capability which are available. So every second day a new AI tool is dropping in the market. And while it is not necessary for you to have knowledge of everything, but broadly you should know in what direction market is going. There are many changes in the text space like chargeability and barred and the hugging face and whatnot, that is tech. And then your images. So just know in what direction market is heading so that you can apply some of those things to your own product. Not everything would be relevant, just try to figure out what is relevant. Another thing is some of them can actually help you to increase your own efficiency. Like I would say don't do menial jobs, like there are things in your PM video to just do stuff. Like if you're not leveraging AI to do that, you are sort of leaving stuff on the table, so just use that. And when it comes to actually working with AI, like I don't believe that you have to be expert in AI because we are the PMs, we are not the engineers for the most part. Some of them are and it is great if you can code in Python and run your own scripts, but it is not needed per se. What is needed is to have a broader understanding that you can work with your partners, your data scientists, or engineers, or XFN partner, so that you can sell the idea of AI if needed. And when it goes to a really detailed one, you can just tell the real experts, like really ML engineers and data scientists to take those deeper questions if needed. And apart from that, it is just regular PMing. But I don't want to say, hey, because it is AI, something's going to change in PMing, yes, things will change, but your data-drivenness is not going anywhere, you still have to figure out your impact and focus on impact and business acumen, all those things are still very relevant. And then there are new things like privacy concerns. Privacy always have been a big thing in PMing, but lately, because for the most part, AI is a black box. It's very hard to explain why it is doing what it is doing. And I would say, rather than upskilling, you should try to figure out what is your expectation and where you can lean on your partner. For example, I don't expect any PM to be a legal expert. It's better to just go to your legal partners and seek their help. So I think as far as you have this ability to figure out things on the go, because things are changing every single day. What was relevant yesterday may not be relevant after a week these days. I think that's the broader thing is and still very evolving field. So just stay in the touch of what is happening in the market and you'll be fine. I think that is the TLDR. Thank you, Siddharth. That was actually very useful. So just to summarize, being aware of what's happening in the market is number one thing, always be on top of it, whether it's AI or any other space, and then use basic of your PM roles, skill set. And at the same time, rely on your partners heavily. You're not doing every job. It's not a job of PM to do everything. Rely on your system and partners. It takes a village to get a product out. With that said, maybe now that we have established what AI means to a PM, Prashant, can you walk us through some of very unique problems? Siddharth already mentioned that privacy is one of the unique problems you always have to think of the AI space. Maybe a few more when you're managing AI-enabled products and how do you typically approach them? Yeah, sure. First of all, I want to build on what Siddharth said. Most of the problems are really not unique. We see that in our, these are just problems which have changed their structure a bit. They have just a lens. The way of looking at the problem has changed. But one few unique one which comes to mind is one, Siddharth already touched upon, it is solving everything with AI. So like that can become like a fallacy where every problem which comes into mind, you go to solutions first and becomes very tempting. AI is very powerful. Machine learning algorithms is very powerful. But in most of the cases, at least in my experience we have seen, you start with heuristic. You start with things small, start small without solving everything with AI. So what can a PM do? Always look at the problem. What is the problem we are trying to solve? Who is going to benefit? And then will AI knowing those capabilities which AI brings to table and what application of AI are you going to use, or is it actually adding incremental or step function value? So without getting caught up into AI will solve everything. I think that's one is unique because it's very tempting. Second is again, which amplifies, AI amplifies uncertainty. It's a probabilistic technology and it's a black box, I said that pointed out. So it is even more than set in. So you have to manage expectations. The road maps could be longer. You may not have like as quick as impact on metrics as you would do with a non AI product. In a lot of cases, you might have to deploy a lot of capital investment. So how do you mitigate that? AI is first managing your own expectations. We all want quick results. So first looking into yourself and say, hey, look, this is going to be a longer run. So I need to be much more patient with my own expectations that where you start first. B is managing timeframes and expectations with your partners. Explain them like being, making sure communication is on point and explaining why this is taking longer or why this investment is multi half and not just single half. So managing your timeframes and expectations is important. And C, and I would love to have the panel contribute to this one, is the C from my angle is how always have a plan B. From my own personal example, there are definitely places where we thought the ML will help us, but it did not, the results were not what we needed. So having a plan B that what are we going to do if this doesn't work out always helps, especially when you're dealing with ML and AI technologies. Thank you Prashant, that was really insightful. Shantanu, maybe you want to add something more and I think both Siddharth and Prashant touched on it. They are like you are engaging with new stakeholder here, which is data scientists, AI focused scientists, data and research scientists. So maybe what is a new engagement style needed for PMs to work with them? Maybe you can highlight a little bit more on that. That will be great. Sure. Well, I think Prashant and Siddharth outlaid some of the very good challenges here with changes engagement model. But when I take a step back, I actually see a paradigm shift in some of our PM approaches. A traditional PM approaches move fast, break things and build. But with the AI and working with data scientists, research scientists, technical architects here, I feel it's more about experiment fast. So you don't break things later. And I think that is a fundamental shift as we go into more AI intensive products or applications. I want to touch on one important aspect that Prashant mentioned, I think it deserves a spotlight on it, which is the ambiguity and the opacus of such projects. Let me think about it. Traditionally, we know that, hey, this is not the project about, this is the value proposition. This is what we are going to get out of the project and these are the schedule timelines. Now, let's take that and put that under the lens of an AI product. And then things start shifting dramatically, right? For example, talk about the fact that it's a non-deterministic outcomes here, right? It's not a one-to-one equation anymore. Even data scientists, these are scientists working on the model, cannot tell you what the output is about. Try explaining that to senior leadership that you're not sure what the output could look like, right? Or talk about the opacus, right? I mean, you don't know which model is being used because various models are being experimented. You're not sure about the training data because when you slice and dice data, you have to, you know, not cherry-bick, but filter the data into something which is reasonable for your product, right? Let's talk about trade-off decisions, right? Something which a product managers need to excel in. And now what do you trade? Do you trade experiment versus speed of execution? Do you trade, you know, moving fast or trying to get more training data and more features built in over there? Because if you think about it, coding is actually the smallest part of the puzzle, the smallest lead time in an AI data. It's a training and model selection and data cleanup which takes a lot more time, right? So when I'm working now with a team of data scientists and others, I actually feel like a warrior trying to, you know, way through and pushing off all of these challenges and boy, do I yearn for the hills at some of these days. But I think it's a very evolving landscape and as it keeps evolving, I think the PM frameworks may need to be enlarged, you know, to understand the AI umbrella as Prashant put it forward. That is so insightful. I can totally see that in a PM world, it changing from that traditional to AI where there's so much of uncertainty, very well said, all three of you, like you know, like Shant, Siddharth, Prashant and Shantanu. With that said, I actually want to make switch gears a little bit moving away from challenges. I want you to get a take from Omkar. I'm sure as Shopify, you see it all the time. But I want you to understand from you, where do you see are the most opportunities for disruption with this advent of generative AI which we have been talking about. Maybe you can share some thoughts you can make from experience on Shopify. How does it AI impact on who comes and advertising? Yeah, for sure. So first of all, I think like all of the panelists have like, you know, are so erudite and then put things together and so beautifully. So like I'm glad in terms of like actual applications of AI, I'd like to like take that question in two parts. One is like before the advent of shagging, generative AI specifically and before that AI in general. Because AI again, I think it's a whole universe and right now the spotlight is so much on chat GPT that there is, I don't think we've even like taken full advantage of pre chat GPT AI applications. Like when it comes to e-commerce throughout the value chain, right from supply chain optimization to inventory management to content generation to personalized recommendations to like, you know, reaching out to like customers that have left your item in the cart, et cetera. There's so many wide variety of applications. I think even those people haven't still mastered a lot of the companies are still nascent in that sense. And with the advent of generative AI, especially at Shopify, for example, the way we've applied that is I'm sure like most of you have seen it is through this product called Sidekick where it's like a GPT type technology that looks internally at our own data and essentially performs as a Sidekick to a merchant who's running the business. In previously, we had to build all of these like growth tactics that we had to like get in, figure out how do we get in front of the merchant to tell them, here's what you should be doing next in order to grow your business. And we had to like, it was almost like a growth PM's job to think about how do we get the merchant to like see this, use this, et cetera, et cetera. Whereas now it's almost as if the merchant is asking the questions themselves. And on whatever question they ask, we can look at the treasure trove data that we have and tell the merchant what they should be investing their time in. As you know, these merchants are always time strapped and they wanna know what is the best use of their time in order to get maximized returns. And so that's where I see the biggest application happening, at least from a Shopify's perspective. Advertising is, again, I think advertising has been one of those domains that's already pretty mature. And like we've, from a programmatic perspective, I think we've done quite a bit over the last many, many years. But again, the challenges within the advertising world as you all know is around privacy, around ATT, around cookies. And so with all of those challenges, now how do we build better identity graphs for people? And that's where I think, again, AI can really, really help and improve the return on ad spend as well. So that's on the sort of the faggin of advertising. And then there's also a lot of applications in terms of just building the creatives itself. Like I honestly feel like old school ad agencies are ripe for disruption right now. And people can very, very quickly come up with high quality creative content and can completely run an end to end flow of like running an advertisement all by themselves or with little help from technology or tools that are available with little help from agencies. So I think there's a lot of real world use cases that generate AI and AI holistically has I think impacting in an e-commerce and advertising in a very, very positive way. That is amazing Omkar. And I think you almost ordered a part of our question which came from our audience of tools apart from chat, GPT and Bard and other LLM models which we should be aware of. So that's great. With that said, like Shantanu, I actually wanted to get your take also on it. Like you come from a little bit different background. Would you agree with what Omkar said? And also maybe like expand on it. Tell us, what could be the biggest disruption by AI on a company's capability building? Thank you, Khushu. And Omkar shared really well on advertising and on the Shopify e-commerce space, some of the things which can be done. And again, we have to understand that with so many new things going on when a company's like Google and Amazon and basically all tech companies can get caught off guard which suddenly generate a way I coming into picture, right? These things keep evolving. But if I had to look at a trend or a resurgence of a trend which failed because the tech, the analytics was not keeping up the business need then I would be excited about the digital twin concept, right? It's a concept which came a few years back, failed, came again, failed and largely because, you know... So just for the audience, digital twin essentially a virtual representation of a physical object or a system or a process, right? Essentially allowing you to simulate and model those behaviors, there's interactions with the real world. And I'm not talking about metaverse here, metaverse is very different just a heads up, right? So this is more about, well, a good example would be, let's say a dynamic supply chain, right? Imagine the COVID era, if a company could actually model exactly what's going on from e-commerce point of view, the warehouse delivery, the sort centers and the last mile or from a cloud point of view, the racks, the amount of demand coming in and how the servers are interacting with the larger picture load balancer. So if digital twin now finally been generative LLM this seems to be a chance that we can build a better digital twin product, right? With lower investment upfront and higher investments, higher returns coming back. For example, when I tried to implement a digital twin three years back, it was a very tough sell because it needs a big investment from a tech point of view. And there were enough returns in that case that we could build it. But now if I look at the same project with generative AI sense, right? The fact that now the system will be able to give us a robust logical framework on how to simulate, how to, you know, a module or these different concepts, how to get analytics and extra insights across it, whether it's advertising, customer excellence, people training, supply chains, the example which I gave. I think that is where I'm really, really excited because I think re-invents of all these concepts which are out there but did not happen is going to make it a very exciting world for us PMs, right? So we better, you know, get a growth mindset on and start focusing on understanding all these concepts. I hope that answers your questions, Khashog. It definitely does, Shantanu. And with that said, I actually want to take some questions from the audience. They are finding it very useful to learn from you all. So one question which actually got my interest, like, and I also want to know it, can you give some examples or like maybe someone else from the panel also can give some example and how do you leverage AI to reduce the cost for PM functions? Is there any tools which you guys use? Like maybe I'll charge you, but you write your blog post. Something like that, fun. I can take it. I can take the first tab and I'm sure. This is a fun question, right? I mean, at least at my company, we are exploring multiple ways. One of the things which we all know, writing is a big part of the PM job. Writing is in communication, especially writing. How do you communicate and distal your ideas? So I can give a personal example where I find it useful. I almost brainstorm and it reduces my time while the critical thinking still comes from, I don't expect chargeability or the meta is a llama to do critical thinking for me, but it gives me framework. Framework on which I can then apply my knowledge and build it and mix. So that takes away 30, 40% of my time. I'll give another example. I am right now building in like any role you do community and build a community. I'm part of building the community at meta and I need to come up with things we can do, such as these tech talks and I would just reach out to lemma, give like a bunch of ideas and then I can be thinking of itself. So that's one example, which I think is really taken off that low level work, which I could expedite using these generative AI models, but I would love to hear from other people. Yeah, then I just want to add to what Prashant was saying. I see the question that's from Vinit. Vinit and I actually used to work together a long time ago, so hi Vinit. So anyway, I think the way I at least use it, and I'm probably paraphrasing what Prashant is saying here, is I kind of like, when I have to like think about any sort of proposal or a brief that I'm thinking of, obviously the most obvious application that everybody is talking about is like, get AI to grow your draft and then you can like, fine tune it and so on. But the way I think about it is usually a decision tree that you are thinking through and the model is your own, right? Like you're thinking about various aspects of a particular strategy and you can almost chart it out as a decision tree. And then while you're evaluating each mode of that decision tree, I think that's where I think chat GPD can be really useful where I'm quickly able to like, analyze a couple of segments, think about like, what's happening in a particular decision point and so on and so forth. So that way I'm able to really accelerate my own sort of validate a lot of my own assumptions or things like that while I'm working through a strategy. So yeah, I'm sure other people are using it in other ways. Maybe Sidharth, I think you have something. I just do add one more thing into it. Like most of those larger companies have a policy against using this open source models. So when you do use them, you have to be very careful on what information you're allowed to put there because assume that whatever you put there is there for eternity sort of, it's just there for anyone to use. Just imagine any of the global companies, some product manager put their strategy into the thing just for something, for rephrasing and now their strategy is public. So while it is very important, just be very, very careful about what you put there. And again, it also depends on the size of the startup or is it a big company, a smaller company? But again, that's something to be very careful about. All right, I know we are at time, but I would like to ask one more question from my friend, Aadesh. And I think it's a very important question. What's the role humans and AI tools can play, design, implement, like laws, regulation around these technologies? And as a follower question, he has his technology industry doing everything possible to keep checks and regulates on these things. I know for a fact, like Biden administration is looking on like, you know, regulating on AI right now, there's a discussion going on, top companies which you are part of has actually agreed voluntarily on it. So any take on it? Well, I'll attempt to answer here. And I guess a simple answer which I can give is that, I think these things are still being worked out. And let's face it, even things like social media regulations have not been closed out correctly. We uncover so many different things all the time. Talking about AI, something which even these tech companies are struggling to figure out, right? And then you're expecting any government anywhere to figure it out and come up with regulations. I think that's a tough ask. However, if I look at the questions about, how can we be a part of this regulation framework? Right? I think that's where it's important to understand that our own experiences and our own findings shared with the regulation teams in our companies will help a lot. Like for example, the company which I belong to believes very strongly in ethics and having an ethical framework around all the products we sell. So if there is something in my product which I find out or something which I am an operating principle which I govern, I have all, my company has given me all the power just reach out to the ethical team and talk to them about it and see how that can be implemented and they can see whether that has to be rolled out across this, right? But at the end of the day, sharing the cautionary note from Siddharth, proprietary information on open source models plus having a very clear understanding from a viewer point of view that's what's created by AI and what's created by humans. I think these two things should be upfront in any regulation framework. I agree with you, Shantanu, a hundred person. And I think people need to think about it. When you're thinking about AI, it's about training and using that information and creating a logic on top of it. There's no, majority of the time there's no manipulation of the data. And there's always a human who's making the end decision. They use AI to give them some inferences, some logics, some useful data points. But in the end, the human is the one who's actually controlling how to use that information. With that- One thing I'd like to just add, I think this is such an interesting topic just to add what Shantanu was saying. What I'm hopeful of, when social media came, it was a wild, wild west. We did not know what we were jumping into and the regulations came way later. It was too late and Europe was the leading the GDPR. And now, as Shantanu said, we're still figuring it out. So one thing I'm hopeful for is we have seen those movies. We have been there. We have seen what lack of regulations does. So I think everybody should do more and we are a little bit more cognizant and we all should collectively be more cognizant of how to use this technology responsibly. And I do have a hope that this time, if not everything, but more better efforts are being done to make sure that we build it in a way which is actually truly add values versus take away value from society. Definitely. I think, yeah, we all are more on the AI lens. Everyone is being more thoughtful considering how everything has been evolving in the world. With that said, it's a very good note to actually end our discussion on today. I would like to do a special thank you to Shantanu, Omkar, Prashant, and Siddharth for taking out a valuable, their valuable time out of their day and help these aspiring and existing product managers on how to be part of this AI trend. And AI is not just disrupting the way we get out our information, but how even as we are building those things. So thank you all for joining us and thank you everyone from the audience for our amazing questions. Feel free to connect with any of the panelists and anyone on the product manager group. So we will be happy to answer your questions offline as well. Cool. And thank you, Kuchu, for moderating the panel. Thank you so much. Thank you.