 Hello. Hello. Hello and welcome to this talk to talk about the importance of learning from errors. My name is Rujeka Stays. I'm from Barcelona. I have 20 years of experience in IT. I started my career as a admin, later known as DevOps in these companies. Later I switched to product management role and I worked in these two companies, King, where I am currently and also I worked at Global, which is a quick commerce company. So what we're going to talk about today is about something like that, right? About stocks. So I hope that you all know these memes that it talks about the ideas to talk today about this feeling that we all have and it's kind of normal to say that everything is going well, right? So in this report, this is like a whatever report. It could be a yearly report. It could be a quarterly report. It could be your own objectives. The idea is that we have this tendency to say that everything is working fine. And the reason is because no one wants to share something like that, right? If you say that everything is going wrong, that the impact is not good. There are going to be more problems because people in general don't like to talk that much about problems. And if you present something like that, you'd like to really give explanations about what's going on. And maybe the reality is something like that, right? You've done some progress. There was not expected impact, but it's important that you explain the learning about the issues, right? So this is the agenda that we have. We're going to talk a bit about errors in general and what they are and, yeah, a little bit of context on why it's important to detect the errors and all of that. Later, I'm going to share some techniques about how to detect errors. And I'm going to share some learnings that I had. And I consider them as an example about that it's okay to talk about these errors. So first of all, what's an error? I looked at the Wikipedia, of course, and the error is an action which is inaccurate or incorrect, right? And here I put a real fun cartoon that when metrics go up, it's like, yeah, everything, congratulations, everybody's doing great and everything is good. But when metrics go down, it's like, oh, yeah, this was a result of many external factors. And the thing is, it's hard sometimes even to know why some metrics are going up, but it's even maybe sometimes it's harder to know why some metrics are going down, right? And doing this analysis is something that we should always try to do. Where can you find errors? The reality is that errors are everywhere. Everybody, everybody has done errors, everybody does errors. So it's pretty, pretty normal. Some examples, I mean, there are hundreds of products and maybe released every month or every week, every year. There are a lot of products that are released and they don't get expected impact, right? Or the expected share of the market. And the same goes with features. You are all the time trying to add features that could have an impact in metrics, but the reality is that the expected impact, it's not what it is. And it happens because there was an error somehow, right? Maybe a wrong assumption or something like that. And why it is so hard actually to recognize errors? First, fear and ego. Fear to maybe not get promoted, maybe to get fired. You also have the ego that you feel that you are like the, I don't know, a senior product manager, organizing errors is hard. Imagine if you are a director. So it's really difficult to embrace and to talk about errors. And also because the companies, it is hard for a company, right? To encourage this culture of sharing what are the errors that we all have and it's quite normal. And also because you are being more vulnerable with the others if you share that. Imagine that you are in a meeting with other product managers, temporary managers, and you share your errors. This is not really common and we should all encourage to talk about errors and know that this is a phase that we all need to have in order to learn. So what comes in an error? There are a lot of things that could cause an error. Wrong assumptions. You thought that the users wanted to have something, but the reality is that this was not what they actually needed. Wrong expectations. You expected that one feature would have like a 10% increase in whatever metric, but it didn't happen. Lack of alignment if you need to align between different teams and you do something that the other team is doing. So that's also an error, right? The ground processes, lack of knowledge, of course. If you don't know how to use certain technique to prioritize, it also can have an impact and cause an error. Miscommunication. It's another classic. Not understanding a user. Not understanding what the UX is proposing. So there are a lot of things that could cause an error. And why it is so important to minimize errors for a product manager. So the way I see it, or the way I see the role as a product manager, is that it is really, really important that we try to minimize the number of errors that we do. And we need to work on that. We talk with the users, we talk with the business, we talk with UX, we talk with the engineering team. And we need to be able to detect where there is a possible error in the process, or who is maybe not doing the right thing and all of that. And the impact is that if you're a product manager of a team of six engineers or with seven, if we call the UX and all of that, the product manager is telling the team what to do. So to some degree, you are convincing them to go into one direction, right? And well, these numbers are totally random, right? But you have six engineers, imagine that the salaries and the cost for the company of these six engineers is like $300,000. And imagine that you have a feature that you need one month to release. This is the cost of this feature. And the same goes for one year. So if you have your own company, of course, this is much, much clearer for you, the impact that it has to release wrong features. But also it's really important to take that into consideration when, at least I take that really, really serious when I'm a product manager to try to minimize the impact that it could have releasing something that it's wrong, right? So because at the end, this is a little joke, but at the end what you're doing actually is burning money, right? If you do things that don't get value, you're doing that. And to some degree, it is fine if you do errors. I mean, it's not a problem, but you need to try to learn from these errors and then minimize these errors in the future so you will become a better product manager and also the company will have better results. So it's important to take into consideration that when looking for errors. So this is like a super basic flow on what you do when you're doing software development, right? Here we're assuming that there are no errors in the strategy and the objectives of the company or of the team. And in this time when we see that we start with the opportunity with the problem, trying to understand the user needs and the pain points that will help you achieve your objectives and then you start talking about, okay, possible solutions that could help you. If you can, you can always try to validate this with users if you have the opportunity and then you start the development, the release and then you finally validate the solution. In here you have a lot of loops in the sense that maybe you have an opportunity, you validate it and it's wrong, then you maybe go back to the solutions and you validate the opportunity if possible. So this is actually an iteration that it's a loop that it happens quite often. And the important thing for me is that the sooner you detect the possible error, the better. So I think that's my personal opinion. I think that we need to pay a lot of attention to the problem and the solution and the validation before start coding. So the cost, as I was mentioning before of telling a team to do something that it's not the best thing to do, let's say, it has a huge cost. So I think that most of the errors should be detected the sooner the better so that the impact is less important for the company and for the team. So what is then a learning on... When you are learning, of course you can learn from two different phases, let's say, right? Before a decision is made and after a decision is made. The learnings that happen before a decision, when you are a product manager, you need to take a decision. These learnings that happen before is what you learn from a workshop, what you learn from books, what you learn from learning from others, like here where we're all doing. So this is the learnings that you happen before you take a decision. But the learnings that are also super important and that will make you grow as a product manager are the learnings that happen after you take a decision. So you took a decision for whatever reason, you had your rationale, and then you need to evaluate, okay, this decision that I took, what is the impact that it had? Should I maybe took another different decision? So this is the learning that you will have and this, as you can see here on the arrow below, later you will use when you take, you need to take a decision in the future. So this is where you're learning and it will be used for you later when you need to take another decision. So another important thing, as I was saying, mentioning here before about the learnings that you have before taking a decision that you could learn from maybe from a book, is that there is a difference between theory and practice. Theory is important and is key, of course, to learning, but practice and learning from the decisions you made in the past, it's really really important. I think, this is my personal opinion, reading books is great, but it's not the only way you should learn. So you should really, really, really practice and be exposed as much challenges as possible. So my personal opinion is like, it is better if you are in different product teams and working with different people and not get used to be always in the same team because the amount of experience that you will have is better, right? So if you are in a company, it's better if you can try to switch and be in different teams because then with that you will be exposed to new challenges and you will be exposed to new decisions and you will become actually a better product manager. So more practice and try to learn from the decisions that you made in the past. So now let's go to how to detect errors. I'm going to explain some basic techniques and things that will help you to detect errors after you took a decision. So this is the list of things, experiments. You can ask questions to metrics or to the users or to yourself to learn about errors. You can have also formal meetings and formal documents, review your processes and your ways of working and also ask for feedback. This is, of course, there are more things that you could do but let's go with them. These ones are like the classic, right? So the first thing, experiments. Let's go with the basic one, A-B testing. This is a technique that helps to protect the business. As I'm saying here, not everything should be A-B tested. I remember there was a situation where things were so weird that it seemed that the people wanted to A-B test if it's better to have the accepted button with green or red as an example, right? So not everything should be A-B tested and the reality is because it's expensive, right? It's not cheap to A-B test things. But again, yeah, you try to do that if you don't want to have a big impact in your key metrics especially. So the other thing that you should actually do to learn from the decisions you made and to try to learn from these errors is ask questions to metrics, right? Numbers don't lie in the sense that they are raw. If your gross booking is going up or down, you will for sure detect if that's true or not. And you need to ask a lot of questions. First of all, and most important, of course, is if you are using the right metrics. We could talk a lot in maybe a different session, but I think that it's really, really important to be sure that you are looking at the right metric. And then you can ask questions, right? If you took a decision and you deliver something, why you didn't move, why it went up, why it went down. If it went up, which teams could affect this metric? What are they doing? Is there any A-B test running? So as you can see here in this joke that I was sharing before, when something is going up, you need to ask why it's going up. And the same when it goes down, right? Why is this metric going down? What happened? And it requires a lot of effort, but it's actually key to understand if something was wrong in the business or in the ways that you worked. So this is really, really important. The other one is ask questions to users. The follow-up after the delivery is key. It's key to learn. And you can do that in different ways, right? If you are lacking, you can do interviews to users. Of course, please do. Doing interviews is not easy. I think I'll talk a little bit about that later. But trying to understand the point of view with the user is key. You have also these tools in the market that they are known as behavioral learning tools that allow you to record sessions. And they are really also good to understand and see. You can even see the video on a video on how are they using your tool without you being there. So that's an amazing opportunity also that you have with these tools. And of course, later you have service. It's not my favorite one, but also it can help you to collect a lot of feedback coming from users. So the other one is like these formal meetings, I would say, that are retrospectives and post mortems. They are great to look back. They have a specific like a structure and they are really, really useful to learn. It is not easy to run retrospectives and post mortems in a healthy way. Sometimes some people fall into the blaming mindset where post mortem and retrospectives, the main, main trigger is to learn. And again, this is related to what I was mentioning before. If the company doesn't encourage this mindset to learn, you could fall into these anti-patterns. So yeah, I think that it's a retrospective and post mortem are really good ways to learn and doing things differently on detecting why something didn't behave as expected. Another important technique to learn from errors is the PRDs or product requirement documents. These documents are formal, but they are really, really beneficial to review certain decisions that you made in the past. This document has a structure where you put what is the problem that you're trying to solve and why you have this rationale behind solving it that way, what alternatives you have considered. So it really has pushed you to think on why you're doing this thing. And I've seen some people even looking at PRDs that maybe were shared one year ago, right? So people really look into that and try to understand why certain decisions were made that way. It also is a really, really good way to align between stakeholders, so I really encourage to have something like this formal doc that will help you learn from past decisions. Another one that is important is review your processes and your ways of working. So as a product manager, you work with several actors, right? You work with UX, you work with the engineering team, you work with the stakeholders, you work with the users, with the business, so you work with a lot of people. So review always your processes. How often do you talk with users and stakeholders? When do you check the feasibility with the engineering team? In which phase? Are you just telling them what to do? Are they engaged with the good things that they are doing? And do you mold them in the aviation? Should you? I don't know, is your team mature enough? Do they know enough things about the user? Should they know more about the users? How are you sharing what the users are sharing with them? How do you evaluate the success or failure of a feature of product? So review your process. It's really important. Review your ways of working and try to detect if there is something that you could improve and it will make you available. It will make you able to do better decisions in the future with the team. So another thing, asking for feedback. Asking for feedback after you took a decision is quite hard. Normally the mentorship and one-to-ones is more about how can you maybe improve and the communication and all that, but I think it is good if you have someone as a reference that you could ask freely and openly it's like, okay, I did that decision. It was wrong. What would you have done? And having these discussions that are open are really, really good. Also something that, in my experience, it's not that common. It's sometimes weird that we talked about that with some colleagues. It's like, ask for advice to other product managers. The product manager role sometimes is a little bit lonely. We are all the time working and talking with a lot of people, it is weird that it's not that common, I would say, to talk with other product managers openly and being open and humble about how you work and what you do. And my experience is not that common and I really, really encouraged to do it because I only can say good things about that, really, really good things. So now let's go about my learnings. I'm going to share some experiences that I had in the past that... The interesting thing here is like when you read something in a book, it has certain impact, right? But the impact sometimes is not that big, right? You could maybe talk about jobs to be done, which is a technique that is really famous and you can read books about that. But later put that in practice, sometimes it's not that easy. But if you do an error and you try to remember, okay, what I write in jobs to be done, a book, then, okay, you could really, really learn more, right? So what I'm saying about that is because these learnings, they might seem some of them a little bit obvious. You know, like, well, of course, I already knew that, right? But when you learn from an error, it really gets into your mind and you remember them and then you are becoming actually a better product manager. But you need to pay attention to that, right? That's why it is important. So yeah, the first one of the learnings asks why? Of course, but to the right person. And the reason is because I remember the situation and this was actually one of my first experiences as a product manager. I was working as a product manager and another manager came and said, okay, we need to do that because leadership is requesting this, right? So I said, okay, I understood what he was saying. We were working with the team like one month and a half, something like that. And when we had actually the front end, the team didn't have front end at that time. So they learned how to do it and they delivered the first draft in the first session. When I share it with this person, the one that actually originally requested the idea, I'm sorry, the need, that person said, okay, but why don't we do this and why don't we do that? And I was like, whoa, at that time, to me it was super clear, right? I was like, first of all, why didn't I talk to that person on the first time? And then I think that we shouldn't have needed to develop a front end and everything to validate the solution. So with the PowerPoint, the same graph that they created and the same product they have created, I could have checked with the user directly with a prototype that I, with the PowerPoint actually, I could have detected what was actually the problem underneath, right? So the learnings that I had, what they were really important to me is talk directly with the requester to understand the problem. Why? What is the problem that we're trying to solve? What is the value that it will provide? Because you're going to put effort from a developer team to do that. Small valuable iterations. So don't go with the final thing and if possible, try to do it always a small and valuable iterations. And of course, what I was saying before coding prototype, it seems I know it's super obvious, but sometimes you can fall into these errors. And then the good thing that you need to ask about the request is because you are full of assumptions, right? So when this manager came to me with the request, I was making a lot of assumptions and I was talking with the original source. So this is one of the learnings that I had at the very beginning of my career and it really, really, really helped me in future and decisions. Another line of engineering. Engineers, of course, love their work and they really like to be up-to-date with industry and they really like to know the new techniques and new technologies. And here, well, this is a joke that we have here, but this is real. I mean, it happens. And this goes again similar to validate your assumptions and solutions. You need to talk with the team and it's totally fine to ask to the engineering team why they are proposing this solution and what is the problem that this is trying to solve. So having this conversation with the engineering is really valuable. The thing is like sometimes engineers think that they know some problems that the user are having and they do a lot of assumptions. So this is fine in the sense that I really like that the engineering team proposes ideas, but your job as a project manager is to guide them. So first of all, of trying always, of course, to explain the problems that users face, explaining how they work so that they could have ideas, but sometimes they have ideas that are like that. And on my experience, it has happened that you have a technical solution that is super great, but it doesn't actually fit any of the needs and you're trying to fight saying, okay, but how can you use that? And yeah, for engineering it's something that you need to also pay attention and you need to be kind of not close, close, close to the engineering team, but you need to be there and be able to challenge their decisions as well. On other learnings, it is okay to kill product and features. So I remember one time that we were ambitious with one project. We wanted to do something that it was complicated, right? And I think that we were like two or three months without expected results. And when you have something that is yours, that is your product, you get attached to some degree to the product and you want the product to succeed, but you need to be realistic and you need to be humble and it is fine if you kill a product because otherwise, I mean, it is difficult, right? Because sometimes you think that you can make it, but you need to be realistic and humble and maybe take hard decisions. Also, it will help you to explain the rationale behind the decision, right? Like, okay, we have been working on that for two months with this amount of money and I think that we should stop it now. Otherwise, we will waste maybe twice much. And also, this is really important. And related to features, it is the same as sometimes you release something and you leave it there and just maybe expecting the users to use it, but the reality is that they are not using it. So less is more. And you need to be realistic and humble. And sometimes you could get, again, attached not just the product, but also to features that you think that they are great, but maybe they are not that great as you thought. So that's also learning that I had that it's really, really important. Another learning, of course, I mentioned metrics. I really like metrics. It's not the only indicator about what you need to do, but it helps to have a lot of discussions. In that case, I remember a situation where in a team we were having a solution that had a good percentage, I would say, of errors. So instead of having, of course, 100% of success, we were having like a 20% of errors on the process that we were running. And we wanted to decrease this 20% of errors, of course. And we were talking with the team, what is happening and all of that. And I remember one of the engineers saying, okay, I know what is happening. This is what is happening. And I was like, well, how can you know that, right? So then I encouraged them, okay, guys, let's use some metrics here. Let's try to collect what are the possible problems in here. They invested, I would say maybe, I don't remember, but actually, I don't know. One two weeks, one sprint to collect the metrics. And when we looked at the metrics, there was another one. So my takeaway is like, it is always easier to talk about metrics and not about assumptions. Assumptions is one of the, I would say, one of the biggest learnings that I have, and I tried to fight them with all my heart is that, right? These are the assumptions. And to me, it was really interesting because when we collected the metrics and we saw the graphs and we saw the other member, but I think that 80% of the cases where this process was failing was one error. And one backend when the backend saw that is like, whoa, I think that I can fix that in one week. And he fixed it and it will decrease this percentage of errors. But imagine the cost of not using these metrics and maybe solving something that it was not actually the problem, right? So it is, again, a waste of development time. Another learning, oh, I love that one. The art of interviewing or discovery. Wow, this is really, really important. And that's why I said at the very beginning also, when we were talking about the process of software development, how important and the key that this is. I've been, sometimes, you could have a UX ideally. If you have a UX, it's much easier, of course. But if you don't have a UX, sometimes as a product manager, you need to do this discovery and you need to talk with users and you need to interview them and try to get the best out of it. So, this one actually happened to me maybe twice or three times and it's already frustrating. When you are working with users, you understand their problem, you present the prototype, they give you the thumbs up, you then deliver the feature functionality and they don't use it. And it's frustrating, but wait, I've been talking to you, I've been sharing with you the prototype and you said, yeah, thumbs up. I released it and then you didn't use it. So, this proves that it's really hard, this discovery slash interview phase. It's really, really hard. So, as a product manager, I think that it's important. You don't need to be an expert interviewing. It's not that. I think that's why we have UX because UX, I consider UX as a crucial role in a team. But, yeah, I think that it's important that you learn to interview. You learn what questions should be asked. You learn what are the resources that should be used when you're doing an interview and when you review the way that the users work. So, the more you can get from these interviews, the better, but it's really important to learn how to do the interviews. So, yeah, UX is crucial. You need to spend more time with the problem than with the solution. Once you have detected the problem, possible metrics that you should tackle, then it's much, much easier to work on the solutions that spend a lot of time always trying to detect the right problem. Another learning, yeah, get out of the river from time to time. So, yeah, this is an analogy that I make, that is when you're new to a team or to a company, of course, it's different if it's new in a company or a team, you ask a lot of questions. You question everything, right? Like, okay, why you're working like this and why this decision was made, right? So, you ask a lot of questions and that's great, yeah? I mean, it's great. I think that it's really, really beneficial. Maybe you learned techniques in your previous team and you're trying to maybe try it in this team or in a different company. They were working differently and okay, let's try this here. And this is great, but I've noticed on myself and of course also I've seen that in others. It's like, at some point you get used to work in that way and then use top kind of questioning things that should be questioned again. So, my learning is try to take a step back from your day-to-day work, right? You work eight hours and you're continuously doing things. Continuously doing things. And use the stop from time to time and question yourself, right? So, I'm saying like every quarter to try to rejoin your team. Yeah, that's more or less what I'm trying to say. Like it's okay, we're using the spread metrics. Why are we doing like this? So, again is what I said before also, it's healthy to move between things because it's a way for you to rejoin a team and also learn from this process. And another aspect kind of related is that never criticize the decisions made before you go to a team or company because you need to know why these decisions were made on like this, right? And it's really, really, really bad to do that. And actually a lot of the things that I'm saying about this learning is actually from a book that I totally recommend that it's from Melissa Perry called Escape the Build Trap. She talks a bit about that and how when you are in a company or in a team you tend to at the end go with the flow and you end up doing things that maybe if you are a rejoiner or a new team member in the team you wouldn't do. So, yeah, what are the conclusions about the talk is write your learnings down. This is really important. First of all, for you as a product manager to become a product manager, try to do that quarterly if you can or something like that. Try to review them yearly. Look back at the errors that you made in the past. It's great to look at them. Sometimes when you look at them after two years some of them look super basic especially if you are starting, but it's fine because this is your learning curve and it's perfect and maybe even you are writing a book. Who knows, right? So, if you write down your learnings maybe that's going to be also beneficial for other people and also for the company. If you are learning things about the business, about the users, any learning that the company can get from you this is really great. Sometimes I've seen in companies like, okay, but we tried this in the past. What are the learnings? And it is hard, I know, to keep track of things especially in companies and especially if they are big but it's important to keep track of that. The other one is, of course, fail fast. It's fine. The reason is if you take only 10 decisions per month or you take 100, of course, it's better if you fail much more. Also, please remember not releasing something that is wrong but try to fail at the very beginning of the phase. This is really, really important. And the last thing that is really difficult is promote the culture to talk about errors. I'm putting here, but not errors, of course, is learnings. Promote the culture to talk about learnings as I said, everybody does mistakes. We do all errors. And the important thing is to learn from these errors and to become a better product manager and to have a better company that is beneficial and that is working well. This is really good for you and also for the company. So that's it. I hope that you find it useful. Please reach out in LinkedIn if you have any question or a little bit more about any technique or anything, glad to help. I hope that it was good and have a good day. Bye-bye.