 Okay, my name is Yoel, I'm from Indonesia, as you can see from my accent. I just landed like two hours ago from like a business event in Seattle, and I think I smoke too much weed there. So if suddenly I speak in Bahasa, then you know that that's because of the weed that is still in my brain. Last year I was here too, leading like a workshop about UX research, but today I'm going to talk a different kind of topic, which is like how to build an effective product design team. I work in a startups called Bukalapak. It's kind of like Shopee. It's one of the four unicorns that are currently available in Indonesia. We have around 2,400 employees. 1,000 from them are engineers, product team, design team, and from that 1,000, there are around 121 designers. Some of them are here actually. They all flew from Jakarta to attend this event. So, I will start with this story. Like for the last seven years, there has been so many things exciting, like there are so many things happening in the UX world, right? Maybe six years ago, seven years ago, no one knows about what is UX. No one knows what is UX designers, what is interaction designer is, but now everyone is more familiar with it. And the team is also getting bigger. I will start with this story. So, around seven years ago, I saw that there is like a job opening that is looking for UX engineer. I was excited, right? Because, oh wow, finally, there is a company that is looking for someone in UX field because at that time, when I tried to look for some jobs, especially in Southeast Asia, it's very rare that you find a company that is looking for a UX position. At that time, I was studying in the US. And that company is actually Trefaloka. As you know, it's also one of the other four unicorns that's in Indonesia. So, I reply, I apply to that role, and this is my email. To whom is my concern? Please find and test my resume and cover letters. Even though there is only an opening job position as a UX designer, actually what they put there is UX engineers. But I assume there is actually a UX designer. But a UX researcher's position is more interesting for me. I'm pretty sure that you will need this function in your startup sooner or later. So, this was like in 2012. I was like, hey, finally, an Indonesian company that's looking for a UX role. But, unfortunately, somehow, Ferry, the co-founder of that startup, the company missed my email. So, he replied a few months after that, I'm not sure how I missed your email. Earlier, we'd love to grab a coffee and chat with you. But I was like, shit. At that time, I already took another intensive role in Seattle. But I could not meet him again. I was like, shit. If he replied that email, I would be like a crazy recession now. I would be like number seven employee in travel local. I would be like just throwing money like crazy now. But just by missing this email, I was like, ah, now. So, like, few years ago after that, they send me an email again, hey, we're looking for UX manager, blah-blah-blah. No, you missed my email. No more, no more. But, what I want to say is, look at the development. Traveloka, I can say that one of the best, a company with one of the best design team in Indonesia, six years ago, could not differentiate between UX designers and UX engineer. But now, they become like one of the company with the best design team in Indonesia or maybe arguably in Southeast Asia. So, what I'm going to talk to today is like, because the number of, the size of product design team become bigger and bigger, how we actually deal with those challenges. Okay, so there is like the story, but before you listen to my shit, I just give you some background so that you know that the things that I talk to you is legit, right? So, okay, I smoke too much indika. Okay, so, the number in the middle is like how many people that I manage during that time, right? So, the first one is I studied about UX in University of Washington. The number of people that I lead is one, which is me. And then, I moved to FTO. It's also in Seattle. The number of people that I lead is one, too. We are in like a team of four people, one UX manager, and three designers. And then, I moved to Singapore because I didn't get the visa. I moved to Singapore to get the visa and I worked in another company with the number of people that I lead is two people, which is me and another remote designer in India. And then, I worked in Singapore for two years, but because you guys are so kiasu, I don't really enjoy living here. No, no, I'm kidding. I think that's because of the weirdness. Then, after two years working in Singapore, and I moved to Europe to Adidas. They had quarters in Adidas, in Germany, I mean. And the number of people that I manage is like two people and then three other vendors because, yeah, Adidas is not a tech company, so you need to work with a lot of other vendors. But working in two years in Germany was really fun. I can smoke with every day. But then, I think maybe I really want to work in a really cool tech company. So then, I applied to Uber and then I moved to Uber like two and a half years ago. And there, the number of people that I lead is like two full-timers, six people in different countries, and one vendor. But then, because of you, six months ago, that you like grab more than us, that because of you, that you like more cheap discount than better products, then I moved back to Indonesia. Thanks. But now, from two plus six plus one of people that I lead, suddenly I need to manage 50 plus people. There's like a team with around 50 designers and researchers. And after seven months, the number is double, not even more than doubles, not we have around 121 designers, researchers and UX engineers too. So, it's crazy. I never expect that I will need to lead this like a really big team. So, what I do, and then, before that, like two years ago, I came to this event in London. It's called Leading Design Conference. It's really a good conference, especially for you who is in manager role. And this is the quote that I got from one of the presenters. He said that the most valuable asset of the 20th Century Company is the equipment. But now, is the knowledge workers and their productivity. So, even though, even though like now, they are more like, like they are more machine, but actually, the most valuable asset is us, the people. And the other fact is that, if people do not see a feature at your company, they will find it somewhere else. I think I guess a third of the people in this company, in this room, are looking for a job in Lincoln, right? Or maybe a quarter. But actually, the survey say that 50% of employees are actively looking for new opportunities due to poor leadership or management. So, if you have five people in your team, half of them are actively in Lincoln at the moment. Half of them are making their resume better and better every day. Although maybe, I think the survey is quite biased. Maybe it's like to all industry. I think maybe the number for our investment maybe like a little bit much slower than that. But that's the fact. And this is like top reasons why people left their last job. The first one is poor leadership management. And actually pay or salary is actually the last one. And maybe the number, this is from, I think I will give you the link of the survey later on. Maybe again, the number will be different. Maybe in Singapore, better price number one. Because of the high renting cost here. But this is like, if you can see, the first four is something that you may not notice. It's about lack of carry ladders, lack of meaning, stop learning and poor leadership of management. So, looking at all of these notes that I got two years ago when I attended to this conference, the first week when I joined Bukalapak, this is what I did. So, I send an email and then I schedule one to one to all of those 50 people. I say that in 15 minutes I need you to present this following for topics. Who you are, your present projects, your past important projects and the brokers and aspirations. So basically doing like a UX research to own my team. To my own team. And I say that we will spend the last 10 minutes to know more about the interesting project that you share. So, I have this like one week every day talk to this maybe like seven to eight designers and researchers to just know their pin points and to know their aspiration. And from those, basically it's like I use UX research, something that I know, this is my forte. From those, then I get this laundry list. Which is like some of the problems that my team face at the moment. You can see here it's not about the design work itself. Sometime it's like something that is outside of the design work. It's like about the structure, carry pad, level calibration, performance review, mentoring, how is the working process, how is the design process and so on. So, then from those laundry list, and we talk one by one how we can kind of solve this issue. So, the things that, the four topics that I'm gonna talk today based on this laundry list. Some of the plan that I made after I kind of figured out the issue that I got from my team. And what are those, there are four of it. The first one is about the structure. The second one is about scaling your team. Second one is about carrying ladders. And the last one, which I think is very important, is how to get the seat at the table. Let's start with the first one. I love that the session before me is about active listening, so that I don't need to work quite hard to get your attention. But anytime I see someone opening your phone, I would say, oh, no phone, no phone. Active listening. Okay, the first one is about organization structure. At the beginning, we think that, oh, I just need to get the design right if you're like a team of two, right? But then, after your team become bigger, you will think, oh, I just need to get the strategy right. But when the team become bigger and bigger, you actually need to change your mindset to be, I just need to get the organization right. This is another quote I found from Twitter, it's interesting to me that we try to design software that doesn't have a single point of failure but doesn't set up our orgs with this in mind. We do a lot of UX research. We do a lot of design thinking, design sprint of making a product. But we never do that to our own organization. So it's kind of like an irony, right? And this is like the team, the organization that we work everyday that you will need to work with all these kind of people. And when the organization structure is not right, then it kind of make the team to not work effectively. So, based on the previous companies that I have worked with, I actually create kind of these models. Like four different structure based on how the companies utilize user insight. Again, it's like a very simple framework just based on the different design teams that I have been part of before. The first one is like a user agnostic. So basically you only have one managers and you have designers. And you don't really use the insight from users. You basically just design based on three things, right? Competitors based on design principles and based on your craftsmanship. Which is fine. That's why we are called designers because we have the craftsmanship of making good design. That's why we are paid six figure salary because we are good in the design craftsmanship. But as the team becoming more mature, then you start to think, maybe I think it's time for us to make users to our decision. And then you go to the next one where now the team become bigger and then you may start doing more like UX research. Usually a company start with the cheapest one, which is like usability testing or survey or interviews. And then the company become bigger so users inside you don't use it only to evaluate product, but also to inform the design. So the first step is you are agnostic about users. The second one you just use the insight to evaluate the product. And the third one now is like another step. You kind of want to inform the design. And it become more complex. Now you have more like a dedicated researchers because it's not easy to do more like a generative research than like a generative research. And the product team become usually more kind of structure. So like every product team will have their own researchers, will have their own designers will have their own data scientists and will have their own engineers. And each of these product teams will work in a gel way to make sure that the insight that you get can somehow inform the product design. And the last one, what is the last step when you can say that the design team is quite mature in terms of using the user insight. I believe that is a mature product team if they can generate innovation like a product strategy from the product design teams. So we as a designers and researchers are not just like a pixel pushers or like a usability testers but we actually help we help the management level the C level to actually decide where the company should go. Where the strategy should go where the product innovation should go. And in a big company like Uber's the role becoming more complex. You have VP of UX you have design or research directors you have a strategies managers, researchers research ops, design ops designers UX engineers and program managers. When I was at Uber, we have around 80 researchers and maybe around 150 plus of designers. With that number of researchers and designers, you really need like much more like a role like design operation, research operations program managers to really make sure that this team can work effectively. So so what I did at Bukalapak is slightly different so I cannot divide my product design into three. And each of that product design team will have their own design lead and research lead. And the other interesting thing is I created this call like a core team well it's basically are design and research operations. What is design and research operations? Who is familiar about design ops? Cool. Who has design ops in your team? Cool. At least there are some. But this role, design and this operation is actually very significant to make sure that your team can run faster than your competitors. And we also make like a functional team like only designers, researchers content strategist, UX engineers graphic designers and illustrators. And these are the core teams. We have a call Scalable Research. I'm going to talk more about that later on what is the function of Scalable Research. We have a research team that we call like innovation team design and research operations one team that only handle the design system and a team that handle like the conversation between our team and the engineers. Let's talk one by one. So what is Scalable Research? Scalable Research, this is like a struggle that most researchers face that you are usually only task to do usability testing. What your PM knows about research is usability testing. But we as researchers we hate doing usability testing. It's like a very monoton job like you want to do more like a foundational research generative research like a research that can be very rich that not only fixing some design issue but also generate like product innovation ideas generate strategy. But the PM doesn't care your PM doesn't care your team doesn't care they want you to fix the designs by doing usability testing. So what we did at Bukalapak is I don't know whether this is ethically right but we hire interns so we hire lots of interns to do usability testing and the PM doesn't need to know they doesn't need to know where does the insight come from as long as their request is met then they should be happy and they are happy and I did this at Uber too because doing usability testing is not a really tough job because it's like a very structure you don't need to have that kind of like an art of questioning it's something that you can teach quite easily I would say that's why we have like a full time researchers kind of lead this kind of minions of interns that every week they kind of fulfill this usability test request so that the full time researchers they can work more on more kind of like impactful research project like generative foundational kind of research we also have innovation team so usually if you work in a gel method you work in a silo you only work on home page chat transaction and one team that kind of look at from the bird eye view and then see where to win and how to win so that's why we have this kind of team that is like not part of any agile team not part of any product teams to kind of do this kind of innovative research it's like a team combined of researchers and designers where they can decide what's next what should we build next quarters and I think it's very important to put them aside from that day to day like a very fast, fast space kind of work design ops and research ops I will explain later on design system is kind of like sexy topic at the moment but yeah basically we also have that team in Bukalapak so the next question is like does your team suffer this issue internally particularly around process, communications and file management difficulty on collaborating with other parts of the organization like of disabilities no one cares about your team they just think that you are designers making pretty interface duplicated efforts kind of like doing the same things non-existent measurements and fragmenting decision making so if your team are facing this issue then actually it's time to introduce design ops usually you will not face that issue if your team is still like 2 or 3 people but when the team is getting bigger then you kind of will face this issue so what do design ops do more like that's why it's called operation the background is more if you are coming from an agency background this is like the account managers so they do more like scheduling budget, annual planning conversation tool and procedures facilities, IT communication and coordination making the process between the team and so on i give you like a more easy example for example research operation are the one who do all the recruiting for the researchers so that we have this we have this saying at Ubers like let the builders build let the designers design and let the researchers research so that their focus can just do design or research or building something while the other kind of like course can be handled by design and research operation for example other things like they are the one who kind of manage all the tools that you want to use all the licenses measuring the impact of your works twice a year when you do your performance review you kind of do not know what you want to write in your performance review what should I write what is the impact of my design works what is the impact of my research work designers and design operation are the one who kind of make like the frameworks how do you kind of measure your impact how about design operation they are the one who kind of becoming like the guard of the design system kind of the whip the whippers of for example like design sprint make sure that designers can learn from other designers and the other important thing is that they can help to bring more about your visibility visibility is very important things I still remember when I was at Uber my managers told me hey YOL, you are one of out of other 79 researchers at Uber everyone can do conjoint analysis everyone can do cool qualitative methods everyone can conduct generative research if you don't have visibility and you cannot make an impact to the company which is true like my peers most of them are PhD they all know how to do conjoint analysis if I don't look if I don't really work on my visibility of my works then I will be able to make impacts the same thing when I look at my design teams if my design team doesn't have the visibility then how can my team can impact the other teams how my team can be respected by the product teams by the engineers by the other teams how can they believe of what we are saying so that they don't just think as old those people are just like indies who like to design things but they actually are not stupid that they are smart they can also help the company to drive the strategy so that's why you need this team to kind of increase the visibility of the design teams so what they do is like for example they send like newsletters once a month they make this kind of internal events external events sometimes it can be like really really petty for example we all take pictures together with the same kind of background and then we all put those in our emails in our telegram chat like a whatsapp so then everyone take oh wow design teams what they are doing is so cool just by taking pictures with the same kind of team but it kind of help us to increase the visibility of our team and now other team starting to ask hey can we also do that to our team so like a small thing that maybe we never think about that but actually it improve how we can work efficiently in our companies that's the first one about the organization that can fit to all cases depending on the situation of your company you may want to decide how you want to structure your product design teams the second one is to scale your team which one that you prefer growing your team or poaching both that's because of the width okay growing or poaching someone say both which is true at the beginning when I joined Bukalapak I need much more senior designers I need much more senior researchers so almost every evening I will have like a dinner with designers from Telefaloka researchers from Gojek researchers from Grapp trying to post them to Bukalapak but it's a tiring job it's a really tiring task for me because I'm an introvert person so every time I need to talk with someone that I do not know it's kind of drain my energy so after like 3 months I said I cannot do this anymore so now I put much more my effort to actually grow my team instead of poaching from other designers or other researchers this is example like I keep doing this in my LinkedIn hey Sakti, are you open for a new gig? I use the same kind of like opening line for everyone but yeah, some of them say how do you work in Tokopedia? sorry but sometimes I cannot keep doing there it's a tiring job and everyone I think knows Sakti you know, remember Ramda who came yesterday after I send this text to Sakti Ramda text me hey get away from my team the same thing with Tokopedia to hijack one of their managers they contact one of my managers so sometimes at that time during that 2 months I was thinking am I like a VP of design or VP of recruiting but I came to one stage when I realized I think the ROI will be higher if I need to grow my own teams because as you can see in this region there are not many talents there are not many talents like a PhD in UX research or went to like a really good design school like Rhode Island school so maybe I just need to grow my own talents so that's what we did in the last few months so that we actually finally successfully to grow our team from VV3 around May to around 121 by like a week ago so how we do that the first one is like I told my team that okay so we should not think at least the recruiting team the hiring managers that we should not care too much about where did they study and where did they work before but just care about their skills and whether they have the critical thinking like the trade off that they want to learn for example this is one of the articles that went viral in Takin Asia we have one designer that is like 18 years old he didn't go to any design school he just graduated from high school but we hire them and so far he has been doing really good job if you will see one of our redesigning maybe in the next few months he is one of the team members of that redesigning team after we hire him we actually hire more people like him because it's kind of like this article kind of inspire more of these people who may not have like degree from NTU and US but they are just good they try really hard to go into to go to this industries so now I think we have two more others designers that just graduate from high school they didn't take any undergraduate school but they are doing really doing good at Bukalapak the other thing is about diversity before I joined Bukalapak I have this bias that I always respect only people with like a PhD degree if they are like US researchers or they should get like my certificate from Stanford or Karen G. Mellon or they need to work in Airbnb, Instagram or whatever I kind of always look up at Angmo's but then I kind of learn that's like a really fuck up thinking, right? and I learn about diversity that if if you only focus on performance here at low diversity then you cannot cover only one kind of these circles but then if you have like a high performance team with a high diversity you cannot cover more circles so that's what we are trying to do we hire people with different kind of backgrounds we hire people from different kind of age and we are still working on it for example in the management level that we should be more open to LGBTs to single moms and can you imagine in Indonesia talking about LGBTs and single moms like they will throw stones to me, right? but that's what I believe my team didn't know about this but I did this test so there was like a pride day like a few months ago and I just want to test the water group, I say happy pride day and I was like shit, what will they reply but yeah luckily some of them say yeah happy pride day with like what's that? rainbow, jeep and everything they are good some of them so it's not only like that especially in Indonesia they also may have different kind of definition of diversity they kind of respect people who study abroad work in a top tech companies can speak in like a perfect English but I look at my experience look at my accent but Uber hire me I think if I was my hiring managers and listen to my interview for 5 minutes I say no I don't want to work with this guy that cannot even differentiate how to speak shit and shit still you cannot differentiate, right? actually her name is Susan Portolope she is like now a research manager in Instagram and from her I learned that how she actually doesn't look someone that looks like her but that can add additional cultures, additional value to the company not looking for cultural fit but looking for additional culture that can give like a different kind of perspective to the team so it's kind of tough for me now because now I need to kind of learn to do it like him for example like I have a very bias toward like Islamic radicalism, right? so I'm okay no bias, no bias it's okay they add different kind of perspective I'm like so there's a point when you add diversity to your team you kind of can make the team to be more innovative because they will bring different kind of perspective a point of view the third one, carry pad what times I have left shit okay sorry I will go it very fast, so carry pad companies with carry ladders had higher employee engagement, lower tuition and higher president available sadly from 100% of companies only 23% that has the pay models and only 5% that has carry ladders who has a carry ladder in your company? can you raise your hand? who has a design carry ladder in your company? raise your hand right? good luck and only 0.4% that have a measurable progression models and carry ladders is very important because it is one of the things that make people want to stay and they can feel that oh actually I grow I grow my skill I grow my growth I have a growth in my company so what we did this is the things that you need when you make a carry ladders a set of established norms this is what we made for our teams the hard skill can be research, design, copywriting the impact to the company how they collaborate with other people the development and the ten years you need to separate the tracks between IC and managers the builders and the managers individual contributors because not everyone want to be a managers so that all designers doesn't think that oh the other ladders that I need to go is to become a managers no we have so many bad managers because we force designers that just like to design to be a managers although they actually not managers they don't have the manager skill so that's why you need to separate it and the assessment system for the yearly review so this is what we did at Bukalapak from associate designers and if they are in the senior level they can switch into a manager level they can choose whether they want to be seniors like really good at your craftsmanship or becoming a manager and we make like a very specific what do you need to do to be able to go to that level so I told them hey let's say you are a designer one just copy all of the rows and make sure that you kind of fulfill those requirements so there is a target that you can go and that you want to achieve and we also make this kind of like a self assessment for every 6 months you kind of do this for your bonus too this is the thing that so the last one, sit at the tables this is what we always craft I want to sit at the tables I want us as designers to have the seat at the tables but how we can go to that level again it's like a combination of all of the things like a small things that we should do to be able to go there sometimes we think oh I just need to do good designs I just need to do good research but unfortunately we are in this kind of unfair world where the loudest sometimes can be seen as the authority or the truth so that's why sometimes we still need to do all those kind of like sneaky things so that we as designers or researchers can also be heard in the company so these are just some of the small things that we did at Bukalapak first I will introduce you with this guy the hippo in our company this is the hippo so can you not record this he is the hippo in my company he is also my boss and he is also the one who pays my bill right but he is like a guy that know nothing about design for example he say something like this your team is really good at doing design can you also design our next company the next building like the interior design hey YOL your design team is really good can you design the boutique uniform for our company actually Dina also ask to do that so he doesn't know any shit about design but he is my boss and he is the hippo so I cannot need to be the bridge between him to my team and the other product orgs to make sure that for him he doesn't care about B2 fuel design just care about business GMV net revenue profit he doesn't care if all of design is like all red or all yellow as long as it makes people click on it so how we handle hippos our hippos as I show this diagram to him it's up to you YOLIX you can choose you want to be business centric or you want to be product or design centric and you can only choose one you cannot force us to be for example design centric but ask us to design in 2 days we cannot do that so just manage your expectation if you want to be like a business centric company then I will make that expectation to my team too so I think every one of your sale effort or management effort need to choose where do they want to go whether they want to be like a product centric or business centric and there is no right or wrong you can be like Amazon booking the design is meh or really bad but it's really successful or you can be like an Airbnb or Apple where they only ship things that you like I just had dinner lunch with one of the designers at Airbnb in Seattle see again reiterate he said that at Airbnb we will still ship a product even though that new product or new design hit our metrics as long as they have the conviction that the design is good imagine that I will never able to talk about that to my hippos they will fire me right away they will still ship that product but luckily when you know about this you kind of can play make a balance maybe for some products you can make it like more product centric and some other products you can make it more like business centric so you will not sacrifice all of your teams to do kind of more like design centric product where you need to design spring we have one team called one tribe called growth where they don't do any kind of those design spring things because for them they just need to ship things fast and as you can imagine this is like the favorite of Felix but I'm the one in the middle who kind of need to make sure that for those kind of things that is not foundational fundamental then go ahead design in 2 days go ahead but for other things that are really important like redesigning your home page redesigning your core flow and it should be really design centric also how you kind of design your working models do you want it to be more like a studio cultures or you want it to be more embedded models and it kind of affect how the product will be made because studio cultures it will take more time to do it we did some pallet in our teams and some of them work and some of them did not work and again this is a preference and based on the products that you want to build and other things like this is how I kind of structure every quarters that we need to do 2 stuff solve the low hanging fruits of the designs but also think about the big bold bed and you kind of need to think which resources I should put on the softening low hanging fruits and which resources that I need to do for big bold bed those minions, interns they are softening the low hanging fruits these people who come here that we are willing to pay for their training they do the big bold bed and they kind of they can also do bold but you kind of need to know not everyone can do big bold bed then they will kind of question the value of your team what are you doing like 3 months research and what you are talking just what is the design accommodation I need that I don't need 10 wise why Indonesia do not have bank account I don't care, he doesn't care one of them the research came from 2 weeks research to a remote island in Indonesia and they came out with this kind of very agency kind of presentation like a deck you will kind of make after you smoking a weed and I told them hey this is fine if you present this to our team this is fine, it's totally fine and we will all be happy with this but if you want to present this to Willix never do this then they will cut our research budget so make sure they kind of teller based on your audience this is the thing I said that we are like the most cool team in the company and send our money newsletter and we really got a lot of visibility just by doing this kind of small things we also do some calibration so when I join I kind of compares the salary of the researchers designers and even to engineers I kind of calibrate the salary I kind of increase their salary so that I can whip them more but I think it's important so that because even though the company has the policy don't talk about salary they will talk about salary between them for sure so that it's very important that someone is kind of valued or getting the rewards that what they deserve it we also have the values self value is very important so that you kind of have something to rally on together so we have like an ownership, speak up and then the Indonesian word for don't be stupid we also had a lot of this kind of events like as you know most of us, most of my managers are not like a manager that has like 5 or 10 years experience so they usually just move from being like a center designers or managers so we are all clueless we don't know how to manage people because like 2 different kind of roles so that's why I usually invite people from like this, that's Susan the ones who kind of hire me with this accent and there's like Molly like a director designing Google and Edu from Amsterdam we invite them to come to teach us we did a lot of this kind of event so we do this like call run up event every 2 months so sometimes we think oh why we spend so much energy for something like this it actually help my teams to feel proud of the company we are actually part of this school team part of this school design teams if we don't do this other companies doesn't know Bukalapak Design Team again like a small things but it actually helps to improve the day to day job of my teams because this is more like improving the internal motivation of these people than like an external motivation of you need to work we send them to conference that's why some of them are here and we do also a lot of fun stuff small things but it helps and it's really important for you to kind of think about this other than design or research work something that kind of affect their kind of mental sanity because it's like a tough job to be designer and to be researchers so this are the result I have I think okay sorry look at that this is our product detail page can you see it? can you imagine this design still make us as a unicorn when I joined Bukalapak and I look at the design oh my goodness I told them this is not korea design and I told them I will never use our app until you change that design and this is our new product detail page with that new design we increase our GMV by 7% that's our home page another shitty design and we change it to be like that so when you open our app now that's our home page what I want to say this thing will never happen if I just ask them to work I just need to make sure that they have this good organization good working environment like good managers to execute this kind of good designs and you will get this designs if you never think to take care of your teams you just think okay how to work for sure you will end up with this kind of shitty design so last one there's a book that is really important I think if you are new in like a design management role and there's like the 101 like the bible for product design of leading product design teams and they have like this a very theoretical kind of qualities of effective design organization and you may can see at the things that we did actually kind of to support all of those points so we are all in this shit together trying to get the seat at the tables as a designers and researchers so I think we need to step up our game so that we can be in the same level to those ex consultants, BCG consultant make it C consultants that usually get the seat at the tables and to show them hey that we, designers, researchers are actually smart and not that go block cool there's our instagram account just feel free to shoot to that email thank you everyone thank you YOL for such a refreshing session