 New完成. As we've gone along, we've realised that there is a few things we would have loved to know that hopefully we can share with you to make your life easier as you do similar sort of things in the future, basically. So, we're going to cover off quite a few bits and pieces as we flow through this talk today with various topics from business-side, through technical side, and bit combination both as well. So, trying to bit something for everybody. Ond y ystod y gallwn gwneud y porff로daeth efallai efallai efallai yn fawr i'n fwy o'r fwy oed yn fawr. Gwyrdigaeth dda, gwyrdigaeth yng Nghymru, mae'n eistafod wedi bod pwysig yng Nghymru. Dyna ymddangos ymddangos, mae wedi bod yn byw yn mynd yng Nghymru, y cyfan y gallwch yn fawr yng nghydfodol. Mae'r cyfan yng nghymru o'ch cyfan yng Nghymru o'r cyfan yng patreon Draw. So it is always good to make sure that is clear, upr siècle, and particularly in the bigger projects projects you find that people have shumped around a lot as well so there have been a different meeting with a whole new group of people and they might have been invited by somebody else they might have no idea so it is introducing themselves in the same way so making sure you all know why you are there because if they have to pass on that invite then this happens a lot ddweud o'r ffordd a'r newydd, dwi'n fawr ychydig o'r newid o'r ffordd o'r ffordd yn ymweld, i ffyrdd y cyfnodol ar gyfer ddweud o'r Gael Llywodraeth, a'r hoffi'r bwrdd o'r pethau a'r gwaith oedd yw'r chlesion wedi'u gweithnodol i'r pryngylach, yna ddim o'r ddweud o'r ffordd a'i ffyrdd o'r ffordd o'r newid. I ddim weld o'r hyn oes, ydych yn ymdweud i'r ffordd o'r ffordd o'r ffordd? A dyna ydych chi, rhaid i'w lleilus mewn bethau ymlaen, rhaid i'w lleilus ar y methu phaith yma? Rhaid i'w lleilus ar y marwng, ar gyfer unrhyw wych fel ymateb o casu cyfwladio, i unrhyw sylfaen ar gyfer unrhyw cyfwladio, ar gyfer 5 yma, ar FBC ac i gael 118, Ychydig yw'r fawr 12-3-3-3 rydych ar hyn, ac mae hyn yn ei wneud 10 oes, o g brakingol. A fyddech chi'n gofyniaith yng Nghymru a chyn rwy'n gwneud ei roi'r hoffa i'w bobl yn llawer o chimweithio cyllidol wedi eu ystyried. Mae fwy o angen, ddim yn ymdweud doliadol, ac mae hyn oedd yn ei chwynhaeth yn banyddol yn ymddir. The reason you'll get a mix of business and technical conversation in this talk is... …'cause I've sat on both sides of the fence... … favourite projects and product management as well as... …a pretty hefty time spent on the people doing software engineering for that as well. So that's how we're putting together this strange amount of information of the two. Both mines maybe ways with them. I'm not going to talk about the enterprise... I am fyddodd yn ymddangos ar gyfer y ddwg i'r cyfnodd. Yn ymddangos ar gyfer ddweud o gydig o'r cyfnodd, o'r ddig o'r company. Felly mae'r fflusiau wedi nhw'n gŵr i ddweud yn gwneud. Ar ddod, mae'n ddim yn ymdaint... ..yng Nghymru sydd wedi'u cyfrifio ar y maen nhw... ..yny'n cael ei ddweud o'r cyfrifio o'r cyfrifio. Ac mae'n cael ei ddweud o'r cyfrifio sydd e'n ddweud o'r ddweud... Ond, if you're in the same steam, obviously it's been a factor a lot for these six months, one year and things like that, right? And from my point of view as well, one of the things that in enterprise impacts is quite the findings, that the rules are ready to find something that you would find when a customer is at the company ward. Some of them have different hats and they were the owner of the board of the platform. Yn ymchwil yr un i gael, oedd nhw yw i'w ddim wedi'n meddwl a llwedd yn rhai gwelio'u cyflwys. Felly, mae'r gThere mor hyn oedd yn cael ei ddweud sy'n dweud. Yn ychydig iawn, am pethau'r gwerthiau, yn y meddwl yw ddeg. Mae'n ffrindio ddeg, oherwydd ein ddeg yn ateb o'r dwydig. Mae'r ddeg yn rhaid i'r bwyl. The size of the project we are also going to talk about six months one year, two years, something is longer. Sometimes even we are talking about five years period because when you are talking with these kinds of companies you want to sign longer and something like five years is totally ignored. We are going to have third parties, a lot of third parties, and with third parties I mean agencies. Sometimes it's an easy bit in the design, sometimes it's a, it could be a partner doing some development, it could be some external project, which is going to hook into your project, it could be internal departments that sidesforce the tree that you have to hook into that, and all of this brings out the complexity, right? The complexity is in development of course, but the complexity is in communication as well. And on top of those complexities, you have to think that we are talking about companies that are really well known, so there is a lot of pressure in terms of if you make a mistake, it could end up in a branch, right? Everything has to be very well thought and you have to have everything, try to have an integration. And next story, thank you very much, and at the end, some of the things are common for enterprise, but you will see that some of them are common also in small companies, right? That's not for companies even, it's even for individuals, but if you reduce costs, you increase your revenue and you have more money for whatever you want, a company or in the research, we are talking about big platforms. Dothraus sites, but very soon it could go into a thousand sites. On that complexity, it needs to be a centralized way of managing that, and that's a common mistake that we see a lot of times, or you jump in a project which has been running for a few years, and then you want to migrate from a different CMS, and they are going to Drupal or whatever, and you can see that they've been running a massive company in different companies, and the department doesn't have their own thing or not, and suddenly after two or three years, they're like, this is company, you need to put some money in that, and you need to put some centralized management in there to make sure that, for example, what they were talking before about the risk, if something happens, it will impact the image of the company, so you need to be engaged with that as well, on how you are going to manage that kind of thing, and I think all of that should be done. And it is quite, well, this is not always the case, but sometimes you have to try to steer the customer in the right direction, how many sessions, it's not always one of the first things that people think when they are starting a project like this, and if you think about that, you wouldn't expect Apple to have a different image if you open Apple in new game, and if you open Apple in new game space, however, a lot of other companies, it's not the first thought, so to say, and that, again, brings more complexities when you have to fight between different angles, maybe bringing complexities or things that are fighting each other, and we will go on to that today as well in a few slides. Cool, so, yeah, like I said, there's big teams with lots of complexity, and the best way to deal with that is to kind of plan to find what you're doing, and the first thing you do is you need to actually work out what you're building and why, so this is maybe one of the products that Guy has on, but it's relevant to all people involved, because if you haven't got a clearly defined product that you're all building, that everybody's working towards, then you can't really all drive down the same road, you're going to end up doing different things, doing chaotic stuff, you won't know what leads to finding, you won't know what needs understanding to breaking down and remaking for development. Sorry to interrupt you, I think we are not recording the room, we'll be able to, I don't think there's a permission, could you, someone in the room, could you start recording it? Oh, thank you. Everyone let the room. We're still here. We're here. Where is? There's a little button on the bottom. You mean that one? Yes, according to the cloud. Okay, so you're going to start the room? Yes, start again please. Okay, cool. Right, so, yeah, so kicking back in then from where we were, so we're talking about the defined product, knowing what it is you're building and why, because you need to know what you're building to be able to all move in the same direction and end up with a positive product that solves a genuine problem. It's also interesting to know why that organisation should solve it as well. That's kind of a key and crucial question which a lot of people don't ask themselves. So, anybody that plays the business strategy game will know this is a drastically simplified diagram, but it kind of gives an idea of what I'm trying to talk about. So at the top there's a business. That business has a vision. If you are someone like IKEA, for example, to give a really well-known example, their vision might be to create a better everyday life for many people. It's like a super top level thing and that could mean many different things. But obviously what then happens is strategies are derived from that vision to then start talking about individual ways that that vision could be realised. So, you know, they could be talking about, in fact, they have meatballs, which means parents insert easy meals to their kids every Monday without too much stress. You know, that's a massive improvement. My wife's have been able to do that much. Or the furniture though. So those are the individual strategies. And then underneath those strategies you're then thinking about products that you can create that help realise those strategies, help implement them and solve problems to actually push that strategy forward. So, as you're building out the products down here, you should be able to track back up to the top, to the company's vision and see why what you're building has a genuine impact. So there should always be traceability up. If you find that you're working on something and it's not clear why you're building that thing, what that thing does for the organisation you're building, I would employ you to ask why. Now, hopefully somebody will be able to tell you why, but a lot of the time you might find out, because it's such a big organisation of an enterprise, you might find that it's actually somebody over here has said, Oh, I've got an idea. And they've kind of run away and might score some money. And they haven't forced it through. And actually they're building something, which is there that will have projects, maybe, which sounds ridiculous. You might be building a very large project, but it might not time the business goals. And then you'd be amazed how quickly you can suddenly find somebody who'll come on to that, maybe in the finance department, will ask the questions of why is this thing being built. And if it can't be validated against those visions and can't actually work, you know what I'm talking about. Then you might suddenly find actually the whole thing banished from underneath it. So we've actually seen that happen. So I would employ you, just be aware and always, always track it up and make sure what you'll do is really evaluate that business. So it makes me, like, someone got the keys of the value and then they were playing with the card. We're not clear on the direction that we're going, they just got a new toy and they were just playing with that. And then, as you were saying, you get the trouble because finance is asking questions. Yeah, totally. And then if it's kind of like scanned down through, so if you've built all this out and you've got down to the product points, what you're hoping is that for all the technical people in the room, you're hoping that somebody has said, I've got products X, that product is going to solve problem Y, and then that problem has been broken down into features like the definitions of the parts of the problem that are to be solved. And it's been built out so that you've got a clear roadmap for maybe 18 to 36 months of features, of things, of kind of defined segments of the problem that all go together to build out that product. They all relate back up, so all the way up to strategy is the business vision. So you can really trace and see exactly what's going on. And then you would always expect to see a decent amount of visibility in this for a period of months. You don't have to have, like, you know, this is the, anybody who's done the agile stuff, you'll kind of recognise the concept of very granular things, you know, down here when you're about to build something, it needs to become an eight-point story that one developer can pick up and work on in a sprint. But in 36 months' time, you might be talking about something that's just like three words and a theory that I want to personalise my products, you know, but at least you know you're moving in that direction. And that visibility, that plan helps you think about what you're building down here, so you might be building out a very small feature down here, but if you've got visibility that in the future you're going to be working on and the things that do roughly this kind of idea, it might drastically change the solution you're going for or it might help inform decisions if there's maybe two routes to take. Like, if you know down here that translation of multi-lingual is a really big thing, you might use one solution versus a completely different one if you know it's going to be in one language. So then, once you've got that and you've kind of worked your way through, just be aware that for whoever's defining it, you need to keep redefining as you learn and for whoever's building it, expect that that roadmap will change, it should be organic, it should evolve, it should be a live document, but it should be there as well. Just it's going to manipulate because you do learn things, you do change your mind, you do see different things as you move forwards. Everyone then, anyone in the process can and should be able to ask why. If you're not sure why, you can't make the right decisions and anybody should be able to trace it back up to the top and should give you a decent answer. If they can't, then start waving red flags, please. You can use methodologies if you're not confident enough to challenge senior management, maybe, or the product owner if you're not that way inclined if you want some assistance in asking why. You can use methodologies or assistance to help you out with that. There's a book by a chap called Master Keydon called Inspired and in that he lists out, this has been happening in many places, it's quite a good little list, quite a simple list in that book. 10 questions you can ask, which would help you decide if you wanted to build a product, things like exactly who's the problem, sorry for how big is the opportunity like the market size, how would you measure the success of KPIs as a product? That's an amazing one. A lot of people don't have key performance indicators associated with the product, so you can never feel like you've used it or won because there's no metric and you can apply to it to demonstrate success. So all these kind of questions, there's a lot of things around it. If in doubt, use those to help you and ask the questions because you should always do so. Okay, once you've got all that to find and the product is there, then you need to start the project discovery, that's our terminology. I know there's many different ways of terming it, each company's got their own thing, that's the active term and I'm not used to it now. But essentially it's the translation process. You need to go from business requirements down into, or business problems to soul weaving is the ideal solution, down into actual technical specification that the developers, the boots on the ground can sort out and start building forwards. So in that process, you've got to deep dive into everything. You've got to really drill down. There'll be a certain period where I showed you that the more granular features are lined up, you're only going to work with a certain point of the roadmap. But on the bit you're working in, drill down right into it, get to know as much as you possibly can. There's always going to be things you don't spot, but you should never be afraid to ask the stupid question in those situations because a lot of the time you might just end up thinking, oh, everybody else knows what that particular acronym means or why somebody's talking about a C-mail. Most of the people in the group might also have no idea, but they're also just thinking it's a silly question. So if you ask the stupid question, you've been made to how valuable that can be in those sessions. Just keep drilling down, drilling down, drilling down until you get the information you need. I actually wonder if you have a couple of next points. Yeah, so the project is going to me is one of the most, not the most, but one of the most interesting parts of the project because it's like a honeymoon, right? You're starting to discover, sorry, my dog is just doing a lot of things. So you're starting to know both the project and the customer. This is where you get to build your new door, right? One of the biggest problems that I've seen on most of the project is that the customer normally knows what he wants or he thinks he knows what he wants. The conversation is really easy. It's going to go in the wrong direction. They're going to show you how they want to build this thing that they have in mind. And this is really wrong because you are not here because you just build things. You are there because you have the dog. You have to learn it to find the business side, right? Find what you want to do. And then once you have all the answers or the questions, how to build it is a technical decision, right? I have a lot of examples that could have compromised a project like you go into a room and the answer is, I want a micro science. I don't know why everyone wants to do micro science. I want users to be able to register when they are in this micro science. You have to link notes, content pages, right? You have to link that to users. If you try to do it in the way the customer is asking you, and you are going to get really silly in each other, right? Because people are flexible about what you should be getting from and how to do things like users are not going to gain what people knows how to do well, right? If you start to see the conversation on what you really want to do and you start to tell people to be able to go to this place, this micro science, but not able to see all this. You start to see partners because you have the knowledge, right? And if you know a little bit of trouble, and you know that you have organics, for example, and you are trying to try that, but it's probably much better than having to be in the whole solution from the scratch and then you come with the price of the mediums of users per day. Other products that can be built around that, right? Yeah, in this space, you have to be very strong in terms of be confident in your technical skills and make sure that sometimes, because sometimes you won't be able to do that because sometimes customers just want something and you will have to. But this is where, if you are the expert in the room, the expert in the room, and you are recommending something, it's very where that you will go against that, right? Because that could compromise the whole project. Textures, right? I love this phrase and it represents a lot, right? If you are asking, if you want, if you just want your customers asking, for example, in the case of Harry, people just want the faster horse. Because there was no cars at the moment, right? So you have to take the knowledge, you have to be responsible, you have to try to steer the process on the right. Next, too. I don't do it in my part, just at the moment, which will be the next phase, and I'll give it more taste on how we sort this from the two-pal point of view, right? Without going into much detail, I think that we cannot go in this talk on the next thing. Next, third. Now, do you know how Drupal Mwtisai is? And you may have heard that Drupal Mwtisai is dying, or if you look at any of them, you will see some things that, especially one post in Drupal that has like eight years old, and that was in Drupal. Mwtisai in Drupal it was meant to die well. Drupal Mwtisai is not the panthea, it's not the silver bullet, but it shows very well some given progress, right? And you have to know what you're using. At the end it's like it's about knowing the tools that you are going to use, and knowing when you have to use them. If you remember when you were, we always started to talk about some enterprise, and what kind of things, or what kind of product we have, an enterprise, we were talking about harmonized product, harmonized platforms. We are talking about a centralized management. We are talking about one team managing all of that to save cost. That is exactly what Mwtisai is going to be, right? You have a single code base, and you can spin a thousand of sites if you want. That's a problem solution. I have seen sites, platforms with thousands of sites running on this intent too. And it's a scour, right? And it's maintained by the community. So definitely Mwtisai is not dead. Now, next slide. Some problems, because that's our saying, Mwtisai is not the silver bullet, you need to know when it's good and what it's not, and you will find problems. It's not necessarily a Mwtisai problem, but for example the commercial management. Conferential management is not my same thing. I don't want to buy it. And if you have two or three websites, it's very good because you've come back everything in your database, but you can go back, right? Easy. Now, what happens when you have a thousand sites? Or when you have a thousand sites? And what happens when on those sites, some people want to make ten years? And that's sad to be a bit of a nightmare at the moment then. For what I've seen so far, configuration management is not what is signed for this. There are work grounds on that, but for example the old way of doing it in terms of packaging features and modules, and then there you end up making ten years with dates. That work really well in the group and segment. This is for example a model that can work really well with a thousand sites. I don't have some sites. Yes. You have to be careful when you start to see that they are trying to push the limits of what that platform should be doing. Because as I was saying, some people will want, or some mistake holders, will want the control on personalising a lot of things on their website, although it's a part of the platform, but when you start to see that this is getting out of hand, then you have to stop and think if that product needs to be spent, right? For example, at the moment where everyone is trying to personalise their own site on the website, is trying to change the colour, so that looks like a lot of work needs to be synchronised between different websites, and it looks like a good compromise. For example, if we are talking about mixing and commerce projects with our preferred website, it's a very different project, right? So, when you see features coming to the platform like that, then you have to question the customer. Usually, you want to go through this, which is going to cost more to maintain in terms of effort and putting things on the platform that could work well with each other, right? And that's where you go on the next slide. Where you have to think about coming on to the next slide. Which is about breaking the product. Do you want to say for harmonised platforms? I aim for foreign sites. For example, or commerce, or you can build products, a big platform around something which is well harmonised. But if you stop to mix things like that, having anarchy is very different in DNA to a foreign site, you will find that you are able to have a lot of clashing between the features that you are trying to build, the ones that the platform really has. That means that what you said is not the right solution, not necessarily because there are things that you can do around that, and it's just being clean. You probably know all the learning institutions or the learning using learning to use that example for operating to work with you once back, where they wanted to build a product which is quite different to the product that we were working on. So the next iteration looked like this is coming to us. The company said they were splitting the code in the core into that, to do with the different products or code bases. If you think about that, we are coming back to the previous scenario where you have this approach where you have maybe a lot of countries, and it's one of the countries that have different products that goes probably in the wrong direction because it can't get back into that Chinese and maintainable source. I was mentioning learning is an amazing way of simplifying that. I ensure for example that when it covers a portfolio cycle it will be very different, but you can't find which is common more. You could build a situation for the company that could solve a problem of having that code base and then give the other bits that the other part of the department want to maintain. This makes a lot of sense for several reasons, but if you are in an impressive project or company, you are not going to need things like certifying a product. If you have a hundred or dozen different projects you will have to certify each one of them and this is an expensive and very exhausting process that can last weeks, months. If you go to the project having a line not a line, it means that you have a code that code can be certified and then you benefit of the other products not having to certify at least the whole product, because the code is already done and you have another big advantage that the biggest part in what the other products are going to use is probably the code on its side but you will also have shared things like sign of things things that will be in the code and the modules for example that are going to be covered on one of the sides and having that in this approach of using a company institution let's call it corporation institution it brings a lot of benefit not just that certificate, so I was talking about it also when she hits a fan all the space but when you have a night in the night that the security team is saying we have a huge problem with security release that provides revision that's amazing and security management we are covered and confident that suddenly when you have a lot of products and maybe have two or three teams that's a problem to bring everyone at any night and having to take all of that right and you have a more centre as a product and you can do the work in one and then it is going to provide those changes to the institutions just the way I know where the other one is to much Cool, so I was just talking very much about managing from the technical point of view like many many sites and these numbers he is talking about are not unusual like the thousands of sites even if in the tens hundreds still there is a few things from the platform management point of view this is the more the business or the user side of the management I guess maybe the utilization of your products there is still a lot of considerations on that side as well that is also worth thinking about as you are building out of the product so for those updates for the visibility you are using building out your products that you can take a one to many approach by that I mean so just as the security team might say hey we need to release this security update to a thousand sites in the next ten minutes or we will be fine three billion pounds for that which is kind of crazy thing to have you might suddenly find that the same thousand sites needs all update there they could be compliance message for something like this because the entire countries change their legislation or many other situations so you need to be thinking about ways that you can handle that in a simple and fundamentally manageable way so if you have got to jump in and you have got to make a text update to all three sites then you can have a very approach so if you have got to all three sites they can be individual sites you jump in and you can make that text update copy and paste it's fine we've got to do that for a thousand sites you might need to employ the entire team of people you're going to solve it with other people or have people running on that for weeks at a time there's so much room for human error there's so much room for confusion sites might get missed off wrong text might be copied it's just not a viable solution so you need to work out ways for how you're going to manage that in a viable one to many way that also goes for things like generating visibility of what the sites are doing so it might be that one of the key KPIs of your site or your platform is for each customer their user base for how frequently they're being used so you don't want to have to log in to each site or each site's analytics to see that leaving that data from central place or it might be you've got some particular risks associated so in enterprise and again security can be a massive thing and it might be that you need to really ensure that only certain people or certain numbers of people or some criteria around say user roles are available you might need to limit the admin user roles to one or two people with particular security theories at the site of that platform so you need a way of being able to access that data have early warnings or if something's obviously going wrong you can progressively check these things or even just say a dashboard dashboards really are your friend when you're playing a platform so at a glance just have an intelligent particular dashboard with key bits of information can really just help you see show or alert you to things that are good or things that are bad so far as whereas otherwise you might never know you might never know if a customer is failing and they're about to leave your site and save down to three users a week you just wouldn't know these things unless you've got the tools in place so I'm a big fan of thinking about that because it takes efforts to build that in across different sites you know when it's developed you know in your eyes or specific mechanisms to do that but it's the effort up front is well worth it and it's something that you can be advocating very strongly to your product owners because they won't necessarily think about it until they're stuck and they don't have that data on their screen because they don't know what's going on if you think about that Mary this works quite well to the contributed approach of the community where you are developing one too many when you are building a specific module you're not building it for yourself you're building it for a massive usage space or as much as possible right so you're not thinking on a single solution you're not thinking on about how can this interact with different environments with different and this is what I find I found that this opens a conversation to try to send to the company that you need to open source that you need to share this with the community because at the end you are building something but I'm not going to go into a why of a choice book but this will turn right to your module, the maintainers that will give you feedback about that you will be missing a security holes that you know so whatever you want to see I don't want to go into a why of a choice book we don't know that sorry Mary so with that said I'm aware we're running shortish on time so I'm just going to steam ahead to the last bit which is the people for the job now again Alex noted at the head of this session that there's a workload of people required for the enterprise platform and just to hammer it at home because I've had some interesting conversations where people have sort of eventually realised they need these team sizes but they haven't been able to set it out so they didn't talk about it early enough with the people that finance their projects they've sold on their dad's car keys so they've never been there to actually build out the correct team for the job and it's really come back to by them so just basically what they're saying there so this is just a representation really it's going to get you an idea even if you look at it from the yellow being say the clients for an agency situation maybe and then the blue being the delivery team so you're talking about even at the client side product owner platform owner that I'm just saying these people are going to spit out they need to be there and they need to be there to you know keep each other honest and they're going to make sure they're getting the right information so they can be down as the program be the technical lead and then you're going to need tiers of management on the on the program side and the tiers of management on the technical side flow down into a whole army of devs possibly and then please please please please ensure there is a suitably sized army of QAs under the first devs I have never been in a situation where I've had too much QA resource on a project and I don't think there can be they are amazing people they do amazing things so make sure they're there for a platform you're not just talking about testing manability because you're talking about regression tests because there are so many people using so many different sites on the platform you make one mistake it's not just one mistake on one side it's one mistake on a thousand sites so everything is kind of exponentially worse so testing it the automation test, the visual regression test QA is there and then you're going to need just a whole separate function to ensure that platform whichever platform you're using is solid reliable and up because he needs to be big companies need a real solid base otherwise they might be losing millions of dollars every second they're not alive you've got any further thoughts on that team structure Alex from your point of view yeah it seems big like what do you think of the technical side your technical problem with you which is at the A is well you don't necessarily have two A's sometimes you have two or three A's when you have two or three A's I understand the need of that once I have to work I have to jump another break where I didn't have the DA to help me because the technical problem is responsible for a lot of things around and the steer, the product and talking to customers so there are a lot of hidden gems hidden tasks and when you see all the machinery working you realize how easy this can get some of the companies that will say in all the flight frauds it can put a lot of pressure on the team and come very easy it's no one to mistake because you're too busy time to buy a flight plane and then even once you've seen all that there's other sections to this group that should be considered and often or not because people will think about getting to the point of delivering products but when people think about the user fund think about it as a funnel but it's not as an hour last because this thing continues on as it goes through life so you need to you know, once you've got the product built you need a support team there to be helping the user base because the engineering team are not the support team if you start with a live product you've got 50 sites running and each of those sites is asking you questions about how they enter content or that this button is not quite what they expected to be suddenly the engineering team is not doing anything so you need that support you need that success team there as well to do the training, do the onboarding otherwise you might suddenly find a product owner that's completely vanished from view because they're trying to onboard 20 sites a week they can't build out your roadmap anymore so you don't know what you're doing amazingly, internal product external product doesn't really matter you do need a marketing team going on so you can assume that just because you're building a product an internal use maybe it might be an internal tool that you can mandate that and everybody's going to use this it's amazing how many different ways people find to exercise their free will go elsewhere and suddenly you're in a situation where you're building out a software as a service and people are not using that this system you haven't spoken about it you haven't advocated it and that can suddenly really backfire when you're losing your use space just as any company might be scared of doing you can get your message in so all these there's many more as well but just be aware that there's a big team surrounding these big projects I'm thinking of one of the product owner a product owner was talking about that and how he can overlap with the other but not really because not all many of the product owner is more business focus on the platform for example it's more technical right and they both jobs are to fight each other behind because well not it's not like that obviously but the product owner will come with requirements that it will come to you as the company will see this can clash with the platform so you can embrace the platform there and show that in order to get some support at the end it's what I was saying at the beginning it keeps the whole ecosystem healthy because no one has to manage to do something that can compromise the product so to sum up Alex I was talking at the beginning about the big picture this is I wouldn't have to say this like a hundred times you need to know what the scope what's going to be the planning here in two years within projects where we were getting requirements by jobs and this had somebody to be able to draw a loving product because a simple example I was talking about I am a builder of products if you only know small drops of what's the whole requirement of the platform you really don't know which one you're going to use at the moment we didn't sign a paragraph because two years ago the IOB was not the same and it was not known when it was going to be a good product whatever the big problem is that a few months after that we discovered that you have to support a synchronous translation and that's a big problem with a paragraph in IOB that is a lot easier to do as I said maybe it has to it's going to make it change the quality technology you have defined and it's going to waste time to modify things like interactions I don't really know how to you could improve that but this is a big problem when you are working with the technologies if you have other requirements but suddenly when you have different deals and you have to both work all at the same time with RUPA that's a big problem because you have to increase it's not that you're going to see a new API it's something that changes with RUPA and it interacts with it's all more complicated because here we didn't both the same time so the communication between the teams is sometimes you have to be out in the room for a couple of days and just talk about RUPA or how when they are in Europe they interact with the whole ecosystem but more about how you are going to talk to each other and that kind of flows into the step 4 which is it's all about negotiation and the psychology and if you really get stuck it's the willingness to escalate those issues and ask for help but in a large enterprise sometimes people just use an escalation as a standard so you might be scared of the cat amongst the pigeons but sometimes that's just here you won't forget things done because to be honest it might delete all your emails especially put them in the CC with urgent on them so you might actually realise that Stakehall hasn't never seen what you're saying until you escalate please make sure you've got the right team the extended team including those other actions like the success and marketing support behind you to mean that the product is a success when you awesome people build it but also it's a success going forwards because that just makes everybody happy and means you're really proud of going at the things you've built Thank you very much unfortunately you've run out of time so we probably haven't got any time for questions but thank you so much for doing this No worries, thank you everybody for listening Thank you Thank you very much Bye bye