 Good afternoon slash evening, everybody. We are presenting on implementation in the MVP. So if you are not intending to be in the session, there are some lighted pathways leading to the back door. Make your way. I introduce the team to start. OK, look at you. Have you ever looked at a really cool product or a website or an app and wondered how it got there? An app or a product that does so many cool things. Do you think that it got there overnight? Or a bunch of people spent a number of hours, sleepless nights, and just put everything out there at once? Everyone, my name is Nagin Hagigi. I'm a program manager at Acquia. I joined Acquia in 2015, and I was introduced to Drupal then through Acquia. And I've been working on many different cool projects at Acquia starting from multi-layer enterprise sites and migrations from Drupal 6 to 7 and 8, content websites, brochure websites, redesigns. And prior to Acquia, I worked in entertainment, where I used different technologies to build TV apps, like Roku, Apple TV, you've all used them. And also, prior to that, I spent years in digital advertising, which I flexed my producer muscles of working with creative people and building really cool stuff, like digital walls and browser extensions. And so I'm here today to share some of my learnings and experiences in the past 10-plus years on how to really arrive at a minimal viable product and scale from there. So I hope that you can find this session useful for your future projects. My name is Jen Stramick, and I have been in the Drupal community since about 4.6, so a long time, working with different customers from small, large, and medium application development. I've worked with offline youth program development and also green building construction. So I've applied the principles of MVP to a lot of different scenarios. One thing that is common among all groups trying to do projects is that they almost never have enough time, budget, or energy to do everything they want. And there's always more to learn than you know right at the beginning. And so it's the tendency of customers to want to do everything, and they have their list of 1 through 100 absolutely critical requirements. And there's still more to learn that even they don't know as you start to unpack and deal with the development plan and et cetera. So if necessity is the mother of invention, then simplicity is the child of intention. And this is just to say that most often, the simplest best applications do not get developed by rushing and hurrying, but by careful examination of the requirements, careful examination of what the project is trying to achieve, and careful selection of what's going to be the most important thing to start with, and then also being willing to listen to the feedback once you put something out there and to adjust accordingly. So the intersection of simplicity and quality really is the sweet spot that we're looking for. When you're working on any project, there's a lot of tension, completeness and simplicity. Sometimes people venture a guess as to what their customers most want, and they miss. And they've spent all their money on that guess. And now once they find out that they've missed, they don't have any budget remaining to do anything different. So they're sort of stalled out for a period of time. There's also tension between speed and quality. So wanting to get something to market really quickly, sometimes you miss things, and you get something that's not quite what's wanted there, and it's not as working as well as it should perhaps. And when you go for quality, sometimes you can over engineer and create challenges for yourself that way. So there's lots of distractions and hidden complexities within a project, and there's always tension between these different aspects of things. If you are over complete and over fast, maybe you are developing things that are not exactly what the customer wants, and you may be guessed, but you guessed wrong. If you're going for speed and simplicity, maybe what you come up with is something that, like early Drupal, like content editors could have a really difficult time using Drupal 4.6. They got in there and they're kind of like, what do I do now? And so you have to adjust for a little bit more usability. And sometimes you over engineer because you're going for quality and completeness, but you're completing with quality things that might not be on target for the users. So we want to try to, when we're working with MVP, we want to try to get to the minimum viable product. The least thing we can do to get the most feedback that can give us the most information about what to do next. And we're going to cover a bunch of tools and techniques that help us do that in a couple of examples today. They're here. Nobody's going into the light. I hope nobody's going into the light. Okay, so some practical techniques and negotiation tools and negotiation techniques. So our first goal when we are working with a new customer and when I have working on a green building project or with youth development program is to really get an understanding of what the vision is. What's the core vision? What is this thing trying to achieve? How would they describe it in an elevator pitch? And really making sure that I understand that and the development team understands it so that as we're listening to what comes next, which is the requirements, that we're listening from the perspective of what they're actually trying to achieve and bringing our knowledge to bear where they've done a lot of planning and detailing up to that point. We really want to align on the vision and make sure that everything stacks up. You all, if you've ever been to a chiropractor or had a back issue or something, you always know when something feels out of alignment and when it gets in alignment, you know it too. It feels right. When you're in a conversation that's out of alignment or there's a lack of clarity, it feels off and sometimes there's arguments and there's tension and there's disagreement. And if you start that first step, the foundational understanding of what's going on and build from there, you'll find that things seem more agreeable. Not every customer is ready to take on an MVP approach. It requires an understanding of how the web works and the value of bringing something quickly to market and listening for feedback. And it can be challenging, especially when a customer comes to you with all of these hundred requirements are critical and required. We don't have any place to negotiate about what not to include. That's a really challenging place to be. And there is not, you wouldn't start to address that challenge by arm wrestling over each requirement. You would step back and say, what are we trying to actually accomplish here? You wanna spend all your money on your guesses or do you wanna spend a little money, learn a little about what that achieves and then go from there. So we definitely wanna make sure that we're in alignment with the vision. We also wanna align on the key goals, overall goal of the project and the key performance indicators, KPIs, also known as KPIs. These can involve people who are on the project but also sometimes people outside of the project team, the immediate stakeholders that you might be working with need to align. I cannot say how many times I've been in a room with people during a discovery. When they feel they've already done all the work of detailing and talking to everybody internally. In the very first day, something comes up, a conversation where it's clear that some key person was left out of the conversation and the database guy or whoever it is and when they come into the room, there's some material input that they have that throws some of the lists completely out of whack and we have to retake the hill sort of. So these are really, key performance indicators are outcomes that are proving the success of the project. So you really wanna nail these down before you start doing anything because if you're doing it right, the MVP approach requires holding each requirement, each feature request up against those KPIs and confirming that it helps achieve them or doesn't and then they should be thrown out. So we worked a lot with poorly defined KPIs. I'm sure everyone here has an example of a project that you've done where somebody didn't quite, wasn't quite clear enough or not all the way clear or once they choose a technology, maybe there's additional details that need to be shared that haven't been and it really takes one little detail sometimes to throw everything out of whack. Just in this example, it's the same faucet, hot and cold, it has a basin to catch the water in, you can mix the temperature, but in this case, if you've ever been to the UK and you've used one of these sinks, you're sort of like freezing cold and then add some boiling splash. So it's much more difficult to use, it's not such a pleasant user experience and yet on some level, it fulfills all the requirements that we're given. It just, once you get the feedback from the user, you might think, oh, maybe let's put this faucet together so it's combining the temperature of the water. So really important to do this, really important to make sure that you're aligned and if a customer comes to you with poorly defined KPIs, explain to them why they need to define these, why they need to have specific measures and metrics so that you can see whether something has been really achieved. Saying something needs like more users or more time on the site, you could literally have one extra user and it fulfills the spirit of the requirement. So you wanna be very specific about 10% more or 1,000 more or whatever that detail might be. Okay, kind of going back to the previous slide, I just wanted to say that the KPI here on the right picture was 10% less burning of the face. You wash your face with the hot. Okay, so key performance indicators, again. They are a set of measurable values that are set to demonstrate the success of a product or a software. And selecting the right KPIs depend on the type of organization you're working with and also depending on which part of that organization, which piece of that organization you're touching. For example, if you're working with the sales department, it could be increased monthly sales growth, monthly margin and things that are sales related. So Jenna and I thought it would be fun to set KPIs for our session here. So the KPIs that we brainstormed on there, we wanna finish on time. We want to engage with you and have your engagement. We wanna have fun presenting and we want you to have fun listening to our presentation. We want to cover the MVP approach prioritization of the features and also provide a project example that views the MVP approach. And want to provide durable opportunities that will support the MVP approach. So these were our KPIs. And these are not the ideal way to have KPIs. So we took a little bit of a different stab at that to improve them so you can see as an example. Yeah, so we kind of broke those down. So the previous screen here, it's just very general. What are we really, what is our goal? What are the metrics? We want attendees so they can be comfortably defining the MVP approach and its value so they can bring it back to their customers, understanding enough so that they can explain it simply to their customers. With everything that you're producing, with every project, there is always a budget and timeline. Our timeline with this presentation is only one hour. We want to make sure that we're delivering our message and giving you what we want to deliver within an hour. We want to have fun presenting. We want to see faces that are happy, that are engaged because if you're working on a project, if you're not having fun, if you're miserable, the project is going to end up miserably. So we want to make sure that that's very important. We want to, we looked at data. We looked at how in public speaking how much people engage with the percentages and the timeline. Our ideal metrics is 100% engagement. So hopefully we'll see a lot of nods and just all of you are having fun. And then we want attendees to understand at least three ways that Drupal can support the MVP approach. And we want attendees to be defined and execute at least two ways to prioritize features requirements and we want to make sure that we review at least one project example that used the MVP approach with you during this session. So after you define your KPIs, you go into writing requirements. Requirements is a capability or a condition that must be met by a service or a contract. When you define requirements, it's very important to understand who you're building this for. Who is this requirement supposed to serve? So for example, you might have a dot requirement to build a dockhouse that has a door and has a roof. But you forgot to indicate which dog is it serving. So in this case, this is built for a very small dog and so it's just servicing the dog that is 50 pounds. So if you define who it's supposed to serve, what kind of dog, what weight, you end up with a happy dog, which could be the user. So in this example, we need the door that fits the height and width of a 30 pound dog and it's made of material that's resistant to water and to snow and to wind. So, okay. So once you have aligned on the vision, aligned on the goals and key performance indicators of the project, you wanna understand who, in terms of prioritizing them, how do you get to, we have 50 requirements, here are KPIs, how do we order these things? So one of the things that needs to be considered is the weight of each of the stakeholders. So how much weight should the IT people's opinions about what the feature should be have compared to the content editors and what the feature should be. So what we look at is who are the actors who are going to be producing the results that are the KPIs and what activities are they gonna be, what are they gonna be participating in that will end up serving those results and so we need to really consider who the people are and what those goals are before determining what the weight is. In a situation like the dog house, the weight of the dog might have the highest, sorry, the opinion of the dog might have the highest weight because it has to reside inside of the dog house. Maybe the owner who has to spend money on it is like, okay, we don't wanna make it out of platinum, so maybe my concerns need to come in there too and so they might have a lower weight and those weights can really dramatically impact the ranking of your features once you get into the process of defining the actual priorities. So it's really important to consider and also to help your team if you're a product owner or you're facilitating, help the team understand why there is a difference between the weights. Sometimes that can create tension and conflict when someone's opinions just come in and override everyone else's because they have more weight but it's really important for people to understand on the team that there's a reason why certain stakeholders have more weight and that that's important because they're more in line to have to act in order to fulfill the KPIs. So that's why the weight difference exists. Not every prioritization process requires weighting but it is a way to consider specific actors and needs and sometimes to not let certain groups override the needs of others. It's a really good way to, if you've encountered situations in which a customer has maybe like a really aggressive IT department who wants to run the show and define all of the rules and the marketing and content editors sort of end up last place or they're sort of like, okay, well, if we can have that, we'll take it. It's a good way to balance the equation a little bit and have the people who maybe, whose needs haven't been traditionally considered have a little bit more stay in the matter. And if I can add, it also creates conversation between different stakeholders because sometimes the stakeholders don't really talk to each other. So when you, it helps them understand the challenges within different departments. Good point, okay. So we wanna score the requirements, not this way but this way and that is each stakeholder looking at each one of the requirements and considering how likely or unlikely is this to support the realization of a KPI? And that's the key difference between haphazardly just doing, being a yes man or yes woman and doing whatever the customer says and actually bringing your expertise and your knowledge to the conversation and helping them make the decisions about what's really gonna actually help them achieve their goals versus what might be noise that might not actually have a material impact on the results. And then the second thing you need to consider is how likely or unlikely is the fulfillment of a particular requirement to be detrimental from realization of a KPI? That could be in terms of doing one thing might put another thing off the list so it's no longer possible to do. It could be that things aren't compatible and it could be that they're here again. It hates us. Our session wants to be visited by alien. I'm not sure where the glitch is so it doesn't seem to be up here. Okay. So this is again where a scoring process can happen so you can have the feature and where it is on a bulleted list, not prioritized. The weight of the actors, the stakeholders and if I'm at a three weight and I rank things one to 10 and then someone else comes along and is a two and a one, they can actually equal my weight if they team up together and decide they want something instead and you can essentially get to a point where you have a forced ranking of the things based on weighted opinions about which features are more important and less important. There are some ways also to prioritize once you get there so you have the weight and you sort of have a sense of what those requirements are. Some alternative ways to do that that aren't weighted which can cause tension honestly so if you have a team where that's gonna cause tension you can use this method, the sort of final four bracket method. If you know how brackets work you don't necessarily have a rough ranking of features and put one against two, three against four, you do one against 16, two against 15 so that the people who've earned the status of number one team throughout the season get the benefit of playing the easiest team for the first round. So you don't wanna pit team one and team two together in the first round, it loses the excitement of that final four. So this is a way to do it, you can do it from different stakeholders and each one comes together and they battle it out on the playing field for those last key features. But this is one way to do it that can work with teams that might have a lot of tension or where there isn't necessarily a clear weighting where it's just like a general service to all the teams and we don't really know how to give a particular team more weight or less weight. Another way that we can do it is where we've done it in workshops and such is to list all the requirements on the wall or in a spreadsheet or whatever and you give each person stickers and you say okay each person in the room who's a stakeholder has 10 stickers and you can put them on whatever features you want and you can put up to three on any one feature if you really, really want that one. But only 10 and that's your limit. That can have benefit and just, there's sort of an egalitarian approach to it, everyone's putting stickers, there's sort of like excitement and movement in the room and it can also be somewhat vague in terms of who said what and who wants what. But if you use colors, then you can say okay the content editors get blue, the IT team gets pink and these other actors get orange and black or what have you and so that way as a product owner or a facilitator you can see here's the rank that we ended up with and this seems to be more important to content editors and IT and this seems to be more important to end users and such and such. So you can kind of balance which features you're going for based on the need to keep the piece if you need to sort of like step slowly toward benefits for each group. It really depends on the directive of the project. You can also prioritize with points, give everybody 50 hash marks and they can go score and draw lines next to the ones that they really care about or you can combine approaches and force rank with numbers and stickers and then do it in the brackets. Whatever you need to do, whatever means to get to a point where everyone agrees on the priorities that you have and what the order of operations is. Some key tips for product owners in this kind of decision making process and especially early on as people are getting used to having their decisions vetoed and their priorities nixed is to as a product owner, be very clear with the team as you're having these conversations. What conversations are you gonna bring to them and you're just really bringing them to let them know what your decision is. They don't really have a material input. Like we're using Drupal or whatever it might be. The next is to, if you're gonna make a decision on their behalf but with their input or if you're gonna be making a decision or letting them make the decision themselves. Usually a really project that's really embraced by the team has a balance of these things. There may be unilateral decisions that you're making but especially when you're making unilateral decisions if you want the team to adopt the application it's a good idea to give them some control about like, hey, even though I dominated 90% of these features unilaterally I'm gonna give you 10% of things like just you want, just you want. Some points that you can put into the project just for your own convenience. All of these tips, alignment on the vision, alignment on the KPIs, ranking, et cetera can be used outside of Drupal. It's really, it's project, it's technology agnostic. Heck, you can do a construction project. But since we're at Drupal.com we wanted to mention a few things that are related to Drupal that really support the approach of a minimum viable product or a minimum lovable experience depending on how you've heard it. One is distributions. So for those new to Drupal, Drupal has lots of distributions from open classroom ones to open atrium for intranets, conference organizing distribution, lightning for just general like Drupal 8 projects start. Starting with those things can be a boon to projects if you're trying to get started quickly, if it's relatively simple or if you share the opinion of this distribution. The thing about distributions is they always have an opinion and it's making some decisions for you as a starting point. But you have to be really clear that those decisions are the same ones you would have made and you're not creating a situation where you have to undo a bunch of stuff. It negates the benefit of a distribution. The second thing is obviously the contributed modules. So if you're working with an MVP you could sit and have a week long conversation with some customers about a calendar and what it needs to do and what color it needs to be and the arrows and how many rows of weeks show and whether it's Sunday to Saturday or Monday to Sunday or they're all these details around a calendar. And then ultimately get to a set of very detailed requirements and then ultimately have to use a contributed module anyway. So why not start with, hey, here's the two modules that we think are pretty good and they work well with the other things that we're planning on using. How about these? Does anything need to change about one of these? Or is this good enough and we can move on to giving you something else of value. And then that also opens up the conversation for, okay, I get that you wanna add alternating colored rows to this calendar but is that really going to help you achieve the KPI better than a calendar without alternating colored rows? If it isn't, then really, it's sometimes a hard conversation because the designer has like in their head they want alternating colored rows but that's the kind of like the little tiny pieces of development where suddenly you don't have to do a custom calendar because you got them to agree on this simple version that's tested that works with other modules and it's one of the powers of Drupal being able to do that stuff quickly. The third thing is the integrated modules. So the ability to not make Drupal have to be a goodle-goodle analytics engine or a search engine or whatever else, a Salesforce CRM. Instead you can integrate and integrate with the best of class tools that you might already be using. That might be a vision of yours to eventually have a CRM that's embedded in your project and you don't wanna use Salesforce. But in the meantime, you can use integrated modules to just get to a place where you're dealing with the most crucial things. If you have a lot of critical requirements and all of them are integration, that can suck up a lot of time and you can do these instead and save time for custom feature development. And the third thing is just the fact that Drupal is easy to use and set up in terms of a basic site. You can get there pretty quickly. In the keynote, they had the little like da-da-da-da-da-da-da-da. Getting to a basic site and you could pretty easily just upload a module, show the customer what that looks like, get them into the tool and start to talk about requirements from there. Kind of a famous place where a lot of people have requirements from my experiences in the editorial backend before they've ever seen Drupal. They have all these requirements that are solving the problems of their old CMS. So you're like, oh, we have to have all these different things. And it's sort of intimidating because it doesn't seem like why do we need to change the entire Drupal administrative interface? Isn't that why you chose it? But a lot of those are coming from the polluted past what they're trying to get away from. And they're sort of like, don't do this to me again type requirements. And really, once you show the customer what it's like and how easy it is to use in some cases and then they abandon all of those requirements and stop trying to fix something that's not broken. Okay, are we having fun yet? All right. Still fun, still fun. That's just one of our KPIs. So to bring more color and more fun to our session, we're using a case study of Instagram from an MVP camera app to the second most important social media app. So I'm pretty sure you all remember all three logos, do you? The one in 2010, you don't remember 2010 logo. So that was the first Instagram, literally like a Polaroid camera. And then 2012, which is a very, very popular and the recent one, recent being three years ago. So basically Instagram was distilled from bourbon. Yeah. Prior to co-founders Kevin Systrom and Mike Krieger came together. Kevin Systrom had just left Nextop, which was a online travel site, which was acquired by Facebook. And then after that, he spent a year at Google. So he left those jobs and he was messing around with a few ideas, different ideas. Back then in 2010, location was really hot. And a year prior to that, Foursquare had launched. So basically checking into places had become a social phenomenon. So he messed around with a few ideas and he created a very simple app. And because he really loved whiskey and a whiskey expert, he named it bourbon. So bourbon was very simple. It had four tabs. You could check in, you could move and you could share photos. Meanwhile, he's to become partner, Mike Krieger had left Meable, another startup, and they came together. So they looked at bourbon and how it was doing. One of the most important things that every organization needs to do is to listen to their users and look at the data, which these co-founders did. They looked at how people were using bourbon and they realized that they were not using it to check in. They were not using it as they were using Foursquare. Instead, they were using it to share photos of their dogs, their kids, the reflection of themselves in the mirror, flowers, food, all that stuff. So basically creating a photo app was a very, very good idea. So they scraped bourbon and they started prototyping and they created another with the precursor to Instagram, which was scotch. But it was very buggy and it didn't have any filters. So they stopped that, but they still, the idea of a photo app, because they saw how that was taken by users, that was still very strong. So they started over and they created Instagram, which was born in November, 2010. So I remember this version and all these filters. So it was a few filters, Xpro2, Lomify, Early Bird 1975, which was my favorite one because it made my pictures look like old times in my childhood pictures. On the first day, 25,000 users. So they did listen, that listening to do the favor to them. January 2011, they added hashtags to help their users be able to find each other's photos easier, one million and counting, they kept growing. February to September, they added a few team members so they had to move to Twitter office in San Francisco. They raised seven million Series A funding because they were growing so fast and they had their user base was growing so rapidly they needed money to be able to support all the bandwidth and hosting of all the images. Number of users, five million and still growing, 150 million shared pictures. Did you know all that? Okay, version 2.0, September, 2011. So that brought life filters, instant tilt shift and high resolution photos, 10 million and still growing. Six months later in April, 2012, they made the app for Android on the first day, one million and 30 million users by then between for both iPhone and Android. So then they wrapped up a 50 million Series B funding days before they were acquired by Facebook for a billion dollars. Then Facebook ate Instagram, but that didn't stop there. So they launched sponsors, sponsored posts in 2013. They added a few more filters and some system upgrades and release in-app features and launch Instagram stories. And number of users by end of 2016, 600 million users. So this is their growth picture. There is not a dull moment here. From when they started in 2010 to 2017, they just kept growing. They didn't launch with all the filters, they didn't launch at once with the stories with sponsors. They launched the first time with a basic photo application with a few filters and they scaled from there. Along the way, they listened to their users and that was the key to their success and they focused on growth and they focused on their users. And of course they're gonna keep growing. The future is really bright for them. Few predictions of future. The stories will be even more important and popular and the ads will become a critical part of any brand marketing and videos, there will be more emphasis on the videos and video content and will continue to roll out new features because you wanna make sure that your product, you're constantly giving updates and something new to your users to create that hook so that they can come back. And shopping options will become even more prominent. I don't know about you, but I don't click on banners for some reason. The Instagram ads, they're done so subtly and so nicely wrapped into the content that your friends share that I've actually bought stuff on Instagram by clicking on the ads. I don't know, you're smiling, maybe you have done it too. So the fun facts of Instagram, a few key takeaways. They kept their app lean. They didn't try to add this and add that. They moved with a laser focused goal. That's really, really important. Having clear KPIs and goals and KPIs that will support those goals. They listened to their users. However, even though they listened to their users, they wanted to stay through to the essential experience of the app. So they also did say no to users. There were requests for certain things that didn't serve their app. So they also said no, but they listened to their users and they updated often. They kept giving users new things and they didn't change much of the app's essential experience. So that goes with every successful MVP. You start small and you solve one problem or you roll out one feature that does that really well and then you scale from there. So it should be clear if you use the MVP approach that you'll all be a billionaire within five years of launching your new app. I'll be back on Facebook. Wouldn't that be nice? Just a finer point on that. I wanted to note that the key elements of Instagram success and of any success if you are taking an MVP approach is to start small and be willing to completely change what you assumed you knew as soon as you listened to that initial data. It's smart to choose features that you're gonna launch with that you actually can measure. So if you put something out there and there's no way for you to actually determine how it's being used or what people like best, then it's not gonna give you any information. So think about if we're gonna do this, then how are we gonna measure it? It should always be if then, if we're gonna develop this feature, then what KPI is it helping us achieve? And really some of those hard conversations are having people let go of the fact that what they assumed is gonna help them achieve something is not actually gonna help them achieve something or might not or doesn't really have any impact whatsoever. Pixel perfection and theming is one of those things where it sometimes feels really critical to people that everything be branded and perfect and like a magazine and it can kill you as far as like making something useful for your users that really matters to them and is impactful in their life. Pixels are rarely impactful in people's lives. So we're gonna cover next an example of an association magazine. This is an online magazine for a large association that is member-driven and their build and migration into Drupal 8. So the backstory of that was that we were involved that this is not Instagram, this is a project we actually worked on. So it's an online publication with a variety of different viewers. Viewers mostly families who love travel and who love food and they were built on Drupal 6 which had already introduced a lot of challenges for their content editors and they didn't have a lot of traffic. They were comparing themselves with another publication magazine website which was done really well and they were getting a lot more traffic. So and then the content editors were not able to roll out new content as often. So as you see, they came to us with a lot of problems and challenges with a very small timeline and budget. So we had to really connect to their vision and understand their problems in order to be able to help them. And also by doing that, when you connect to your customer's vision and really show them that you understand their problems, that creates trust and that trust helps the path of helping them and negotiating MVPs and features and makes it a lot smoother. So with our expertise and Drupal knowledge, we were able to go through their KPIs and requirements and really consult them on what is the most, what are the most important challenges and problems that they want to solve within the timeline and the budget that they have. It's really easy when customers have a lot of business issues because in this example, they were not getting any revenue. Their only source of revenue was ads. And if you're not getting traffic to your website, nobody's clicking on the ads. So you're not getting any revenue. It's really easy to fall into an emotional state and think that everything is important. We need to add this, we need to have video to do, we need a video to do this, we need a carousel to do that. So it's really important to lead them and really help them take a step back and understand what is the most important thing they need to tackle first because if you focus on the less important stuff, you're basically losing time and money and then somebody else might come in and do something better than your website. So you really missed a mark. So basically the things that they came to us with, they wanted a modernized look and feel. They wanted a content strategy for relevant content so that they could engage their visitors and improving sites ranking in SEO. They wanted faceted search to improve the site's searchability and improving their authoring experience. So this was the KPI they gave us. It was literally just do something and fix it. And they wanted more traffic, more return visitors, faster form creation and make the site searchable. Just make it awesome. But more and make it better, those are not metrics. Those are not measurable. So we work with them and we arrived at more defined KPIs. This can be challenging to arrive at. You have to find something that's achievable, that's not overpitching what's possible within a time period for the project to return some results. What we came up with here is a 20% improvement in latency on page creation. And this is really the act of filling in a forum and saving it and getting to the website. They wanted to improve the speed of that because it was quite slow. The ability of an editor to complete a content form in five minutes. This was given by the Drupal 6 version of the site, which was really complicated. A lot of labels were missing from the fields. It was hard to train people to use it because it was unclear and you sort of had to have this insider knowledge already to use it well. And so adding labels to the fields and things like that can help it. Putting things in the right order, that kind of thing. And then having the content structured increasing the site traffic by 50%. This is one of those that obviously doing something to make a new website is gonna have an impact on that. But it might have other, there might be other actors within the organization that are really in play at, whether the site traffic is increasing by 50%. Obviously better content will help. But if they're not doing other advertising or other activities to drive traffic to it to learn that there's better content now, then you might not be able to achieve that. So some KPIs actually you'd have to check with other teams and make sure that that's an achievable type of goal. Increasing return visits by 30%. That's also again, if people are getting served by the content and there's a reason for them to return, they'll come back. And then increasing such, optimization by using best practices in content production. This was more of a general requirement. SEO is sort of voodoo, it's changing a lot all the time and it's an imprecise science in terms of like, how can I get my page to number one? And so really they were about adopting best practices within the content forms like alternate text and things like that that were gonna help naturally sort of improve their page ranking. And also Drupal 8 actually, the out of the box was able to solve some of these problems really easily. Just using the best practices for SEO and just changing from Drupal 6 to 8 already can address the authoring experience. So. So we came into this project with a giant list of requirements that weren't necessarily prioritized. They were just everything is required. This is kind of evidence for a customer that is not taking a data driven approach to their needs. And so if you have a project starting this way, that's often the first thing you need to say is like, why are these requirements? What brought these things to the list and who put them there and why? Why do you think that's gonna help? So really just asking questions about how these things ended up on the list is really really important. So we really had all these things, they were specific features, specific requirements. What we did first was just to look at impact, high impact, low impact, or whether the user expected them or didn't expect them. So it was like a pleasant surprise or table stakes. And we worked with them to really for each requirement and it can be tedious and time consuming, but you can start with a small list, get them going and then leave them in the room to do it to finish the job themselves if you're good at it. But essentially having them look at each requirement talk about where is this? Is this a high impact requirement for your end users? Is it likely to make a big difference in the engagement that audiences have in the site? Or is it low requirement, low impact? And is it something that people would be totally surprised by or maybe unpleasantly surprised by? Or is it something that's expected? Like we have to do it because it's table stakes. So high impact things are table stakes, low impact things are like who cares about this thing? Why do I need this? If you have 10 pieces of content, do you need search? Probably not, everything's findable. So really getting, forcing people to have these conversations and then also looking at some of the features and taking specific things from that list and determining whether does Drupal out of the box already improve this or do we need to really add more? Could we try Drupal out of the box and see if that does enough to make an impact so that we don't have to do these other five things? So it's really having these conversations, breaking it down into detail, sort of forcing people to come up with a, why, which KPI is this helping realize? Why did this end up on the list is really important. Then looking at the weight in this particular case, they're looking for end user results, but the end user results weren't really in the area of functionality so much as they were in the editorial experience and that they could get content up there that was new and fresh. When you have an association magazine for travelers and everything's stale and seven months old because it takes a developer to put content on the site, it's, that's the problem. It's not necessarily that the users are having a problem getting to the content or finding it. So in this case we weighted the content editor as number three, three times the weight of a business user, in terms of the site user they were a two and internal stakeholders were a one. They just needed the data coming out of the site to really see whether it was making an impact. And what we came to was from the 51 requirements that were so small you couldn't really see them for reasons of what was happening. I don't know what happened here. I don't know if that's a fade or what that. Something went wrong with everything. They're here. I swear it's Poltergeist in the PowerPoint. What this is is it's basically a snapshot of a spreadsheet that we use to rank. So we have all the features listed on the 51 items down the left side. There is a column for each of the stakeholders. So content editors, business users and users. Each column for a user is split into two columns. One is the impact of the KPI actually being realized. The impact on the KPIs of it being developed and the detriment, so the relative detriment. And then that actually is calculated out times the weight and that gives you a forced rank. So that will actually give you, if this were working, it gives you a different picture completely. So you'll just see it very quickly. That SEO list as it's coming up, the things that are highlighted in yellow are the things that we actually developed. And you'll see that there are things on here that we did not even do because in the conversations and in the re-prioritization of all these things, they were determined to not be something that, this is so annoying, to not be something that was gonna have a really serious material impact or that it wasn't worth doing it until we could listen and see the right way to do that thing. So if it were something like we need a 70 slide carousel on the front page so that all of our content can be highlighted, that the kind of thing where if you just bring in information from research that's done, you can tell them like, hey actually, having a bunch of slides on a carousel does not increase the search engine optimization usually users don't click past that first slide and so all the other 63 to 70, those items don't really matter. So we ended up building a successful, well-used application that had some really great results, not necessarily the 50% of user improvement that they were hoping for, but that might not have been the best, most appropriate goal from where they were to where they wanted to end up in a short time. But ultimately they have budget left after the main features were developed to continue to listen and adjust and adopt based on what their users were experiencing. We'll fix that when we share this so that it's actually appearing in there if you wanna look at it. Okay, so just a summary of what did we learn. It's really important to have a clear set of project goals with every project. If your customer comes to you and they don't have specific goals, you can help them, you can really ask the right questions and arrive at those clear goals. Identify those KPIs that are in support of those business goals. So there are always, you can come up with great KPIs, but if you're not hitting those goals, if they're not supporting the main business objectives, they could be de-prioritized. It's important to, just because they're KPIs doesn't mean that they don't need to get prioritized. KPIs also need to be prioritized. Tie all choices to a result and focus on the main problems and address those first. You can always grow from there. It doesn't have to be a problem. It could be a goal, it could be a thing or functionality or solving a problem or social problem or anything. As long as you solve that problem really well and focus on that, you can always grow from there. And it also helps you plan for future development. It also helps you not burn your time and money and resources on things that didn't matter right now. You can always, you can plan for adding later. And starting small, build, test, launch, listen. Listening is really important. Also, when you start small, it gives you the opportunity to listen to the users and learn from there. Maybe that's not how they want it to be done or maybe they need this. So your goals could actually change or your features could change depending on what you're hearing from your users. So that's really important. You can always repeat. Build, test, launch, listen, and repeat. And that's how Instagram did it. They listened, they tested and they continued to add more stuff. Leverage Drupal Core because Drupal Core gives you a lot of features and you don't have to customize everything. So find out what can be achieved through Drupal Core before you talk about customization. Distributions and contributed modules can also serve this purpose. So we're gonna move to questions. And number one questions, did we meet our KPIs for this session? Thank you. Thank you. Plus the added bonus of the extracurricular, terrestrial visit. That was a bonus. Do people have questions? Yeah. You know, I think one thing that you use for your live is something that wasn't a problem for Instagram, but like, you know, the necessity of like they get a really positive first impression and like the fear that if people aren't really impressed the first time, they use your product or visit your website, that you know, they won't return or you know, you all the harder time getting them back. I think you guys address, you know, things like that in the phone client. Do you want me to? You can. So that goes back to solving a problem or building a feature that does it really well. You want it, that's why you have to focus on small. If you try to do so many things at the same time, then you could actually fail because you're doing haphazardly in each of them. So when you do something really well, your users aren't gonna like it. If it's not buggy, if it's not breaking, you will keep your users. And then from there, you can listen and see how they react. There is always a risk. They might not like it. So like Jen mentioned, you should be open to changing course. Sometimes you have to listen to what data tells you and then change course from there. Also sometimes this is where sort of beta or you know, alpha releases come in taking a subset of users and having them actually test it. Then you can see how it's gonna really land and very early on make adjustments based on where people are not adopting it or not impressed and go from there. So it depends on the situation obviously, but I know branding is obviously really important and that's where that pixel perfection that I call suckable frowns about it back there. Sometimes that's like top of mind is like do people recognize it's ours and if they don't, it's a big problem. So you can even boil that down to the most critical thing, the logo, the color scheme. Those kinds of things can be recognizable. Like certain customers, you're like, if I see that color, I know it's this company or if I see that shape square, I know it's the gap or whatever it is. So even that can be like minimum viable branding. What does that look like for this company? That's definitely a conversation I've had. Minimum viable interactivity. Sometimes people's brand is the fact that there's this interaction happening with the customer and you have to preserve that stuff, but it's just in balance with everything else. So definitely talk about it, but that's definitely a challenge and there's a couple ways. Yeah, that's a really common problem and a really good question. So from my experience, both at my previous employment and at Acquia, usually a pre-sales or sales process involves some conversation about that and often it's just enough to let the customer know that we might know something that they didn't think about and then they're like, oh, I didn't really think about that. Maybe I really should talk to you guys more about discovery. So for us, really a detailed discovery is important when we typically give a pitch for a project. It's got a pretty big range on it and that's because all we know is what you included in your RFP. Some groups are very good at writing RFPs that have all the details you need and some are legendarily poor at it and so you need to know who you're working with and how likely it is that they nailed it. But also really demonstrate your expertise. I can have a conversation with someone about an RFP and in five minutes rip holes in it usually and be like, these are the questions we have, the margins full, we don't know enough yet. As I mentioned earlier, there's always things like every type of project I've ever worked on. Almost always people think they know everything about it that they need because they know what they knew they needed but they didn't know what they didn't know they needed and so the conversation's always getting more detailed and more mature as you go. A customer that you wanna work with will understand that and if they don't get that, then you probably don't wanna risk working with them in the first place. And one more thing is those conversations you have before the project starts but not necessarily sometimes you have those conversations during grooming. So you have certain degrees okay to have those conversations. So you have to always build in some time for those possible kind of backhand forts into your budget, building enough extra budget and time kind of some kind of padding. And also be ready to have those conversations even during grooming because something might change or the designers might give you designs that are introducing all these new cool stuff but that are not necessary. So always be prepared for those conversations but the ideal situation is that those are all resolved before you start on a project. And your second question? Yeah, it is a fine line because I think, I mean for me personally, when I'm entering into a thing that I'm new at I kind of like my fear is looking stupid. And I think if you can delicately handle the question asking and not make it that they feel stupid for not knowing but it's totally understandable that you don't know this. Nobody ever knows this when they start and really kind of temper the fact that you're asking the questions with that kind of stuff. Then they can understand why you're asking them. From my experience I've worked on a lot of enterprise application development projects from like weather channel down to like tiny little organization. One thing that's very common is the CMO or the CTO or whoever that person is in the role of being the product owner or managing working with you. In their role like replacing their CMS is not often something that they might have ever done. It's not something they're doing every year in their role at IBM. It's not something that every organization does every five years. And so they might be an awesome CMO with lots of experience in marketing and like really, really generating effectiveness and results but they've never done a CMS replacement. Why would they know how to do that? It's perfectly reasonable but they'd be completely ignorant about what they need to know. So really like meet them there and say, hey, that's perfectly understandable that you don't know this stuff. Why would you if you've never replaced a CMS? Sometimes people appoint a project manager at the customer with us and it's sort of like when we start working with them we understand like they're used to launching a new product to the website not like a whole website project. They don't know how to do this. And so we have a lot of instructional materials that we use to help the project manager understand like here's what we're taking on here. If you're only 25% allocated we need to talk to your boss and get you fully allocated. It's gonna be a little bit longer a row to hoe than you're used to. And really just understand that this is new to them. It's a totally different world, CMS and big application development and they are new to it and it makes sense that they don't know what they don't know. At the stage where you're understanding the weight of the stakeholders and I have a feeling that the peril in this stage might be greater for in-house agencies or in-house... Oh yeah, because you're breaking your friend's hearts then. Is that a conversation that should be had by closed doors or with the stakeholders themselves? From my experience it's usually with the leaders of the stakeholders like the content editing department and the marketing department and the IT department that you need to understand this is for all the teams and help them manage the people who are making the demands. Otherwise you can end up with a lot of upset and people feeling slighted or hey, I've been complaining about this for two years and I didn't get it in this latest replacement of the CMS or whatever it might be. So really just letting people first talking to each other and discovery a lot of times they're just learning each other's pain and so it's like, oh, I used to just be like you don't need that and now I understand why you're asking for it. It takes you 20 minutes to publish a page on the site. And so that kind of conversation if they're all in the room, if they're all separate they never hear that from each other but if they're all in the room they start to get each other's pain a little bit and it can make that pill a little easier to swallow. And they can make that decision on the weights after they hear each other's pains. Oh, that is a good point. You can leave it to them. Like who's opinion should we count more here? Like who's actually material and the results that we're trying to generate? Is it the content editor and it's their job and most of the things we're needing are on them to produce and if so it makes sense that they have more weight.