 Well, first of all, thanks for coming to this panel. I was realizing before, I think, this might be the first in-person presenting experience, so hopefully we'll not have any nerves, but definitely with a smaller group. We have plenty of stuff to present, but hopefully it also can be moments to pause for conversation and questions as we go. So I will quickly kick us off. So first of all, actually, the topic, just to make sure y'all are in the right place, obviously, we're here to talk about continuous optimization and consensus building and approach to web and platform work, and so we'll do a quick intro of ourselves. I'll start off. I'm Katie Jamison. I am a senior VP at Blue State, and I'm gonna hand it over to Taylor for quick intro, and we'll talk about where we work. Hi, everyone. My name is Taylor Darnell. I'm a senior website and marketing manager at MSFUSA. I'm probably not the only one whose haircut has changed dramatically over the last two years, but we are here to talk about a different sort of change management, and I did plan that joke two days ago. So what is MSF? Well, it's an international organization that provides critical medical humanitarian aid around the world wherever it's needed most, whether it is epidemics, pandemics, conflict zones, natural disasters. What we're trying to do as an organization, obviously, is save lives, and are often the first people into really dire situations to provide that care. We want to stress always that it's independent, impartial, something that is independently funded, doesn't operate based on the mandates or approvals of governments and governing bodies. Yeah, and it's been around for 50 years if I hadn't mentioned since 1971, and I'll get into a little bit later, kind of the specifics of what we're talking about when it comes to the web property, doctorswithoutborders.org, there is kind of a different relationship or it's not as direct as, you know, we worked on a site for all of MSF, but I'll get into that in a second. And I work for Blue State. So I have a colleague of ours over here, Sam. We presented in the same room a few days ago. Blue State is a digital agency. We do work primarily in the nonprofit and advocacy sectors. And we'll talk about this in a second. We've worked with MSF for many, many years, and Blue State is an affirm that basically does work across the spectrum, including web platform design, but then also runs full service campaigns, spanning email, social for a number of nonprofits for fundraising, advocacy, and brand engagement. So we have a pretty strong and robust digital platform practice, which is actually led up by my colleague Sam over here in the front. But we also have a unique POV where we're not just looking at that channel alone, we're really looking at the full suite of things, which I think is a good segue to some of the ways that we've approached thinking about work in partnership with MSF. So I'll kick us off. So just in terms of, you know, the topic at hand, you know, part of what we're really wanting to talk about in this is, you know, a lot of the panels that I know have been central to the programming of DrupalCon are focusing around tools, technology, all the specifics of how we're approaching major rebuilds, projects of that nature. And, you know, a bit of what we're wanting to talk about today is actually beyond just the specifics of tools and solutions to all of the fun change management, process management that really is, I think, the hardest part of our jobs, oftentimes in navigating large scale projects and redesigns. And so I wanna talk a little bit about kind of what I see as the state of the, you know, dot org redesign is what we're saying. And again, we kind of have a little bit of a focus in leaning into nonprofit sector, of course, given the specific use case of MSF. So what I like to talk about is, all right, we're all in this, you know, ideal mindset before we embark upon a major redesign or project for our organizations. And we kind of begin with that ideal. So it's the ideal, it's a user first mindset. We're gonna approach it with the best in class design and tech solutions. We're gonna have empowered lean teams, agile data driven approach, run by those with real expertise, well-budgeted and of course, optimized over time. In the reality, I think for most of us who've been a part of website redesigns of which I have spent about the past 15 years playing many roles and from project management to the kind of senior relationship managing with our clients is obviously a very, very different story, which is lots of contradictory opinions, little process or empowerment for brokering consensus. Stakeholders rarely looped in as consistently as we'd like, probably under budgeted more than it should be relative to other sectors of the organization. And then the most critical point is, you know, we oftentimes look at these major overhauls of web and platform experiences for our clients on, let's say every three to five year cycle tied to some big RFP or it's kind of like your one shot to try and really approach how to improve the experience. And then the way I like to call it, it's sort of like the, you know, hot water heater or, you know, repair the roof. It's like you do it once and then you just, the organization overall is not thinking about the continual investment and maintenance of it. It's kind of a one time thing that you do every three to five years. And then it just really doesn't have the kind of central approach to iteration and optimization over time that ideally it really should. So what that leads to is what I call the bento box effect where you have a really complex organization and everyone on that website gets their own little square. It's sort of a consensus building exercise of how do you, how does everyone cram in that one thing they need every three to five years? And so I think we all know a lot of websites that look like that, a million little boxes and every team has their own little portion of it. And that's just a natural reflection of sort of the complexity of consensus management and stakeholder management in big projects like this. So, you know, in essence, what these cycles of redesigns look like is sort of like this. Every three year cycle or so, big website design project, website implemented, then a cycle of neglect. And then we gear up again and do the whole thing over again and then we slide back down into that kind of cycle of neglect. And it just, you know, we keep doing this and needless to say to all the people in the room that's preaching to the choir, we all know that this is a just, you know, crazy way to think about this central piece of the experience for our clients overall. So, Blue State's mindset is really, you know, how can we go from this, which is frankly the reality for most organizations, to this, which is not getting away from still doing major overhauls and redesigns every, you know, every so often. But how do you kind of build in more of a continuous cycle of iteration and optimization in the chapters of time in between? That is a major mindset shift. It's a major organizational shift, process shift. And then also a budgeting shift too. I think that's a big, big part of why we are where we are with these things only being approached every so often is oftentimes a financial reality of that too. So we'll touch on all of those things about how we work with clients like MSF to try and shift that mindset and frankly approach the website in the same way that we would email programs, social, paid media where endless amounts of resources and time often are allocated to those areas. And like we'd never dream of launching an email program and not interrogating and optimizing and testing that continuously. And the question really is why wouldn't we be building a culture of understanding of our websites oftentimes, you know, our websites often being as central to the outcomes for organizations as any of those other channels are. So with that, I'm gonna hand it over to Taylor again to talk specifically about how we did that in partnership with MSF USA. Thanks. So yeah, a lot of what I'll try and do is use our project as just a way of kind of building this example or drawing these connections. Like how do you kind of socialize a user first sort of mentality across stakeholders? What's the reason that you're doing that? There are so many things I could talk about. There's only so much time, but I really wanna leave space for a Q and A. This can be an open thing. I wanna hear about your experiences too. Any questions? You can share any challenges that you've had about how you manage stakeholders, how you need to try and build consensus in your organization. We can commiserate a little bit if we have to. But okay, I'll dive right in. First, what are we talking about? Who is MSF USA? Well, MSF, international movement, we often say movement, so I'll be using that for the organization shorthand, has 25 associations. MSF USA is one. And while I think they're, before my time, I've been in my role for almost two years, there was some consideration or inquiry into whether we'd want to unify all of the web properties for these associations, use a shared framework. As it stands, these are separate entities on the website in their own governance, on their own stacks, et cetera. So we are talking about doctorswithoutborders.org, which is the web presence for MSF USA specifically. You know, while all the funds we raise can go anywhere internationally, we are communicating primarily to a U.S. audience. We are fundraising primarily from a U.S. audience. We are recruiting primarily from a U.S. audience on doctorswithoutborders.org. So I don't have the whole tail of the tape, but I think the web presence for the organization is mostly mirroring how the fundraising is working as it currently stands. But who knows where we'll go in the future, especially in learning all that we did about the things upcoming in Drupal. So what's the website for doctorswithoutborders.org? It's for a bunch of things. And the governance of it is similarly, you know, spread across many different teams. On the one hand, there are lots of communications needs and goals. You know, it's spreading awareness about what's happening in the movement, what's happening in operations, what we're doing, bearing witness, you know, saying what we see in the field impartially, holding entities accountable. This is a part of our charter, something we need to do. And so in that way, you know, the publishing that we're doing on the website is actually a critical function of what the organization exists to do. Similarly, you know, all going hand in hand, reporting on the operations to our U.S. audience. And also I wanted to include this, you know, holding the movement accountable as well. It's a space where we want to ongoingly publish, you know, things about self-inquiry, being critical of ourselves, making sure that that conversation is out there too is something that we want to have the space for on our website. So these are communications kind of pillars or goals. Development, not software development, but of course fundraising, I'm sure you're all familiar, is another critical need for the website. It's raising funds for the medical humanitarian aid that is being delivered around the world. It is providing resources to the donors, you know, that specifically the development audience, potential and existing donors may need documentation, instruction, support, and then also cultivation for some of those kind of one-to-one relationships we have with major givers, those sorts of things. The website is often used as a step in that process for kind of one-to-one communication. Field HR is another critical need, while over 80% of MSF staff are actually hired locally in the countries where they work. This is kind of a misconception that we want to do away with, you know, that everyone that works for MSF is a volunteer who's flown from a Western country elsewhere. That is not the case, although it is kind of a prevailing misconception, but I want to note that here. There is still though a need for us to recruit international staff to fill any gaps that we might have in the operations where we're working. So it is still a critical need for the site, of course, to be getting recruits who are paid staff, by the way. And there's a whole lot of other needs. Domestic HR, meaning recruiting and hiring for the New York office specifically. We have events teams, student chapters, and the people who manage those relationships. Advocacy, programs and operations, a medical team that actually publishes research. And as an internal resource, this is something I was actually a little surprised by when I started the role. You know, the website is constantly used as like the record of truth internally as well for people to understand the history of what we've done in areas, to understand our stances on sometimes complicated subjects. So this is all to say that there's a lot of things the website has to do, and governance then is spread apart in many different ways. Certain teams own certain pages. And the one thing I wanted to note too is that there is a very cherished culture of debate in the organization. We don't want to mince any words when we're debating things, when we're trying to figure out what the right approach is. But obviously when you're working on a website or an application, sometimes debate is a little scary. It can be a hard thing to manage. And I bet you all have some experience in that. So when we started, you know, we thought that we would focus on experiments, you know, in kind of smaller areas of the website to raise conversions, that being the goal from the start. But we really thought that after, you know, talking to all of our stakeholders and honestly just going with our gut and seeing what the website was, that none of these pillars were kind of being delivered on. We weren't meeting their goals. By trying to do so many things, nothing was quite sufficient across all those different pillars or teams. So instead, we pivoted to build a better UX and UI foundation to make a better communications experience, to make a better development experience, feel HR, ultimately to make a better visitor experience. And this is the thing we really wanted to socialize and have people understand at every level of the organization is that this was what we wanted to accomplish by embarking on this project. And that segues into how you can undertake change management. We wanted to align on objectives and KPIs ongoingly to be sure that everyone understood why we're doing what we're doing. We wanted to ensure that the highest level stakeholders are aligned on goals. And of course, you wanna go through that debate upfront because you'd much rather surface and resolve misalignments early than see the symptoms of that later on when it actually turns into scope creep, more expenses, hard pivots. So while it may sound like very general to say we want to make the website a better visitor experience, you need to have goals that match your stakeholder's place in your organization, match the level that they operate on. Because something as general as that can have real meaning to someone who's not so close to the project. And there are KPIs, accordingly will be some of the highest level ones, overall revenue, paid views, things like that. But this needs to ladder down through the organization. When you get to the specific teams that have governance over specific pages, they're going to have narrower, more specific goals in KPIs too. And you want those all socialized and you want them to kind of ladder up into each other or be like Russian dolls where if you have any sort of debate or disagreement at one of the lower levels, you know you can lean on the people one layer up to help resolve and arbitrate. But having that, the goals confirmed goes very far, obviously. Kind of what I was saying, once you're aligned on goals you need to understand not just who like the ultimate decision makers are if you do need to escalate up through the organization, but you want to identify more specific supporting roles. Who's informed, who's consulted. Many stakeholders can be involved as subject matter experts, participants in surveys, reviewers of design and everything if those lines of arbitration are clear. But hopefully through some of the things I'll share and as the next note, like you don't need too often to resort to that sort of escalation. Because this third point is maybe the most important one to me at least or where I saw kind of the most payoff in how we manage this project is that we wanted to really socialize this notion of being user centric or putting users and visitors first. So I said, socialize user first consensus building, like make it known that this is kind of your decider. The ultimate guide in swing boat, you know, is what your audience wants. And that you can really manage stakeholders and keep them along for the ride and make them have buy-in in your project. If you're engaging them by educating them about the decisions that you're making, educating them about the techniques you're using, the conventions in the industry or space and what you're doing, that's a way of keeping your stakeholders involved. And frankly, it's a way of priming them to the work that you're going to do and the solutions that you're going to deliver. I think that if we had reached and finished the project exactly as it is now having done it without involving stakeholders, if people saw it today, they wouldn't be happy with it because they weren't involved. They might see it and say, like, where did this come from? But they were involved. We socialized the practice and everything that we're doing. And I think that's why people are so happy with it now. So one note though, is that we wanted to build the project or structure everything around this notion, this goal of having the space and room to take stakeholders along for the ride. And so the way that we structured the contract was through an annual retainer for a website evolution with a bucket of hours and no fixed deliverables upfront. There are some other things too that we did, such as kind of decoupling design sprints from development sprints. We wanted there to be a healthy amount of space between when we design a component and when we actually developed it because by having discovery with so many different stakeholders through the whole project, we knew we'd want to revisit a component's design. We would surface and discover more use cases than one teams. And so by spacing things out, when we were developing them, we didn't have to make hard pivots. We were able just to expand or create more flexibility in the designs before we made too hard of a commitment. And I should say too, maybe this is better for the end, but a lot of this too, I think is something that you can do as an internal stakeholder. It didn't require me working directly with the blue state team to do a lot of this work internally and taking them along for the ride. So I see it as just wanting to have this space in your role in the empowerment, the time to do all this people work, frankly, that can help you have success in your project. All right, so I'll fly through these. Some quick examples, I think we're doing okay on time actually, but some examples of specific things we changed on the website and how this kind of ties back into the socialization, the stakeholder work that we were doing. So first, the navigation and information architecture, the deep bento boxing, as delicious as those look. So this is the old site. This is the old homepage and a snapshot of the navigation. It's a lot of menu items. And we kind of knew from the get go that this put a big burden on the visitors to the website that if they were looking for some specific content, they had to parse through a huge number of items to try and find it. It's a similar thing underneath all of these different main labels. Sometimes there are even, you know, sub-sub menus appearing afterwards. And it reflects that, you know, dispersed kind of governance of the website, you know. There are so many different teams that knew they had to have their content on the website. And instead of thinking, where does it fit? It's just make a page, add it to the menu. But it doesn't lead to a very navigable or intuitive experience for the users on the website. Some things too semantically, you know, are just kind of tricky. You see, there's links for support us, take action and donate. But what is the difference between supporting us and taking an action is donating not both? You get the picture. So what we landed on, if we look at a snapshot of kind of our new homepage and navigation, something has way fewer items. We wanted to introduce things like labels, you know, contextual clues to increase the information sent, you know, give people clues about the content they'll find in these subsections. To make it, you know, just a simpler thing to glean. You know, people don't have to try going into so many different pages to find what they're looking for. The thing I really want to call out though from kind of like the stakeholder management perspective is that we cut a lot of content or we consolidated it, we moved it into other pages. Like, there is no longer that designated space for press room, for a video page, for the research. They were reorganized, some of it was cut, but a lot of it was moved to different areas so that we have just clearer lanes for users to navigate through. Like clearer user journeys where we're actually building a path and sequence to the content that we're organizing. And we were only able to do that by getting that buy-in on the goals that we had for the site on that visitor-centric kind of vision. That was the thing that made people okay with hard choices of cutting or parsing down content or moving it or inter-tangling it with another team's content. It was all for the sake of making that clear user journey for visitors. A specific example too in how we wanted to socialize UX best practices to reach a better end. You know, in that example of the previous main navigation, there was a very prominent careers link in the main navigation. And the field HR team and domestic HR teams, when they saw that we had removed that from the main navigation, were understandably concerned that this would lead to fewer visits onto their pages, fewer views. But, you know, it is a convention we were able to show this to our stakeholders that across pretty much every industry now on the worldwide web, you'll find, you know, links to recruitment or jobs in the footer and sometimes in this top right kind of, or top left like eyebrow position. And it was after they understood the conventions and best practices that they were okay with us doing this. But we had plenty of time, you know, to take people along for the ride and really teach people the reasons behind the choices we were making. And just a quick note, you know, we actually did user testing to validate the new approach. We asked users, you know, where would you find this piece of content, compared it between the old navigation and the new and got really tangible data that showed the improvements that we could expect. And we did make alterations to our proposal, you know, based on that. Okay, another thing, critical homepage content. I'll fly through this. The old homepage had a lot of news and stories content which is critical, something that we want to include and have kept. But you can imagine, you know, stakeholders within the development department, we'll see that this is lacking something that has like a clear appeal or fundraising ask. And the changes that we made, I think, produced a lot of win-wins. And obviously that's a great thing to go after when you're managing a project like this is surfacing what can be shared wins across different teams. When we're thinking about what are the gaps here though, what was lacking on that homepage? Well, we actually had evidence that showed this is the internal site search feature that people could not find a mission statement. You know, for someone that's unfamiliar with what our organization does, this is of course critical and something we want to introduce. It's something that is a shared win across all departments if we can actually get people understanding of the work that we do. This is also reflected of course in the data we have in Google searches and everything. So if you scroll below the large image at the top of our new homepage, this is the new component that we introduced, something that actually has a clear statement about what we do, links people into upper funnel content, what we do. So already kind of building in more paths for users that are clear. And it introduces impact statistics and statistics around how we use funds that really is a shared win across both departments. You know, impact statistics being something that is the communications department really cares about but how we use funds, we know potential donors need to see that information when they're considering whether or not to support us. So leaner content and new calls to action. This is a big one. If we look at an example of a old country page, here's the country page for Syria. After the large image up top, you get into a lot of content. So if you scroll down a little further, you'd see something like this. And you could actually scroll through multiple viewports or full pages worth of scroll and just continue to see something like this. It's a lot of text because we have a lot we need to share and be very specific about what's happening in these different environments. If you reach the very bottom of the page after, again, multiple viewports worth of scroll there, here's something that would stand out to a development stakeholder as being lacking. This call to action is just links, hyperlinks added to the bottom of the page that obviously is not going to stand out very strongly to someone visiting the page and engaging with the content. And sure enough, this helped us get buy-in from our communication stakeholders to make these changes. If we looked at like scroll mapping, we could see that the vast majority of users on the site never actually reached those CTAs. And one thing I should tie back to when it comes to the shared goals and how you need that to help arbitrate the work once you get into the specifics. Up front, we had asked all of our stakeholders to confirm for us what are the main things this website has to do. And the communications department, field HR department, let alone development all agreed that raising funds is the most critical thing this website must exist to do. We need those funds obviously to do what we do out in the field and provide the life-saving care that must be delivered. So because we aligned on those goals up front, it made it much easier to harken back to that, get people to agree and confirm have buy-in and the changes that we'd make to this page. So a quick look at that. This is what you'd see kind of the top of the page. You notice wanting to make things kind of more graphical and stand out, introducing styles that are different from other content types on the website that you still have kind of the same pattern. But then again, it's kind of like a shared win is we found ways to address the needs of multiple personas coming to the website. So the development persona is often someone who's visiting the website who came from an email or who might have came from a social network. They may be in a mindset that is more operating on a quick kind of scan level. They want to see the summary information and get through the page. They may already feel compelled to donate after getting kind of a high level synopsis. But for that communications persona, we also need to address the needs of a visitor who wants the entire story, who does want to understand all the nuances of what we're doing. So a component like this is a win for both teams because it can shorten that scroll, make things more scannable, but we don't actually have to parse down the story and the needs of those stakeholders in the communications department. I'll say too that all these components have many use cases across the whole website. This is not something that we designed to operate and work only on our country pages, including that kind of image model below that has the statistics next to it. These all of use cases have crossed both departments in many areas of the website. So because we had stretched out discovery, we were able to deliver or design UX and visual design components that satisfied many, many use cases. It just made it an efficient use of our time and unlocked lots of content work later on. At the bottom of the page, we introduced a new call to action kind of system. And the thing I'd call out here is that this is one of the hard things for communications to accept sometimes. The thing that we're always hesitant to do is to place CTAs that could infer restricted giving, meaning we don't want someone on the Siri page who makes the donation to think that their funds are going directly to Syria. That would be, you know, we don't want to accidentally make the wrong impression when these are unrestricted funds that we can use anywhere in the movement. So it was through educating our stakeholders and kind of the techniques we can use in visual design where we got buy-in and people to accept this kind of new design where we use the visual elements to really separate it from the body of the text on our different article pages, on country pages. And that got people really accepting and on board with what I think is a much more kind of dynamic call to action that can really stand out to users and make a good impression. And one thing I'll note is that the vast majority of content was actually left alone. This was something that we decided to do pretty early up front as a way of streamlining a lot of the work that we were doing. You know, we really wanted to focus on the information architecture, giving ourselves a strong new foundation, organizing the content we had, knowing that so many people can get hung up on the messaging, on the words on a page. We really wanted to focus on kind of the system that we were building, and that has unlocked lots more optimization for us to do on the content side. So what's next? Like I said, you know, components were designed to be flexible, so now we can have a parallel work stream that works in content with the new components that we've built unlocking a ton of new things that we can do. And now that we have what I'd call, you know, a tightened information architecture, you know, it gives us kind of constraints to work in. People are thinking about how can they collaborate to improve on the areas or the subsections we have instead of thinking, I need content, so I'm gonna make a new page, and I'm gonna add it to the menu. And I'd be remiss if I didn't know that, you know, content work doesn't require development time usually, and that can be a great way of getting wins with stakeholders. Now that we have this kind of new baseline, we want to measure the effects of what we've changed. We want to understand what kind of the new performance levels are across the website, and react accordingly, you know, with new hypotheses, new tests to run, and ultimately changes to make. But that's all simpler now that we have better defined user journeys on the site. And again, the ongoing, you know, socialization of techniques, best practices, and this user-first ethos. You know, everything can be more efficient if you have that buy-in and knowledge across many teams. So that you're cultivating, you know, holistic approaches and collaboration to avoid siloing and bento-boxing. Instead of someone thinking, my content needs to go on the site, so I'll add a page. They can see what these new lanes are, what the new information architecture is, and really keep top of mind, like these personas we have on the site, and figure out where is their content most relevant? Where does it make the most sense, and who should they collaborate with? What are their owners to make it happen? And with that, I'll hand it back to Katie. So we wanna just do a wrap up of a few of the things that we feel are like principles, key takeaways, things to always be coming back to and reminding ourselves when we approach this category of work. Again, not even just about the specific solutions that MSF was able to arrive at and through this process, but things that I think are just universals for how to approach in the mindset of these kinds of projects. So one is really think about designing your project with Consensus Building from the very beginning, and really thinking about what's the core group that you wanna have pulled in from the, like there's almost no way in which you can do that too early in the process. Websites, website redesigns are product, I mean, these somehow become the most political fraught projects of all the projects that I work on with clients, which span well beyond website work. And it really is because it's this one time where people have a shot to have their portion of their initiative featured, and we know how those politics can play out very, very quickly on the limited kind of one shot of presenting your organization to the world through the website. So that's one thing I think is just, even if you make a different decision, if you've really thought about how to bring in the right stakeholder from the very beginning, you're gonna save yourself so much pain in the long run of having to kind of retroactively find a way to address those concerns of people feeling as though their organization wasn't heard or left out in the process. Second, really don't skimp on educating people about why a certain decision is the right one. And I think showing, like show, don't tell is a very, very important thing here. What Taylor talked about, like just even being able to kind of show, we'll look at these other organizations or these other brands, and sort of the way in which these conventions are evolving to kind of assuage those concerns or feelings that people would have, is always gonna go so much better than just saying, this is why we need to do this. And I think that's just intuitive, but yet we, in the rush and limited time and capacity that we have around these projects, just oftentimes don't create a lot of space for that. And then just further those rifts and divisions that sometimes can form internally around projects like this. Building the case for ongoing investment with performance data, so one thing we didn't really talk about but I think is generally true is, we're really close to the numbers, for example, on a fundraising side for many of our clients about how many dollars are raised from email versus paid versus website. And while it's not always universally true, usually the website is bringing in substantially more than any one of those other channels, but yet again, the resource investment for that doesn't always reflect what it should be, where there should be the same kind of resources and investment approached with the website. So know your data, know your performance, own it, socialize it. At the end of the day, senior stakeholders listen to this more than anything. And I think that that's something that we oftentimes forget to champion, particularly around the website channel, regardless of what your organization's goals are. Really be unafraid to show that data and then hold yourself accountable to that data ongoing as a way to build support for further investment. And then I think this is a really important thing if you are able to shift the organization from the mindset of that you have one shot, one redesign to get it right. That's a really great way to start to remind people that it's not as though you have just one shot to get it right either. And so that can really diffuse the attention or the conflict that can arise around these projects. And I'm kind of making it seem as though these are terrible projects. That's not it. It's just we know that these things can arise and we've probably all seen it. And so I think that just reminding people that the launch of the site is just the start and that that is not a set and stone thing. So I come back to that point time and time again when we're debating, trying to arrive at the perfect decision. It's like this is not about that and we have to shift our mindset. If you think of a big brand like Nike up the road, they would never think this is our one shot to get this one feature right on the homepage once every three years. That's just not the way the rest of the world thinks and we have to do a better job of socializing that in our own organizations. So with that, we've wrapped up our portion. We've got about, I think about 10 minutes for any questions that folks have. So happy to field anything from the crowd if there are any questions. Yes. Yeah, thank you. So the question to repeat it, yeah, was around user testing and whether or not we were running user tests kind of before, during and after kind of when we were doing these tests to kind of give us data to act on. Is that right? Yeah, there was a wealth of data we had for the project but often kind of unfiltered that we wanted to surface insights from, glean from, you know, things like what's available in Google Analytics, doing kind of a page performance analysis to see how users were navigating through the website and that really surfaced like upfront, this clear picture that people were entering the website from a million different places and they were quitting from every one of them. There weren't kind of clear funnels for people to move through engaging with content. So a lot of the upfront work to kind of create a problem statement was done with data more so than direct user testing. But once we got into designing components, it was especially that navigation that we showed that also being the IA, like the real kind of organizing structure for the site where we wanted to validate a new approach first. Those are the two main areas where we started with along with just making sure that we're doing all sorts of, you know, accessibility testing for the new designs that we had. That we all wanted to be very confident in while getting to the launch of like these changes. Now that they are live, what we want to do is assess more clearly, like how people are moving through the site, wanting to get into more anecdotal research, both with our internal stakeholders and visitors to surface the new hypotheses. So given that we haven't changed too much content just yet, we felt confident that we were just doing or covering the bases for kind of a new organization on the site. Now that we're gonna get more into messaging and further optimization, we really wanna get into some anecdotal research, yeah. Yeah, the question is how are we doing research? Is it real time on the site through a modal or a pop-up? Is it anecdotal? Different areas of the site, we've run different kinds of tests. So for example, the ways to give section of the website is obviously something kind of in the development department's governance. We've run lots of surveys there through existing third party tools to create pop-ups and ask people whether they're finding what they need, what their experience is, and inviting them for focus group sessions. We're often using other channels too, like our email channel to invite super supporters to be involved in focus grouping so we can continue doing work there. So it's a mix of both. There's lots of insight we can get obviously from analytics and parsing through everything there. In areas where the persona is so specific or clear, that's where we're really excited to get people into focus groups. The people who are navigating those ways to give pages, for instance, are a very specific kind of subset of donors. It's distinct from the first level donors we have. So yeah, we're trying to kind of scale up or scale down, get more or less bespoke in how we're doing research, depending on the persona we're targeting. Yeah, thank you. I'll actually add one. Can you guys hear this okay? All right, one other thing I'll say, not with MSF, but with a couple other organizations that I've worked with over the years. The other thing that we've done is try to set, what I would describe is, I mean we actually use this phrase and I think it's helpful. It's called a learning agenda, right? So what are the things that we might have hypotheses about post-launch, that we wanna run longer term, tests and experiments on, leveraging tools like Optimize Leader Run, specific tests, and even using tools like that to make minor front end changes that don't require a lot of reworking of the underlying architecture of the site. So a great example of this is many years ago we worked with the Nature Conservancy. There were a handful of things that we were curious about changing on their homepage, but didn't wanna make anything that was potentially gonna negatively impact fundraising performance, so not going in and making larger changes. They didn't have the resources to do a big redesign. So we ran a number of concentrated tests around some high traffic areas of the site that we knew could really drive performance, ran those tests re-optimizedly. It wasn't even necessarily a formal user research experiment, but was able to really give us the data that we needed to know if we were safe to go ahead and implement some of those changes in between larger redesign cycles. And that, I don't know the top of my head, but resulted in some pretty substantial increase in your revenue for their end of your fundraising, for example. So that's another great tool that you can use when you're not necessarily positioned to have access to regular ongoing user testing. One thing I'll add is that it was a real benefit of having ongoing discovery through the whole project as well. That helped us to surface insights from lots of different team members at different times. In my experience past, when I was agency side, many products would have very concentrated upfront discovery with a limited set of stakeholders. But in working on this project, we may have been months in and we talked to a specific team member who's aware of a specific set of market research that we surface and ends up generating or highlighting some very powerful use cases that we wanted to target against or helps motivate our content strategy in a certain area because we realized that we did have this documentation on hand that said, they engage with this kind of content best and that sort of thing. I think having this space for that discovery over time and all that stakeholder involvement really made sure that we weren't missing things for us to kind of build into the UX that we're delivering. Maybe we have time for like one or two more. We've got five or so minutes left. If there's any other questions. So what kind of like all of the things I said on the most, what do you think about that? Yeah. Oh, sorry. But I could also experience with the NGLs and you know that some of the programs I would ask are in different ways and people when they need to be in the same structure it's very difficult for them, I think. Yeah. So the question is kind of around, you know when you're having an organization that has its own website or property in general, brand expression, but there are many different sister organizations doing the same thing. What arises from differences? Is there a tension or is there a need to kind of spread and share what's changed? How is it navigated? That's kind of the general question. It's interesting. I'll say first that we absolutely want to socialize and share like a lot of the work that we've done because we could see it being creating strong improvements in different other areas on some of the sister sites. And I think that the ways that we could socialize that and get that adopted is through a lot of the ways that we've got buy-in from our internal stakeholders. If we are really able to say through evidence and data that we are seeing better performance or a better user experience through the design changes we've made, then absolutely, I think there's a much greater chance that it will be kind of accepted and adopted and that will make a more consistent, you know, brand across the board. And that is all sorts of benefits in terms of just visibility, association with our brand. For instance, you know, one of the things that we changed is that all of the headers we used to have on our site used to be white text on a very red rectangle. And what do you know that looked exactly like our buttons? And so it's very easy to like make the case for other organizations or other websites, you know, to maybe create some distinction there, avoid rage clicking and a confusing user experience. So some things will be easier to socialize than others, but I definitely see there being benefits and want to help make consistency across the board. Did it, was there any like more specifics you had in mind or wanted to explore? Sure, thanks so much everybody. Thank you all.