 Alright, so we were just discussing up here whether we should scrap the original session and actually make this session the new Instagram icon, yes or no, but we figured we prepared some slides on this one, so we'll do this one. Hi everybody, my name is Adam Cap, I'm with the Sierra Club. This session is building a top 10 non-profit website, colon a project management case study. So there are actually three of us here at DrupalCon this year and here today, myself, I'm the director of digital product management at the Sierra Club, my colleagues Sophie Mattson who is our senior digital product manager and Jesse Brown who is our senior front end developer. And we're all trying to think of, we all want to do like a fun fact, like while we're most excited to be in New Orleans, mine is that I work in Washington D.C. and it has been cold and raining there for the last three weeks, so this has been nice. I think Sophie, what was your vote? Food? Jesse? Yes, expanding horizons beyond Tremay. Cool, so our overview for this session, so I know that if you went to the technical project management session, we do not have as many cat photos, we've already lost that battle, we've just admitted it, but we have some in here, we'll do better next time. We're going to go over like a sort of a brief history of how the Sierra Club got into both project management and Drupal, which sort of happened at the same time. We're going to talk a little bit about our team structure, our product process, some tools that we use, and then we're going to talk about things that we've learned and things that we haven't learned yet. We would love for this session to be interactive, it is five o'clock, so please feel free to raise your hand and ask questions at any time and we'll definitely set aside some time at the end of the presentation for Q&A. All right, so about the Sierra Club, we were founded in 1892, we are the nation's oldest and largest grassroots environmental non-profit organization. The keywords, oldest, largest, and grassroots might be a clue that change does not happen quickly or easily in our organization. You can see some of our current campaigns that we're working on here. We have about 500-ish paid staff. Our national headquarters is actually not in SF. It was in SF when I made this slide, but it's now in Oakland, California. We have a legislative office in Washington, DC, where I work, and offices across the country. This timeline is sort of an overview of, on the top, our process of getting into Drupal and on the bottom, getting involved with project management. You can see that these things sort of happened in fits and spurts and overlapped. Maybe not the best to try to relaunch your public website on a new platform at the same time that you're undergoing a large amount of internal change, but that's the reality. I think the favorite acronym at the Sierra Club is Rebuilding the Airplane in Mid-Flight, but we're an environmental group, so I always thought that it should be more like, I don't know, we don't fly in airplanes, but we came here on snowy owls. So just a couple of things to point out on here. We have a Digital Strategies Department now, which started at the end of 2011, and I'm going to talk a little more about that in a minute. Some other sort of keynotes, so project management certification through PMI. We're going to talk about that in this session a little bit. You can see when we joined PMI there in 2013, and Sophie and I did our certification at the end of 2014. You can also sort of see that we initially started with a Drupal site that was built for us by an external developer for one of our campaigns, sort of our introduction to Drupal, but then we sort of took the plunge, and our public website launched on Drupal also in 2014. Very shortly thereafter, we also did an integration with Salesforce, which we use for our CRM. If you have more questions about Salesforce, we can totally talk about that at the end, but it may not be of interest to everybody. The general theme here is that I'm going to maybe cover a lot of ground, so I'm going to sort of skim things, but if you're interested, again, please ask questions, and we can circle back to things at the end. This is what the team structure looks like in our Digital Strategies Department. As I said, we didn't always have a Digital Strategies Department. This happened a couple of years ago, and before that, that department basically came to be by pulling some resources out of the communications department, some resources out of the IT department, and then probably bits and pieces out of advancement and some other departments so that we would have a greater focus on our online efforts that were sort of housed within the same unit. So we have a Chief Innovation Officer, which is who I report to. I'm the Director of Digital Product Management. Sophie and another product manager are on my team. In green, you'll sort of see the development team. We have four developers. Jesse is the Senior Frontline Web Developer who has a star there next to him. The other sort of units in our department are still IT, digital engagement, digital innovation, and the relationship between the product management team and the development team is mostly, I would describe it as like us attempting to protect the developer's time from people who would like to go to them with questions directly. But we also try to have it be very collaborative. The developers are welcome to be as involved in the product planning process as they sort of choose to be. So they have the option to join from the initial kickoff meeting. Many times we'll even sort of give them a preview of requests coming in, just to sort of put it on their radar, even before the planning process has actually started. And then they can sort of choose which meetings they feel that they need to sit in, which they feel like they can skip. Obviously, by the point that we're getting to development, they're heavily involved. Jesse, do you want to say anything else about that? We ask for bug reports to come directly to the product manager, and we then file them in a ticketing system with the developers. So again, there's sort of some buffer space there, unless it's an emergency. We also obviously consult with that team pretty heavily to try to do estimation, planning when there are going to be windows of development time available to work on a given product. And we also issue a joint product update report every week that sort of talks about just sort of status of what we're currently working on. And in terms of the product team, I'm going to ask Sophie to talk a little bit more about the process of how we came to become certified project managers. So who in this room considers themselves a product or a project manager? Yay, you're in the right place. Yeah, everybody's in the right place. And is everybody familiar or at least has heard of PMP or PMI? So some familiarity, awesome. Adam and I are both certified PMPs, which stands for Project Management Professionals, if you're not familiar with that. That's through PMI, which is the Project Management Institute. As you can see on this slide, there are a lot of requirements for becoming certified as PMPs. There are 4,500 hours of project management. There's also some education requirements, and that was actually one of the... Oh, our window is opening. That's OK. One of the more challenging things with getting the PMI or the PMP certification was finding training courses. I did a PMP prep course through continuing education at a local college. There's also a pretty intensive exam that you have to sit through. We studied a lot for that and actually learned a lot about project management through the process of studying for the PMP exam and have used that to improve our product management process at the Sierra Club. It is a pretty big commitment, as you can see. So I definitely recommend getting your organization or your companies buying if you do want to do PMP. But also it is very valuable, so I definitely recommend it. It was interesting for those who were in the technical product management session the other day. So there were probably some pros and cons to a way between getting certification and not having a certification. Certainly having a certification is not essential to being a project manager. I think that it was very helpful to us sort of not coming from that background, just to sort of have both the competence as well as the knowledge to fall back on. But it has been great to have the thing that I think that is important here that I'll note is sort of like the continuing education requirement. And just like anything else, I think that project management is a skill, something that hopefully coming to sessions like this, some of the other project management sessions on the track here this week sort of helps you learn and grow, see different perspectives, and I think that's probably the most important part. I will say that I think that this was very helpful for us in terms of this particular case of launching our public website. I would say that coming into that process, that documentation was not one of our strong suits, and I think that that was something that really made that process much less painful than it could have been. This is what our product process looks like. At first glance, you might ask, oh my gosh, this is super complicated. Why would you show someone this diagram? But actually, I think it sort of serves to underline that anytime you're deciding to build a product, obviously there's a lot of time and resources that are going to be involved in bringing that to fruition. And if you don't have a process that's very clearly laid out, even with down to sort of this level of granularity or even more granular, I've seen ones much more detailed than this. You know, you run the risk of not having clarity or having a misunderstanding between people, like how this process is going to work. When people run up to us and say, can you build us a website? I need it on Friday. And I say, this is what we need to go through in order to build this product so that probably can't happen by Friday. Sometimes it has to happen by Friday and we make it happen, and sometimes we don't do every step. But this is what we try to do, and we're going to walk through these a little more closely. But the main reasons for having this product process are to have a team that is basically shepherding a product, which can be a website or an app or a data visualization or just about anything else. Across the various departments that it has to touch to work with those stakeholders to make sure that it's delivered on time and that it meets the product specifications that we set out, basically to allow us to bring our expertise to have a strategic user experience. We were just in a session where they were talking about content strategy and the idea of what the user wants versus what the organization wants. I think we try to be an advocate for the voice of the user in the process to make sure that this is going to integrate with the rest of our systems. Like I said, we have Salesforce for CRM. We have several other tools that we use and to make sure that all of this data flows together so that we have it for future use, that it's captured correctly is really important. And just to coordinate resources to make sure that everyone in the organization understands what the priorities are and that projects, we sort of have to schedule windows or things like that because we're a pretty small shop. We have a really phased approach to development. So often when we get to the end of this cycle, we're capturing feedback, hopefully from users, from other stakeholders, and feeding that back into the process and then sort of starting like a phase two and capturing those learnings. Yes, we're going to zoom in. So I can't actually zoom in on this. We are going to go through each of these stages on another slide, but just sort of quickly, there's a, sorry, I don't think that I can zoom in. There's a planning stage. There's a user experience design stage. It's actually sort of like a pre-planning. There's like a request phase, planning, user experience design, visual design. Development and launch. Yes, the slides we can also share. So we're going to talk about tools that we're going to use, and I just put a little disclaimer in here, mostly because I wanted an excuse to have this fun photo of a terrifying crow using tools to accomplish things, which freaks me out. So we're just going to talk about some tools that we use. We're not getting paid to talk about these. There are other tools that you could use. If you have tools that you love, we would also love to hear about them. So these are a couple of the ones that we just are going to sort of mention as we're running through here. We use most of these on a daily or at least weekly basis. We definitely don't use all of the functionality that any of these tools offers, but we have found them to be useful. And as we walk through our sort of product process, we'll sort of mention them. And in many cases, we have screenshots and stuff for you guys to look at as well. I'm going to pass it back off to Sophie to talk a little bit about the first step. The first step in our product process is the product request form. So the way that this works is we have a product team, product management website. It's an internal site, and we have a bunch of these forms on our site. As it says here on the slide, there are several different versions of the form, depending on the type of request. Generally, product requests come in from Sierra Club campaigns, like the Beyond Coal campaign, the Beyond Oil campaign, through the online organizer who represents that particular campaign. And so whenever someone has a product idea, we ask them to fill out one of several different forms, depending on the type of the product. For example, a brand new campaign website would have a different form than a data visualization because we need to know the answers to different questions in order to figure out what the product looks like in those cases. We ask folks to fill out the product request form as soon as they come up with their idea so that we can be involved from the beginning, sit in on meetings, help them form the strategy, rather than them coming to us with a fully formed idea that they need, like Adam said, on Friday, and it's not actually going to meet their needs. The questions on the form, which there are a few examples over there on the right-hand side of the slide, those guide stakeholders to think about the strategy. So for example, what do you hope that the product will accomplish? What metrics can be used to judge success? Yeah, those are some of the examples of questions. And filling out the product request form helps everybody be on the same page as far as what the product looks like. So that's all the stakeholders, the campaign folks, online organizing our team. Once the stakeholders are filled in the product request form, they submit that to the director of digital product management, who is Adam, who is going to talk about the next step in the process. Oh, yeah. So one of the things that we do is, we use user testing.com. Have people heard of that or used it? Some folks have. It's a user testing service that we've started using more in the past few months to get feedback during the planning and UX phases. I used it recently to perform a comparative analysis between one of our sites and a couple of competitors and got some good feedback there. You can also get feedback on prototypes, although we haven't really started using that feature yet. Basically, user testing sends out a bunch of questions to people in the world. They contract out to folks and they go through your site and try to answer questions to try to accomplish things and you get feedback on whether or not they're able to accomplish what you are asking them to do. Anything to add? So the next step in sort of our planning process is creating a project charter, project initiation document. There's probably 100 names for this thing. There's a screenshot, which we're not expected to read there on the right. It's basically a document that we keep in Google Drive. It sets out objectives, the project scope, team roles, a very high level timeline, costs, and sometimes even risks. It also sets expectations for stakeholders. And I think this is the most important part. So this is basically going to be the cooked down version of a lot of discussions and probably an initial meeting or two. And once things are on this paper and it's been approved by everybody, then we have this to sort of fall back on in terms of this is what we agreed was going to be the scope of work. So when people start talking about features that aren't on this list, on the scope document, then we know that that's probably not going to happen. There's a couple of the objectives for our public website when we were going through this process. The first one was just to move the site onto Drupal. That was the big one. We had a previous content management system that we were using for a small part of our old site and in order to avoid re-upping that contract which was going to cost a significant amount of money, we knew we wanted to move over onto Drupal because that was the direction we were headed anyway. Obviously, whenever we're talking about our website, online fundraising is a big deal. So it was really important that we wanted to better optimize the site for conversions both financial and list growth versus the previous incarnation of our site, which was actually built at a time when online fundraising was really in its infancy, at least for us. The site had to be responsibly designed. Our previous site was not. We're talking this was happening in 2013-2014. We were probably a little behind the curve on that one, but we knew that the site needed to be mobile accessible. Increasingly, we're trying to reach out to audiences that we know their primary mode of internet access is via mobile devices. And so that is important for our diversity and inclusion efforts, as well as just being a modern organization. We also wanted to be able to provide a localized experience to have people give the option to opt in to see events and news from their local chapter or group. And also another important point was to work with our content creators to be sure that the content that they were posting on the new site always reflects current messaging and current work. So translating that to a scope, we knew that this was going to mean a new Drupal site. We knew that this was going to mean new user experience, new visual design, doing an inventory of our existing content and figuring out what we could keep and what we were going to get rid of. We also have a member magazine, Sierra Magazine, which has a huge website and a ton of back issues, so bringing that content with us and giving them their own home. We knew that it was not, fortunately, going to involve our other campaign sites, which we had already moved to Drupal, so like our beyond coal site, for example. This is like a clip out of a spreadsheet. Sometimes the scope document is more simple, like it's just bullet points. Sometimes the scope document is more complex and it involves spreadsheets and it involves breaking different features out, identifying which parts of features are A priority versus B priority, anything that's not an A or a B. Sometimes anything that's not an A is probably going to get pushed to phase two because we want to identify the, obviously, the minimum functionality, get this thing out the door, start getting some feedback and sort of iterate from there. This is a Darcy chart. People familiar with the concept of a Darcy, anyone? It stands for decision maker, accountable, responsible, consulted, informed. There are some variations on that, but basically we're breaking out here who on the product team, so the decision maker sort of has the ultimate sort of bottom line authority. They're also sort of usually like the champion for that portion of the project. The person who's accountable is sort of doing most of the day-to-day, usually that's the project manager. Responsible, those are people that are getting tasks usually delegated to them. Consulted, people who are being told about what's going or people who are being asked about what's going on and invited to provide feedback informed. These people sort of get CC'd. They know what's happening, but they're not really, the project is proceeding with or without their involvement. This is a very high-level project schedule. This was for our .org website. You can see in the beginning, we're mostly looking at what's the planning time, what's the time for user experience design, for visual design, when is the development window, are we doing a soft launch with a testing period? And this is all just in a document. I mean, this is not fancy or high-level. We also try to do progressive elaboration. So as we move forward in the project and we have greater clarity about what the next steps are, we can come back to this and dial more detail in. And then often we move the project schedule into Smartsheet, which we'll see here. Smartsheet is a tool that we've been using recently to track tasks, to track task status. You can see on the right, which I could pull over if this is the live site, but I'm sorry, it's not. You can do Gantt charts, if that's a thing that's interesting to you. You can do dependencies. You can also publish these, and we also often link to these from our internal digital team site, so that stakeholders can sort of see the status of various tasks without having to email us and ask us what's going on. But they still do that, to be honest. You know, we can also roll these up into higher level dashboards. That's something that we're still trying to figure out exactly how to do. I know that Smartsheet has a resource allocation functionality, which we're investigating using that. But again, I think that the core project management is really documentation, obsessively document everything, and breaking tasks out into the smallest manageable chunk. So that's really what we try to do here. And Sophie's gonna talk about wireframes. So after we are done with the planning phase that Adam just described, we move on to user experience design, which for us primarily means doing wireframes. We also do site maps, but it generally depends on the size of the product. So larger campaign sites would require site maps, but like a data visualization, of course wouldn't, because it would only be one page. So the product managers, Adam and myself, are the ones to draft the wireframes. We use a program called UX PIN, which you can see there's a little logo over in the right-hand side of the slide, and that is an example wireframe over there on the slide. We sometimes use different exercises to determine priorities beforehand before we start getting into the actual wireframing in the tool. So UX PIN is a browser-based wireframing tool. We started out using Google Draw originally, but realized that that wasn't dynamic enough. So we did a competitive analysis and found that UX PIN was gonna meet our needs the best. We really like it. There's a lot of flexibility within the program. You can do drag and drop, and also there are a lot of existing UX elements from across the web that you can pull in there, so you don't have to do it all from scratch. You can also do some interaction and prototyping with UX PIN, which is nice, and you can do responsive break points. UX PIN makes that really easy to do. We are still learning to do wireframes mobile first, but we've gotten to the point where we're doing them doing mobile and desktop at the same time, so we've made progress. Another nice thing about UX PIN is that these wireframes can be shared by a link, so we can drop the link into our project charter and onto our product management website, and the stakeholders can comment directly on the wireframe using that link. One thing that we've learned is that not everybody knows what wireframe means, so we sometimes called them page diagrams or mock-ups depending on the stakeholder audience. Adam, did you want to talk about smart elements in UX PIN at all? I mean, it's... Smart elements are another cool part of UX PIN. You can basically create... So if I know that the navigation on a bunch of wireframes is all going to be the same, I can create those elements and then save them as a smart element, and then it's basically CSS. Then I can put that element on everything that I create, and if I change it one time, it'll update everywhere. And again, just the fact that it's cloud-based and that we can send that link out to people that don't have to create an account and we can just get comments, because we have a distributed office, obviously, is super helpful. Visual design. This is a comp or basically it's a PSD file or a PDF file that we get back from our visual design team. They are in the communications department. This is now a very, very close representation of what our website actually looks like at this very moment. It's changed a little bit from when we did this almost two years ago, but not very much. We're trying to move away from the idea of doing full page comps and having lots of PSDs to the idea of just having everything, basically having a style guide for the site. There have been some great sessions here, and this is something that I've really been trying to learn more about this week and something that I hope that we'll be able to move to soon. So that it's less work for the designers that they are basically only creating certain new components when we're working on a new product. They don't have to spend lots of time giving us these big PSD files that we also don't really need because the style sheets for the site are already pretty locked and we're not reinventing the wheel. And it'll be, I think that that is something that we hope to move to pretty soon. And again, we're also working with our designers always and ourselves to be thinking responsibly, to be thinking about the mobile experience first or at the same time as the desktop experience. Please. Yep, so the question was how many breakpoints do we have basically? We started out doing three. So we have a desktop one, sort of a slightly narrow one, which would be like a tablet, and then a pretty narrow one, which it would be a smartphone. We don't spend as much time. We don't typically now do the middle one, the tablet one for every product because it's basically just a slightly less white space. Things get compressed a little bit, but it's really the mobile one where things start stacking where it's really like a radically, not radically, but a different experience. So yeah, usually just two, but sometimes that third one. We always look at it when we're doing testing, like on a variety of widths and devices, just to be sure that nothing's looking funky. And if it does, we can obviously go back and address that. Jesse is going to talk a little bit about development and implementation. And now we start having more CAD photos. Thanks, Adam. Hey, everyone. If you have questions or anything or fun stories you want to share that's relevant or whatever, feel free to jump in. I don't mind being interrupted or whatever. So yeah, I don't know. I guess I want to talk a little bit about some of the challenges that we have, what our process looks like now, what it used to look like, and some of the things that are working well. Share a couple of ridiculous stories and walk through some of the tools that we found pretty helpful and that certainly make my life easier as a developer. So as Adam and Sophie have described, we're part of a pretty big organization, our site. Well, we talk a lot about Sierra Club.org. We have a multi-site build that has 47 sites in its dock route. And we've got three developers and only two of us work on that. So when I first started, there was only two of us in the team. And my boss is great, but he'd make me call him my liege and refer to me as a minion and stuff. Totally humiliating. But I stuck with it, you know, and it's been good. But the design issues we face are pretty interesting. Like we have a design team that's in a different department. And so Sophie and Adam all often do wireframes. They'll give the wireframes to the creative department that has the graphic designers who historically had print backgrounds before the web. So then they give me a Photoshop file and I'm like, cool, Photoshop file. Thanks. And I'd started, and they're great. Our designers are awesome and they get it. And they've been really responsive. But one of the first things I asked them was, I was like, can you just give me a list of all the hex codes for the colors in that file so I don't have to go into Photoshop and find them? And that's been really helpful. And they've been like super responsive and open to change, which is great because the web is not print, right? As we know, it's different. And yeah, our process is not perfect, but it's gotten way better. I've been at the Sierra Club for about two and a half years. And when I started, I inherited a legacy site that my predecessor had built. And we have, I guess, three different roles that we interact with a lot in the Sierra Club. So there's the developers, the product managers, and then we've got the internal stakeholders who want the thing built. And so I inherited the site and I'm looking at it and it's like trying to do 15 different things and I'm trying to figure out which one's the most important. So my first question on my first day was, cool, why'd you guys build this site? And the answer was, I don't know. I'm like, okay, cool, who knows? I'm like, I don't know. I'm like, okay, so let's get a list of people who might know and we'll meet with all of them and we'll try and figure out why it was built. It took a month. And we came up with three or four different use cases for why it was built. One of them was to pitch to potential donors. One of them was to get end users to take an advocacy action through the site that would send a message to a decision maker or whatever. And another one was to be an organizing tool for regional people in the campaign in their local states to get to their hugely different use cases. And it wasn't clear which one was the most important. So the next thing we looked at was, okay, out of all of these things, which one is the highest value? Which one's the most important? Like if this thing only did one thing, what would that be? And then we figured out that that was for an end user to take an advocacy action through the website. So we're like, okay, cool, make that button the biggest and change what's there because it starts to measure the conversion rates and stuff. So that was pretty interesting. And it improved where we are now. And now the process that Adam and Sophie have implemented where they make these stakeholders now write down what they want. Like now when we get a new employee and they inherit a site, they can look at it and go, oh, cool, this is why it was built. This is what it's supposed to do. So that's been really helpful. And it's getting better. Yeah, of course. Yeah, sure. So we use GitLab issues a lot. We were on GitHub for the last couple of years, but we just moved to GitLab last week, not because of GitHub's pricing restructure that they dropped on everyone this morning. And I'll talk a bit more about that in detail, but yeah, we use the GitLab issues feature. If anyone doesn't come... Is there anyone who doesn't know what that is? Cool. So it's a great feature. We use it a lot to communicate between the product and the development team, and we also have some internal stakeholders who have accounts and access to that now. So every issue has its own web page. Every comment is tracked. We can reference, like, commits in it. Oh, dear, we've lost the screen. And we've got like a couple of years worth of history in there now, which is great. And we've had, like, as our team's grown, we've been able to point them towards that, and pretty much any question they have, they are able to just type it into the search, and they can hopefully find up, like, maybe five issues that are related to that and are documented. So the things I like about that is it doesn't lock up communication in email threads, so you don't have to, like, try and find someone who was on some email thread when a decision was made, like, two years ago and remember why it was done or whatever. It's all transparent and open, and we provide access to it, to anyone in the organization who wants it. It's like, cool, you want to see inside our process? You can, no problem, we'll grant you access. So we track a lot of that, a lot of that through issues. From a development point of view, I guess, like I said, we have 47 sites in our dock route, and that sounds like a lot, but we just got it down from 70 something in the last few weeks, so it's still a lot, but it's less than it was a month ago. So as a result of that, like, our development team, we do a lot of site configuration. We use a lot of modules out of the box. We've only got a couple of custom modules in there to do really use case-specific things, like interact with a Salesforce API and send it some data or an update a user record in there or pull in, like, some content from a Salesforce API and create a node for every item of content that's been created in Salesforce. So we're doing a lot of cross-platform stuff at the moment. So we've got some custom stuff, but we try and use just straight Drupal out of the box as much as we can, because we are limited in resources. The front-end stuff, we're using Bootstrap, and we do a lot of work in the theming layer, probably more in the theming layer than the module layer. And we try and take a pretty iterative approach to development, so when we get these product requests, one of the first things I usually do, so if you know, I work pretty closely together on a lot of projects, and one of the first things we do is look through their feature request lists and go, okay, cool, I can probably do this one today, that one this week maybe, and the others, like, I don't know how long it's going to take me, but I'm not going to get to it this week, and that's going to be a big job. And everyone always wants to know, like, how long is it going to take? And I have a super frustrating answer for that that I give everyone, which is, I don't know. I don't know how long it's going to take. I don't know. And I'm like, I can tell you when I've started on it, and I can give you updates daily, weekly, or whatever, until it's done, but I don't know. I don't know how long it's going to take. And I think people have kind of gotten used to that, because they don't seem to ask me anymore, which is great. Oh, they still ask me. Oh, they still ask you, so you don't ask me anymore. Yes, we get the fund out of estimating for you when you don't write an estimate. Thanks, team. Keep that up. And, yeah, let's see. What else do we want to talk about here? Oh, yeah, there was one or two, like, I don't know. There was one of the first projects I was given. Someone had, like, a request. They wanted, like, a map and a timeline, sort of, as separate products, and then they wanted both of them rolled into one. And this experience really led me to ask the question, like, can you show me an example of something that's like what you want to see at the end? Because I didn't ask that question. And I was like, okay, cool. Maps are fun. I like maps. I'll do the map first, and then I'll do the timeline, and then I'll do the both of them in one project. And so I did the map, took two weeks, and they looked at it and went, yeah, that's fine, but we're really excited about the timeline. And I'm like, okay, cool. So I spent a bit of time thinking about the timeline. And I'm like, okay, well, this is what I'm thinking about for the timeline. And they said, well, actually, we just want another one of these. And they had one already. And I looked at it, and I was like, how did you guys do this? And they're like, oh, we just used some third-party thing. You just plug the data in. I was like, oh, cool. Give me the data. It was done in a day. And we still have the first one. It's still cool that it's in production, but it was a classic miscommunication, right? Disaster, like, it's super annoying. But anyway, whatever. We still use both of the things that we built, but it was definitely a good use case of product process going wrong. Or not quite right. Definitely a right. Speaking of which, if you're an independent contractor and you want to be part of our process, we're looking for one-person devs who take on contract work every now and then. And Adam and Sophie would love to talk to you if you're here and interested in that or know anyone who'd be good. It's my shameless request for crowdsourcing help there. And yeah, let's move on to the next one. So this is just an example of the format of how we keep track of that communication that you were asking about earlier. It's just an example issue. Probably the two main ways we categorize these issues inside GitLab is we use milestones for big umbrella projects, and that might be a domain name. So all of these issues are related to this particular website with its own database and its own theme or whatever. And then we use labels as well for assigning priorities in just whatever other region. And so with a combination of those, you can filter whatever. I don't know. It's nice. And it preserves the comments. Now we can go to the next one. Thanks, Adam. And it's all right. Just trying to cut me short. I'm running out of time. I think I'll take the hint. Sophie or Adam? I think Sophie, it's you. Thanks for your attention. Question. Yeah. Sure. That's a great question. The most recent one. Oh, it's the community edition. Yeah, it's the free one. So yeah, the drawback to it is it's not cloud-based. I love that about GitHub. Like anyone could just set up a username and log into it. And it was like cloud. GitLab, we have to host on our own server and provide access to it, which we're still figuring that out. So I think the way we're going to make that accessible to more people is we're just going to whitelist like the IP addresses of our office and let anyone from inside our office just get to it without having to go through a firewall. We're still figuring that part out. But yeah, it's the community edition and it seems fine so far. I've only been on it for a week. Yeah, I get it. I work remote, so I have to VPN in. And so our contractors, they're just going to have to do that, I think. Especially now, right? I haven't used Bitbucket and to be honest, like the main reason we moved to GitLab is we moved our hosting to Blackmash and we bought their DevOps tool, which only integrates with GitLab. So we're there. Yeah, otherwise I would have been looking at something open source and easier. Because I feel your pain. I've been dealing with people complaining about the access for days now. Yeah, thanks. Yeah, hopefully that's useful for someone here. So the process Jesse was just describing is the implementation phase of our product process. After we finished building the thing, we launched the thing. And the launch depends a lot on the product itself. Generally, all the promotion happens outside of our team. We have the communications department and we have social media team and they do all the getting it out into the public view, of course, depending on what the product is. After launch, we do an evaluation memo to wrap up the product and gather learnings. The timing of this depends a lot on what the product is and the promotional plan. So it may be a week later that we start on this evaluation memo. It may be a couple of months later, depending on the launch period and the type of product. We assess each objective that we laid out in the product initiation document or product charter and evaluate whether or not it was successfully achieved. Those, the measures of success can be either qualitative or quantitative. A couple of the ways that we use to measure success, we look at Google Analytics to see where people are clicking, time on site. We generally don't use page views as a measure of success for the product team because the page views are based a lot on the promotional plan that the communications folks do. So we're not using that as a measure of success for ourselves. And we also use forms to assess author satisfaction and we'll do some interviews with stakeholders after the fact to see whether they were happy with the product what worked well with the process and what should we do differently the next time. And then over on the right hand side of this slide, you'll see examples of some of the things that we include in our evaluation memos. Yeah, so we often use Optimizely for AB testing, which we will be talking about shortly. We also have a beta testing process. I don't know that that's exactly what you were asking, but between implementation and launch. Yeah, depending on the product, we'll create a testing plan, which at least involves sort of looking at it on different devices and our internal team trying to break it and could be big enough that we'll be like writing a lot of very specific testing scenarios about logging in from different places or trying to add different types of content or sort of whatever it is. In terms of AB testing though, so after we've launched something, we definitely want to start, like I said, getting feedback and sort of figuring out how we can improve what we've just made. Using Optimizely for AB testing is one of the ways that we're doing that. We just very recently wrapped up this test of this thing that you're looking at, which we're calling an opinion lightbox. You might have seen it on like Upworthy or some other sites. My personal feelings aside, I think that the test was very successful, so that'll probably be rolled out. So sorry, this is actually just like the step one, step two. So the AB test was, we also had like just an email capture in like the bottom of the page or in the sidebar and this was like a lightbox that was sort of on top and this was much, much more, I don't want to say popular, popular is probably not the right word, but certainly effective. User voice is another tool that we use on some of our sites that allows people to sort of report broken links or things like that. But to sort of bring it all back here with Hipstercat, who looks very, I don't know, erudite. What have we learned? So we've learned that when you're building a large site with a bunch of different content areas, especially when you're sort of like an older, bigger organization like us, it can be a trap to design based on your internal structure or like how your internal teams create data because what you really want to focus on is the user experience and the user sort of wants to find all content related to one subject area in the same place and doesn't know that the team that is putting out the press releases may not necessarily be the same people who are creating data visualizations or fact sheets or an action alert. Some other things. Digital literacy is a skill that we're still really trying to help our co-workers and our volunteers in some cases develop. There's a big range, but as Sophie said, we still sometimes walk into rooms where we talk about wireframes and people say, what's a wireframe? Or sometimes it's even a little more basic than that. So just sort of developing a culture around digital literacy is something we've really tried to promote. Change management is hard, especially at a big organization, especially probably with having volunteers in the mix because they're obviously not accountable in the same way that paid employees are. You know, we having really good, the other kind of documentation, like having instructions for your products that people can access via Google Docs or via an internal website is really helpful, but only if people read it, only if people know where it is, only if it's written a way that they can understand. We also sort of had this, back and forth between all of our Drupal properties, essentially be like one giant website that just has like different themes implemented in different places versus like multiple instances. I think initially we sort of went with like the multiple instances approach and now we're sort of rethinking that and pulling back some of that content a little bit into the main website. Basically just, I think because of maintenance, complexity, having to roll out new functionality across multiple sites. And so again, choosing like a smaller site to learn on with like a new functionality or something like that can be very helpful. And again, just iterative development allows you to capture feedback, sort of learn while doing and roll products out more quickly. Some things that we have been slower to you learn or have not learned yet. This is a sloth, it's not a cat, I don't know if that counts. But things we're dealing with right now, how to keep content created and I think I'm supposed to say updated actually, sorry, by dozens of authors. We also probably at some point are going to need to revisit exactly like our permission structure and our user roles and how those sort of overlap and are set up. Some of them have like access to different text formats within content types and sometimes that creates issues. Content delivery slash maintenance can be a bigger challenge than setting up a new site page content type for sure. Again, even when you get that initial round of content upfront that you need to design the site or that it's going to be there at launch, following through on that can be a little bit like pulling teeth at times. And just still working like as an organization to try to not always be responding to the latest thing that is on fire figuratively or literally, but trying to have a roadmap for like at least a quarter ahead. It'll be nice to at some point maybe get to a year ahead and obviously we'll always have rapid response that comes up to things that are happening in the news cycle or even internal events. But again with small teams it's really helpful to be able to plan out like a sequence of events when different people are going to be working on different projects and that way we can identify if there are going to be any pain points where we feel like we're not going to have the bandwidth to get everything done when it needs to be done. And just a few TLDRs. The project management process it can be a little bit of a long road to get there but we think that we probably I feel like we couldn't have gotten through this process without it. Jesse definitely told you some stories about pain points that we had before we really sort of realized the idea of having project managers but there's a lot that you can do when you go back to your organization next week after being here just in terms of documenting things defining user roles basic stuff like that. Drupal and project management I think actually work really well together I think they're sort of a natural alignment Drupal lends itself to a pretty structured approach and in the same way that you set up taxonomies or you're associating fields of the content type that's sort of the same level of detail that we try to have in our documentation. We showed you some tools that we use a lot of them are free or low cost there are obviously others out there and we definitely take recommendations and really just the bottom line taking time to set goals and then assess before the product and then assess a success once the product is launched you know it's really the essence of the thing. All right sorry we used a little bit more time on that then I hope we had had more time for questions but we still have seven-ish minutes questions yes tips on getting stakeholder involvement at the right time so right that's one of the steps in our in our planning process is identifying you know who has a seat at the table in terms of these products and sort of creating that that Darcy chart. One of the questions first questions I always like to ask in that kickoff meeting is you know who isn't in this room that needs to be here and everybody can sort of provide input into that and oftentimes those are people that end up just being in sort of consulting roles or something like that but if you think that there's a chance that someone might need to be a stakeholder in your project and have you know a voice I think it's good to ask them up front to sort of move even if you fear it a little bit move towards it because it's better to sort of have your it's better to sort of have that as a known and then deal with it then have it come back later that you should have asked that person and you didn't know the question was did we learn things in the project management certification process that were not related to web development or to building websites that were helpful I think definitely yes I mean certainly I think that project management was probably born out of a sort of like history of manufacturing which can seem very different but I think that the idea of you know just sort of having the idea of of a tolerance is obviously different in web design that it isn't manufacturing but that idea that you know not everything is always going to exactly match up to what's on the PSD right because the Photoshop designer you know lives in a perfect world where they just move things around and obviously when you're implementing that you know at theming it it can not always come out quite as precisely so if you can you think of any other for sure yeah so progressive elaboration just that idea of putting in as much detail as you know right now and then coming back to it later with more detail I'm happy to talk more about that after but other questions I'm a big believer in like Gmail gchat and actually talking to people to their face via Google Hangouts or in person some people in our office are on slack I haven't I haven't consumed that Kool-Aid yet it feels like another inbox for me to check at the moment but maybe talk to me again this time next year who knows yep and get lab in terms of the the tickets support tickets bugs and issues especially like right after launch sort of like a triage so the clients in this case are mostly internal stakeholders so they sort of send bug reports to to the product manager and then we get them to the developers right here sorry him first and then you right so I mean we actually started out like I said more leaning towards having multiple instances of Drupal like for not each site but for sort of like sections of our our web presence and now I think we're moving back the other way the pendulum is swinging back it's less work to maintain fewer instances like I said when we roll out functionality only having to roll it out in one place it's much easier to sort of leverage the power of Drupal for like surfacing content if you have everything living in the same instance and sometimes it seems like they are like I mean we do have several themes like so our internal magazine still lives within the same instances our public website and those look somewhat different but but I mean we hope that through visual design that they still look enough the same sorry say the last part a little louder yep so I think the the two main questions that I sort of heard there were how we apply applied the the process that we learned through the certification sharing that with the development team oh sorry gotcha sure so okay so first question in terms of the the iterative approach I mean I think it's mostly born out of you know we're aware that our our process can can again it's like pretty detailed we're always eager to figure out what like the minimum functionality is and and sort of get that get that out there and then and then build off of that which also gives us the chance to hopefully capture more user feedback and not not have it be you know in development for such a long period of time and also again because we have a smaller shop once we get something you know launched we're obviously still you know monitoring it looking at analytics getting getting feedback but in that time the developer and then the project manager have probably shifted gears to be focusing on another project that is then going to be in its launch window so we might not circle back to that original project for a phase two for a couple weeks or maybe even a couple months and then I'm sorry the second question evaluating Scrum I'm like aware of Scrum as a concept but it's it's not something Jesse do you have I can but uh but yeah we haven't adopted anything that has formal the Scrum mainly because our dev team is so small and it just feels like it would be weird so for any folks listening at home the condensed version of that answer was that because our team is so small we we tend to have like a less formal version of that process essentially we're happy to stay and answer more questions there is a short link up here for the evaluation for this session please do that this is this is is our first time at DrupalCon we're super excited to be here but we would love your constructive feedback on you know how we can do better the next time that we're here