 Kia ora. So I'm happy to introduce Adrian Kingston. He is going to be presenting more than just a website, changing the way Tapapa makes digital products. He is the digital collection senior analyst here at Tapapa. And previously, he's worked across the sector, both nationally and internationally. Adrian and I have to say your bio on the app totally downplays the important work that you do here. But central to his work is making digital collections accessible and useful. Ta-da. As soon as you have internet access, I'll probably start. You're going to be extremely disappointed if you saw the last presentation. Mine is nothing like that at all. I have no pretty pictures, no nothing. And mine's not even about a product. It's about process. So yeah, I'm Adrian Kingston. I'm currently digital operations lead at Tapapa. I've been seconded into a slightly different role looking at operations across the organization for digital. And one of those things is ways of working. So that's what I'll be concentrating on in a second. If you have seen me present at all before, you will know that I usually have way too many slides with way too many words on it. Thank you. And I have that again. So I will be going very quickly. But really, I think what you hopefully will be of more interest is actually the end. And if we've got time, and I'll try and go fast enough so that we have time, be more in the Q&A that might be of value. Just OK. All right. So it's not technical or anything along those lines. So we've kind of done that. So we made a new website a little while ago, our corporate website. But the way I saw it was actually two projects in one. One was making a new website. One was demonstrating to the organization new ways of working. And I very much saw them as almost equal. I'm not going to talk about the website. I'm just going to talk about demonstrating new ways of working. But I'm going to start by talking about the website. So the last redesign of the corporate website in the main to papa.govt.nz was a responsive redesign in 2012. It didn't touch the infrastructure at all. It didn't look at content, which it really should have, because it had grown over quite a long time. But in 2015, we had a burning platform. So we had to do something. And there was a little bit of a lull in digital activity in between restructures and various other things like that. And we had some staff changes and vacancies. So it was a bit of an opportunity to focus on something a bit bigger. So we had to replace aging infrastructure, migrate, siloed, mini sites, all that kind of thing that you should do when you're looking at a new site. But as I say, I wanted to demonstrate new ways of working. So agile was something that was entirely new. We wanted the project to be very user-focused. Our previous main website redesigns had not been. They had been very much brand-driven. We wanted to demonstrate the use of personas. We wanted to check our voice and tone to see what people actually thought we should sound like. We wanted to have accessibility as a priority, transparent, and data-driven decision-making and open where possible. So these were all kind of new things for a digital development at Te Papa. So agile was a big thing that we wanted to try. It was the first project at Te Papa that hadn't been done before. And it was at the team's request. We wanted to run agile, even though we are an extremely waterfall organization. We had to run in waterfall and agile because we still needed to report up. We still needed to meet a lot of the things that an organization has to. So we were running both at the same time. There was definitely distrust of agile because it was a little bit of a dirty word. And we had to make sure that our reporting and governance was in line with what the rest of the organization was doing as well. There was a lot of misunderstanding about minimum viable products, and I'll talk about that. And we had very limited experience with agile. And agile in government is difficult as well. There's been a lot of talk about agile in government in Wellington over the last two or three years. So we're certainly not the first to try it. But there are lots of things that get in the way. One of the big ones being procurement. When we went out to tender through Getz, the government electronic tendering system, you have to fill in this huge multi-page, it's like 60 pages or something, with what you want. We didn't know what we wanted. And we didn't want to know until we'd done the research. But you can't work like that in Getz. So we tried as hard as we could. We had feedback from some vendors saying that it was a schizophrenic tender request. And some people backed out that some people didn't respond because they couldn't understand it. There was also, we have not been agile at all before. And so people didn't believe us when we said that we wanted to be. So that was hard. Our hands were tied. We had to do this because we're a government agency. But it's not what vendors were wanting to do, and it wasn't what we wanted to do. And we had waterfall finances. We had to pitch up front for the whole life of the project. And that was not something that was actually good for us. But it's the way we had to do it. So we learned a lot. We've talked a little bit with other agencies. And the government is trying it out and they're having some progress. The way that I've worked in the past is actually running business teams as agile, not just technical projects. And I've found it to be really good. So we ran our team as, so the project team right from the beginning before procurement as Scrum Band. So we had fortnightly sprints. We had daily stand-ups. We had sprint plan demos and retrospectives. We had a trolley board that was open to the entire organization. I don't think many people looked at it. But it was us being transparent. And the fact that everybody could look at it meant that we were very careful about what we did. In a good way, we weren't hiding anything because we knew we couldn't. We would have liked to have had a physical board for more visibility because we were trying to promote new ways of working. But space in this building is at a premium. We did manage to get a project room for a lot of the time. We wanted to project information radiators of what we were actually working on. We did a little bit of it in a second. And then once our technical partner was on board, Catalyst, we ran a parallel technical sprint board as well because they were used to working in agile and we used their tools. But we kept our project and our business scrum band running at the same time. So the thing about cam band or scrum boards is it's about transparency of work. We managed our team sprints. We actively managed risk and resolved problems on the board, which was actually something that parts of the organization were uncomfortable with rather than having a separate risk register, which you never look at. We had it visible. And again, it was a big part about the transparency of the work that we were doing. If we could see a risk, if it was in the business, it had to go on the board. And that might mean that the person who was at risk could see themselves on the board. But we found ways of making it work. And we found it was much more actual. We got through risks. We saw them coming earlier and we fixed them faster. We managed our pace and our rhythm and velocity, something that Waterfall does not do well. We helped each other. We removed blockages. We dragged progress. We evolved it as we learned because we were very new to it. And I think it was actually one of the most successful things that we did as part of the second part of the project at the trailer board. There's not a lot to see. But this was it fairly early on. I think about Sprint 11 by the looks of it. And we evolved how we used it. It's not that exciting to look at, but it was extremely important to us. This is me writing stories for the Sprint board. So one of the things that we did was we wrote blogs about everything we did in the course as you do when you work in a museum as you try to use their collection to illustrate your blogs. So there was a team of six of us. We were essentially self-forming because of the situation that I was talking about before. We all had varied experience in digital or with web or with agile and those sorts of things. But we were all audience-facing and primarily digital people. So from web publishing and digital collections, we all had a shared common goal because we were self-forming. And that was extremely important. We did have some agreed roles. I was a scrum master and also a search lead. We had a product donor, which was important. We had a tech and design lead. We had a traditional PM, Project Manager, which Jen Marie always gets upset about when I say that. She was essentially, we needed a traditional PM on the project because we're a waterfall organization and there were some things that needed to be done but in a traditional fashion. However, she did not act like a traditional PM at all. She did what she needed to do. Everything else was agile and contributing to the team. And we had content and IA leads as well. All the people were T-shaped. We all worked together across the team. We all mucked in when needed. And that meant that we were building up our skills and everything as well. And everyone was on the same page. There wasn't somebody going off doing something, coming back and updating the team. We agreed principles and we agreed team rules, which we found was really useful. And retrospectives became extremely important because this was a very visible and a little bit stressful project because it's our main corporate website. So it gave us the opportunity every two weeks to nut through any problems that we needed to and figure out the way forward and keep all of our team productive and the morale high. We talked a bit about information radiators. Using stickies and stuff is nothing new, necessarily to a lot of people, but actually making sure that everyone can see them, the rest of the organization as well, is something that was new to Papa. So the fact that we worked on digital principles before we started designing anything was new again. We had a team charter, which we stuck to, generally. And we tried to project all of this to the rest of the organization as well. So in our team charter and the principles, we spread around the building. We put the principles on GitHub. We made sure that we could be held accountable to the things that we would be doing. So we had to have governance and, of course, there's lots of stakeholders across the organization for a project like this. We had very traditional governance to start with and I'm sure nobody would be too offended if they hear me say that it was heavy, inefficient and difficult to align reporting. We did, we found a balance over time. There was, again, there was the occasional hippo. Who knows what a hippo is? The highest paid person's opinion. And the idea of obviously a team that's doing things based on data and evidence and experience and is a very flat structure in the project team, hippos can really throw things off, but that happens with governance sometimes. It was rare, but we had a couple of occasions. We generally had mandate and trust, though, to be able to work in the way that we needed to. But there are difficult decisions. There always is for something like this, but they're not necessarily that hard, although some are. We use data and evidence as much as possible, particularly for conversations that we knew we were going to have that were going to be problematic with parts of the organization. That's the things that we needed to communicate. But there were some processes, some difficulties that slowed down process. Again, that's just working in a waterfall organization with traditional governance and things like that. We did better than we would if we were just purely waterfall. Occasionally, we fell back to hierarchy rather than data or user need. Those were painful times, but we just had to push through them. And this was us, again, using stats to justify why we were doing things and very much being user-focused rather than business-focused or looking for imaginary goals. Again, about data, data, data, so we pushed this a lot. We did need to build trust internally, so topapa.gov.nz is our main digital face. Everybody wants to say, and it's fair enough because it is really important, everybody thinks they own it, which is actually much harder and not practical. So again, we fell back on data and we fell back on the users. And you can't argue with winning with our users. And so that was what most of our arguments were about. Users first, not individual, but it's just units. Therefore, we're not competing against each other. But we did start with a huge amount of stakeholder consultation before we even went to procurement or anything. So everybody was able to have this say. We needed to know what topapa's expectation was. And we needed to focus. This sounds kind of obvious, but agile forces focus. And because it means it shows that you can't do everything and you don't try to. We needed to deliver the things that would deliver the most value to our users first rather than some of the superfluous things that we might have done thinking they were reported if we didn't test them or anything like that. Approximately 70% of the visitors came to the main site to plan a visit. That was the data that we had going into this. And that was the way that we had to decide our focus. That's our biggest audience group of the website at that time. We couldn't do everything, so that's where we focused. It meant that we focused on particular city users and we will come back to the other stuff because this is not a project, this is a product. So once we launch it, that's not the end. There will be a phase two and a phase three. So we will come back to it and we keep telling people we will come back to it. This is really important because we had to keep pushing this. We can't do everything. So all of this needed a huge amount of communication. And it was much bigger than we thought it might be, but it was so important that we did it anyway. We wanted to ring the organization with us. We wanted to talk about process. There's no way that we were going to change the way that Te Papa works with some of these things just by doing it ourselves and not telling anybody. So we did a lot of sharing. We had an internal blog that had reasonably good readership. We did all of staff updates in person. We did individual business group meetings, one-on-ones, et cetera. We didn't have a dedicated comms role. I think that's actually okay because we spread the load across the team. Everybody could talk about the things they were working on or how they were finding it, but it was a lot of work. And the trailer board was part of it as well. So we started right at the beginning, again, using collection items, talking about the fact that we were going to try and do this differently right at the beginning. And of course, user focus is a big thing. So as I said, it's our first real attempt at making a site that was for our users, and it is hard. It takes a lot of work. We need to ask the right questions. We needed to advocate for the users. Internally, you know, that sounds weird, but you actually have to do it. We used personas, which were great for focus and communication, and those personas were not made up. They were based on real data, real visitors. And so because of that 70% of the visitors being, the traffic to the site where it's about visiting, that's where our personas leaned as well. If we were doing another part of the site, we would change our personas. So as part of our transparency again, we put all our personas in GitHub. Anyone can see them. They will evolve over time, or we will have different sets. But again, we were making sure that we were being held accountable for the work that we were doing, and that we weren't just saying these things for the sake of it. So there is a link to the GitHub repository on there, which is a little bit hard to see, but so have a look at them later. I'm not gonna go into a lot of detail about them, but they were there, and they were real, and we used them. They were plastered around the wall as well, so that we were always going, I don't think Wendy would like that. Okay, so user testing. This was again another really big thing for us. We tested over 1,800 people online, on-floor, in cafes, most of them really quick, and this was the six of us. We didn't have a big team of people. We had some that were up to an hour. By doing the testing, we learned a lot about, obviously, what we needed to know, but also we learned a lot that we could reuse later for other projects, because there was so much information that the visitors wanted to give us. We did all the user testing you would expect, but including accessibility things, so like close tests for understanding of complex sentences and making sure that everybody can actually access the words that we're using. We did automating accessibility testing as well. We did lots of interviews. One of the good things about being here is we have no shortage of people to talk to. 1.8 million people through the door last year, so we just rocked on out and we asked people, and we're in a very privileged position where people want to help us. It did mean that we had to sometimes prompt them to be a little bit harder on us, and they were going, no, no, the site's fine, and we're like, no, it's not. Tell us what's wrong with it. But, yeah, and it can be scary, but it's obviously, as we all know, incredibly useful, valuable. Sometimes the testing went well, and sometimes it didn't, but that's okay. That's kind of what testing is, and we told everybody about the failures as well. When we did our first sort of real user testing, the user journey-based and task-based testing sitting down with someone, we failed horribly in a couple of areas that we had designed. It's all right, we learned it much earlier than launch. So we talked about that, and we talked about the idea of that being a really good thing. We told everybody about the tests that we were doing. We've run through why we did them, what the results were like. This is all part of the internal blog. So that, again, people could see that we weren't just making decisions that we thought were right. We were actually talking to our users. Content is another huge area. So we went, as I said before, the content had grown organically over a number of years. We had 4,000 pages for a corporate website. This is not collections online or the exhibition sites or anything like that. It's just purely a corporate website. So we got it down to 350 with a huge amount of work. People were very protective of their content. It got in there for some reason because somebody thought it was useful at some time. So we had a number of questions that made it pretty obvious whether it was still needed. Why is it there as it being used? Analytics don't lie. If the page hasn't been accessed in six months, it's probably not needed. Is the content still accurate? A lot of it was out of date. There were staff members who were dead who were still on the site with contact details. And so the numbers don't lie and everyone does want the content to look good. So with a lot of work, hand-holding and things like that, we worked with everybody who owned content and with experts and with outsiders and with users. We created new content. We merged content. We improved content. We enhanced content. We did all of that kind of thing. But we also, at the same time, we worked on Tone and we did a lot of user testing on Tone. What do you think Papa sounds like? And what our users think is often quite different to what we think. So we did work on that. And we told everybody about what we were doing. So there's more internal blogging. MVP. So this was and remains contentious. The idea of minimal viable product is not a well understood concept here where the goal for most things that Papa does is to put the absolute perfect final product on the floor, which in terms of digital, I still think is chasing the unicorn. So we prioritize the values of, you know, we do the things that you're supposed to do when you're doing an agile digital product that has a longer lifespan than just launch. We prioritize the features of most value to our users and deliver those first. That's what we need for MVP. So that we have the most important things delivered before we run out of time and money, which often happens with big sites like this. And then we push the idea of phase two or continuous improvement or however we're going to manage it. Having a really active backlog that we could point to, we were recorded all of the features that people had asked for, but weren't prioritized to the stage, was super important. But we still have people coming to you saying, I told you what I wanted 12 months ago, and it's like, yeah, that's what you wanted, but none of our users wanted. So it's not going to make it into MVP. When will I get what I want? Does MVP mean that it won't work? No, that's not viable. You're missing the V there. And the idea of it having to be perfect to launch. There's the unicorn that we've been chasing. So, and I put that in there because this is the blog about MVP in the backlog, constantly reminding people that there will be a backlog when we launch. We also had to work with three external vendors because of the difficulties we had with the RFP process. There was little belief Te Papa would be agile, and so, but we were committed to doing this, and we said to the vendors, we got them all together and we said, okay, we want to do this properly. We want to communicate. We want transparency. We want everybody to be on a team. We don't want things being handed over to another team or, you know, us being caught in the middle or anything like that. So we're a team, we're partners, and clear roles and quick decisions made as needed didn't always work. We did have some failures, there's no doubt about it. We had real problems, but we came out with some great new partners. Okay, so we had a sustainable, secure, up-to-date accessible website with all the things that we needed to achieve from a technical perspective, get rid of our burning platforms and everything. So it's good. It's our main digital face, and it is a new way of working. We saw it as we need to treat this like we do our other large products that we release. So we had a blessing. And our commator at first was like, okay, I don't know how to bless a website. But after figuring out that it wasn't that it was blessing the people who had worked on it and people who were going to use it to visit and all of that sort of thing, you know, understanding it as essentially a digital to papa, it made a lot of sense. We even sang Waiata, our little digital team. The vendors came along and were very humbled and overjoyed to be part of it. And this is again, saying digital is not an add-on. It's something really important and integral to what we do. And we did more communication internally again about the work that we did. 1700 people, 4,000 pages, 347, and most importantly, over 10,000 Slack messages. We had a high-key period afterwards, which we hadn't planned, but Melissa Firth, who is our new chief digital officer, she came in about halfway through the project and she said, you guys need a high-key period, which is where the team stays together. We had it for three months after launch for this particular thing. We retained budget and we retained the ability and the focus to go through, continue going through the backlog and priority, but also to watch the metrics after launch. When it's out, make changes based on the metrics that are coming in. That's, again, MVP is great. You're not gonna get everything right. We do any immediate fixes we need. We had two areas that we had put a lot of work into and it didn't work. We made changes. You're also supposed to start moving to BAU, but we didn't actually have a BAU. Business as usual. So it's like, again, not just dropping the site and expecting someone to just kind of look after it or whatever. Have a plan for how it's going to evolve, be looked after, be managed and have continuous improvement. Again, we were going through another restructure and change of roles and we lost momentum and this was the biggest failure. But the biggest disappointment because we could actually see it happening and we couldn't stop it. So not an excuse, but yeah, anyway. But after, not long after, we did have a new dedicated content team and they've done some great work since and they've learned some things and they've tried a whole lot of interesting things since. So we do have to come back to think about what's phase two. Or BAU. We've done some trickle development very slowly to keep our vendor engaged and the really key people that we needed because they were so good. We have been trying content experiments. We are starting to plan the next phase. Alongside all of this has been a move to being hopefully some kind of digital to Papa, setting up a digital team, setting up some new roles and also a framework for developing digital products. No more projects, no more anybody who's got a digital idea can start doing it and dropping it or whatever. The digital product development framework is something that's going to be very important in the upcoming couple of years and we had a little bit of experience working with the website project that we could feed into it. The DPD, if users agile lean and design thinking tools, all great and all quite new to Papa. More transparency, blah, blah, blah. And we're also actually looking at a working environment as well, which is something else that came out of the lack of being able to collaborate together or have spaces where we can show our work and things like that because we have an office, a back of house office that was designed in the 1990s. So we're fixing that now. So what we learned, this seems to be the two slides of the most important. We have a lot to learn. Agile is difficult in a waterfall organisation but it can be done and it's definitely worth trying it if you're able to. Strict roles, particularly the product owner and the scrum master are hard to do without mandate. Really, really hard to do. So it takes a lot of effort. Retrospectives are extremely important because you need to look after your team health, particularly for long and very visible projects. Lo-fi testing is amazing. And the more you can do the better and the more you can do in the team rather than farming it out, even better. Build the skills in-house. And it is possible to run Agile into Papa. It's just difficult. Communication and transparency with your organisation, not just the noisy stakeholders. For these kinds of things, it's really important that everybody understands what it is, why we're doing it and that everybody feels that proud when it's released. Handover or staff continuity is vital because that gives you the momentum that we lacked. Take your learnings onto the next project, of course, please, and create the new normal from all the things that you've learned. Because this is all we were trying to do. These new ways of working are not about trying to be trendy. When you say Agile, people think, oh, they're just trying to be like Google or trade me or whatever. We wanted to make sure that we were building the right things for the right people in the right way, not wasting resources and looking after our working culture, which is extremely important and our working health. That's it. Does anyone have any questions? We've got one minute, apparently. Anybody, anybody? Hi, you said that people who might be obstacles ended up with their name on the board where everyone could see it. How did you deal with that? We didn't put their name on. We put what the problem was. That idea of being forced to write down what the risk is in an objective way or without getting personal domain or anything like that was actually really useful because it then tied us down to what is the problem, not who's the person who's annoying us or anything like that, so it was actually really useful.