 Kia ora koutou. Sorry, I'm just standing or something, but it's sticking under my foot. Yeah, sorry. Kia ora koutou. Valerie and myself were part of a very successful data migration project at the Alexander Turnbull Library. While every project is unique, there's some key factors for success which we want to share with our colleagues within the Glam community who may be considering a similar project in the near future. Firstly, some background. Though I think most of you might know this already, but... In 1918, Alexander Turnbull gifted his collection to the nation. As you note by the year, next year will be the centenary of this gift. And in 1920, the Alexander Turnbull Library was established as a research library, and it's based on the Turnbull's collection, which includes not only books, but a vast collection of manuscripts, artworks, sound recordings and photographic material. And in 1965, the Turnbull Library became part of the National Library of New Zealand. We've remained there since. Okay. From Tapui to Tiaki. People probably forget these dates, but actually Turnbull's first collection management system, Tapui, as it was called, it was actually an acronym for Turnbull's Automated Project for Unpublished Heritage Items was launched. This is also Te Reo for Taneucha, but it's also a name of a well-known Wellington tugboat out in the harbor. So, yeah, 1992. We're a pretty revolutionary back then, as well, with the system. And in 2000, the OPEC went online. And 2014, which is a project we're talking about today, Te Arapanuku commenced, and this is Te Reo for the Path to Move Forward, and it was to replace the Tapui system with a modernised system. And this system, Tiaki, as we called it, Te Reo for to guard, to protect, and also a name of another well-known Wellington tugboat at Anahaba, which is actually two pieces of software, EMU and IMU, colleagues here at Tepapa use EMU, but we call it Tiaki, as one. Key 1, communication. It's absolutely essential for any project like this. You need to make the reasons why, how, and when the migration is going to happen, particularly for our colleagues at the Turnba Library, and particularly with Tapui. 1992, that's a pretty old system, and some of our colleagues had been working only with Tapui their entire professional career, so you can imagine a huge emotional attachment to a system, and that does happen for that period amount of time, and a nervousness about moving on to something new, so you have to make your reasons incredibly explicit, well understood. And a major migration project can be a major change management process. Yeah, you need to build confidence and trust within your organisation, and you need to make sure that the staff are kept well-informed and have the ability to engage with the project, and this is on our intranet. This is where we, was one of the various means that we use to communicate about the project and make it open and transparent to everyone what was happening. And I want to just note there were several unsuccessful attempts in the past to actually replace Tapui. You clearly see by a system that was over 20 years old by the time we were looking at replacing it, there was quite a lot of issues with a system that old. It had become, and staff were very aware that it was increasingly becoming very unstable and it required urgent migration into a modern platform. It was a bespoke system that had been built for us with only one other installation, similar installation in anywhere in the world, and that issue was Hakina at Hocken Library. But ours was a much more sophisticated version of that, and it was based on a PIC operating system, and most people probably don't even know what that is these days. You couldn't do, and it was all based, or any commands within the system as MS-DOS commands, so there was no control C, under no circumstances could you do a control C, it crashed the whole system. Lacked Unicode, and it just couldn't talk to any modern systems, and it was developed in 1992, well before any developed international standards for archival, particularly archival descriptive and encoding standards. And it was by 2014, both internally and externally with the vendor, we had very little support for it. So we kept regular contact with staff through email, through the internet, meetings, newsletters, and staff had a lot of opportunity to provide feedback, ask questions, be part and participate, actively participate with the process, whether it's subject matters and in workshops to capture staff knowledge, and this moves on quite nicely to the next one. I can't underestimate this by any means. This will be the most visibly obvious factor to both staff and to users if it's not successful. Just show that's what Taupui looked like in the staff end. Basically to understand your data is to understand existing and historic business processes. Existing system manuals and documentation is only useful to a limited extent. For instance, latest versions may not have been up to date and previous versions lost, or in our case, we found that many staff decently did not use the manual. And this may be a case for a lot of people, you know, that's something we said. We did interviews and workshops with staff and we're very fortunate, like I mentioned before, that we had staff who'd used only Taupui through their entire professional history, but we also had people who were involved in the implementation of Taupui so that understood and could remember why, when, or why a particular field was created, modified, abandoned, or when sort of thing. So it was really helpful. Over the history, many changes were made, so understanding the background and the context to that was incredibly useful in making decisions regarding the data migration. And a lot of people only saw that first screen, but this is actually the structure of Taupui. It's quite a high-level structure, to be honest, but it gives you an idea of how complex this data, this CMS was really. It was very, very complex. And to understand, the next really key thing is also not just to understand your own data, but if you're wanting to migrate your data into another format or another kind of standard, is to understand that system really well, or understand those standards really well, should I say. And particularly we wanted our publicly viewable data, which is our, sorry, is that right? Yeah, descriptive records, which are these ones here, and all our index records, to comply with two specific sets of standards, and people in this room may know those already, which is ICADG, International Standard for Archival Description General, and ISAR CPF, which is International Standard for Archival Authority for Corporate Bodies, Persons and Families, and their corresponding encoding standards, which is EAD, Encoded Archival Description, and EACCPF, which is Encoded Archival Context for Corporate Bodies, Persons and Families. So that's a bit of a mouthful. One thing that's really, really important to remember is not to expect your vendor, whoever that might be, to fully understand the specifics of every standard and the standards that you want to comply with. Just some very simple instances of this with EMU. They complied with EAD, yes, but it was a very simplified version of it, and in order for our data to be migrated successfully into it, we almost had to use the entire suite of available tags for EAD, so they used basically the full tag library, essentially now. And EMU, though, was not initially compliant with EACCPF, together with a vendor, we enabled this functionality with EMU. Next part. Once you understand your data and the data that you want your data to be expressed in, I guess you need to develop some really, really comprehensive data crosswalks. I can't. This is a huge time saver, but you might think it is a huge amount of time initially to invest into it, but it saves a huge amount of time when you're coming to do actually your data migration. This basically, we developed our detailed crosswalks in preparation of the project. So well in advance, I think it was, we were looking at, we had developed several different crosswalks in those previous attempts as well, actually, so there was quite a few crosswalks, but the one that we finally worked on was far more detailed and it really, really helped. Yeah, I just can't. Yeah, because basically, you'll see why, this is what Tapui Rural Data looked like, okay? The human staff, not even, I think even when we looked at it, we were really, oh my God, how do we deal with this? But essentially, having those crosswalks from the raw data, we were able to successfully do this. Oopsie, where's my slide? Sorry. XML. And we went to a workshop, I went to a workshop with a colleague yesterday and talking about just having XML, CSV, Tapui couldn't export any of that kind of thing into that spot Tapui data looked like. So anyway, so without it, you really, really, really need a decent crosswalk. It saves a huge amount of time. The next one, people will know about this one as well. Data cleaner. Even then, once you've developed those crosswalks, you must accept that not all data will be migrated 100% accurately. I think you need to be realistic with that and actually make sure the expectations of your staff and your users know that as well. And you have to make some really hard decisions about what's the best for the majority versus what's the best for everything. It's a hard decision. I can get into more details if you ask me later. You're going to have to do a combination or maybe all of them pre-migration cleanup. We receive anomalies reports from the vendor so we're able to rectify the data which we're creating, you know, those crazy loops and data where it was linked to itself and it was linked to itself. It was able to identify that for us. And data during your migration, so we developed really detailed migration rules, some very complex rules as far as some of the migration mappings went and that helped to do that during the migration rather than doing it pre-migration. And then, of course, post-migration cleanup. One thing that was really fantastic for us is one reason why we didn't do a huge amount of pre-migration cleanup was because we simply couldn't within Taupui. We couldn't do bulk updates. Some of the things that we can do now in the new system which takes Val and I maybe 30... 10 minutes to do. It now takes to, you know, a minute. It would have taken years to do, should I say. Sorry, I'm moving on. Sorry. Key number three. You cannot expect to continue operating as usual. The entire business will be affected by a migration project and the business needs to be agile to cope with the changes and all staff will be affected in some way, whether they're intimately involved or just supporting staff who are perhaps taken away from their BAU to support you. And you need to provide project workers a space to do their work. And migration can be absolutely transformational to a business. It can be quite humbling and we found it anyway with our system from go. It was a big cultural shift anyway. It's a really big cultural shift. Sorry. I want to rush through my notes. Sorry. Sorry, that's my last page. We developed future business processes with the idea, I think, of new and better ways to do things and the reason for, because we've always done it this way, was no longer good enough. It can be quite enlightening and something alluded earlier this morning, actually, but also can be quite half for some people to realise that some people's business processes would really just work around, you know. So that's something to be really mindful of and you cannot underestimate the staff resources required not just during the project but before and after. And this is something that probably Val will go into next. Sorry, I took your line. So much, Kirsty. Kia ora koutou. So I'm going to walk us through the last few keys to successful data migration. So key number four is getting the right people at the table and this is so fundamental because the success of any project is as much dependent upon the personalities involved as it is with the technical skills and expertise. So we work in a government setting, so we had a really big project cohort. Lots of different people, business analysts, system architects, application support specialist, a testing cohort and then the library staff as subject matter experts. We needed that big project cohort to make sure everything integrated properly but it was also really unwieldy at times. It was hard to get everybody in the same room and on the same page. So a lot of the data migration work which Kirsty has discussed, setting up those rules, looking at the standards, making sure that everything was in place happened on a much smaller scale outside of that big project cohort. It literally was Kirsty and I and maybe one of the two other people who were joining us in a small meeting room where we hashed out the really tough questions and we consulted with people as needed. We worked really closely with the vendor, we worked really closely with our team leader and then we reported things back to that larger testing cohort and they asked us really tough questions and said, are you sure? And we said, we hope so. So a key element of that working relationship was trust because the wider project cohort and the rest of the library trusted us to do our work and to follow the standards and to get everything ready on time and to meet those deadlines and they gave us the space to do that work which was really, really vital. And also project members and wider staff were empowered to ask questions along the way and they thought of things that we hadn't. So that was really, really useful because it helped us address those issues before it was too late. So I just want to speak super briefly about the importance of community here at Tapapa today. The Tapapa community was amazing to us. They've been long-time emu users. They've been using the software for over 10 years. They answered so many of our questions and gave us tips and tricks and insights. We went to a users conference and met other emu users. This was all invaluable. So the more conversations you can have with people ahead of time, the more issues you can address before they turn into big issues that become overwhelming. Also, other institutions are now looking at our adoption of the EACCPF standard and all of the customisations that Kirsty especially designed for the system. So it's kind of great to be able to give back to that wider user community as well. So key number five is that the end is also the beginning. I've worked in a number of institutions where they're like, okay, we're going to launch a new website. We're going to do something. And that's the end. And that's not the end. That's where so much more work begins. Kirsty has mentioned that we've been doing a lot of data cleanup. And I think both Kirsty and I thought that once we went live, we would go back to our regular jobs and that didn't really happen. We have been unbeknownst to either of us. We entered that high care period of the system, that figurative fourth trimester where we had this new thing and then, oh my gosh, we have to take care of it and we have to be responsive to all of the things that people are telling us are different than they expected or all the questions that staff have about using the new system. And that was a massive amount of work. And I don't think that I personally was not so prepared for how much work that would entail. But what we did was we set up a Tiaki Help email address for staff. We held training workshops. We wrote instructions and documentation. We helped coordinate a staff software users group so people could address questions and issues and say, hey, I want to be able to do this with the new system. Can you help me to do that? We developed reports and just helped staff manage the types of queries so they could do the reporting that the system enabled them to do. There was so much more functionality in the new system than they had been in Tapuwi. And so we really had to help staff to learn how to use that new functionality and just show them the possibilities. So a major... So while there is always more that you can do, it's also really important to celebrate those milestones along the way. A major post-go live achievement was the release of two-turned-by-library data sets. One which contained all of the library's descriptive records for collection materials. And one which contained all of the library's name authority records for corporate bodies, persons and families. These are available freely on the data.gov website. And they're also available on the open data page of the National Libraries website. And we're really excited to allow users to interrogate our collections in new ways that were never possible before. Additionally, we've upgraded to EMU version 5.0 in November last year, and we now have the ability to use Unicode. So we've been able to add macrons and special characters to our collection descriptions and our authority records, which has been really, really great. Also prompted much more data cleanup, which is still in progress. But we have updated our EVHAPU list, our Fari Nui Authority terms, and are working our way through others. And we're currently working on improvements to our front-facing collections website. And what you can see here is a preview of what it'll look like in a few weeks when it goes live. There's a more user-friendly interface for accessing digital content, and we now have thumbnails available, which is a new development. So we're really interested in feedback. So if you use Tiaki or if you use our data sets, please let us know if there's things that you like or if there's ways that we can make it better. So in my final minute of the presentation, I'm just going to offer you a special bonus key just for today. And that's... I actually think it's really important, and so that's why I threw it in, even though you were only supposed to have five keys. But key number six is to honour and acknowledge your limits. Because there may not be a finish line, but actually sometimes you need to take a break. And I think this can get really overlooked in the digital sector. So IT projects often have high expectations and tight deadlines. System transformation and data migration in particular can be really stressful because they impact the organisation as a whole, and everybody is depending upon you to get it right. That's a lot of pressure. Nobody wants to lose data or compromise privacy or have something migrated to the wrong place or have it migrated in a way that people don't... the people who need access to that data can't access it easily. And for this project, we were migrating 25 years worth of data, including some 900,000 descriptive records and about a quarter of a million name authority terms. And we were migrating them warts and all. So that was really intimidating in some ways because if there was something weird in our descriptive records, everybody was going to be able to see that weirdness. It meant we could clean it up much easier than when it had been that black box of tapui data that was impossible to interrogate. But it was easy to take on that stress. So I will just say that it's really, really important to talk about stress in the digital sector and the physical and mental challenges of caring for our digital cultural heritage and the data data about it. We're constantly being asked to innovate and be immediately responsive and it can be stressful and overwhelming. So it's really, really important to take care of yourself and it's really, really important to support colleagues and times when they're feeling stressed as well. And so it's not just downing another coffee, it's not just pushing through physical pain or mental fatigue to make that deadline. It's actually taking care of yourself to make sure that your brain is actually as sharp as it could be and that you can get the job done well. So since we're out of time, I'm going to leave it there. We really do hope that our experiences and lessons can be of use to you and we're really happy to continue the conversation as well. So feel free to talk to Kirstie and I at any point during the conference or you can drop us a line via email or Twitter. And thank you all so much for your time. Thank you so much. It was great. Thank you for a couple of questions for those that are staying in the room. Are there any questions at all for Kirstie? Hi, you moved from a system that didn't have a lot of capabilities to one that had lots of capabilities. Have you had staff coming up with new ideas, what they can do? Have they moved into that new mindset? Absolutely. That's one of the parts that Val was alluding to. What we call a tiaki users group. And that's a space where people from throughout the business can raise particular ideas. Yeah, but actually and one thing that we discovered is people were afraid to actually because they were so used to the old system where there was no flexibility that people didn't ask the questions. They were just assuming what they got on when we went go live was what they got. I mean like had to really actually change that culture and say actually no, what you got isn't it and in fact we really want to get your feedback and improvements. Yeah, so we've been trying to encourage people to think about ways that they could use the system, think about the questions that they have and then we do a lot of brainstorming to say is there a way of meeting this need within the system and a lot of times there is and when people discover that they get really excited about it and they're just like wow this is great but it has inspired quite a bit of creativity and I think people are starting to really embrace that. That's great, thank you.