 Wow, it has been doubt. Kia ora koutou. Ko Kali Skelton ahau. Ko Tamiki Painga hira ahau e mahiana i te Ropu, Mata Hiko. So I'm Kali Skelton, the Head of Digital Experience at Auckland Museum. I'm really excited to be here and share with you today some insights from our Hei Taunga Māori audio guide. Just to give you some background. Before I started at Auckland Museum, a mobile experience discovery project was undertaken and that was to explore what a bring-your-own-device mobile experience for an Auckland Museum visitor might look like. This was done in mid-2016 and from the discovery work came three projects that the museum prioritised. So we redesigned our Wi-Fi login to include imagery and much less copy. It was just a bunch of terms and conditions, which was terrible. We created a landing page for visitors so when they're actually at the museum after they join the Wi-Fi, they see a page that provides on-site specific content about the exhibitions and maps. And the third project that we're going to cover off today is prototyping a BYOD audio guide mobile experience. So what is an audio guide? Well, this is what Google says. And I think that's it at a really basic level. What's missing is the value that an audio guide can provide. So they can really provide additional layers of storytelling by offering visitors an alternative way of exploring rich interpretive content. And in addition to that, if the content is available in other languages, they make it easy for non-English-speaking visitors to access those stories. And what's also missing is the effort that goes into producing and rolling out an audio guide. Getting the first one out the door is actually a really big project. So before we get into looking at what we did at Auckland Museum, let's take a look at a few other audio guides. And I was really fortunate to travel to Australia and the United States in the past year, and I saw some really interesting examples that I'd like to share. So the first is at the National Gallery of Victoria in Australia, and they had the Dior exhibition. And there was a cost to use this audio guide, but devices were provided. And it was an absolutely stunning exhibition. It was packed. There were way too many people in there though, but the audio guide was really great. And rather than just looking at a bunch of frocks and reading labels, I found the audio guide made the experience much more engaging and interesting and provided me with insights and additional layers of content that I couldn't get just from walking around the exhibition. At MoMA, which is the, as everyone knows, the Museum of Modern Art in New York, they have an audio guide that spans across multiple galleries, and there's multiple guides within that. Their guide is free, but they also provide devices, so there's no charge. Interestingly, it also comes as a downloadable app that you can download onto your mobile phone. And they translated it, into 10 languages. But unfortunately it didn't work. So you can see the sign says that they're all in use, but clearly they're not, because there's a bunch of them sitting on the desk. They actually had issues with their software. So this really highlights the risk associated with providing devices to visitors and running software. Then the 9-11 Memorial Museum in New York. So they've got three guides or tours. They also provide devices, and the guide is available as a download on your mobile device. It's in eight languages. But the super cool thing about this audio guide is that Robert De Niro is a narrator of the Witnessing History Tour, and it was really moving, as is the entire museum, as you'll know if you've visited. It's actually worth downloading the app and having a listen. We visited Bowie Exhibition at the Brooklyn Museum in New York. So this was a touring exhibition, and this was the last stop. And the audio guide was actually provided as part of the exhibition with your ticket entry price. And so everybody was given one upon entry. And this really made sense for me, as an experience within this exhibition. There were lots of clips of David Bowie talking and snips of his music being played as you were exploring the exhibition. And I went with a friend, and we both really enjoyed it. We kind of did our own thing and made our own way around the exhibition. And it really added so much for me. But I've spoken to others who really didn't like the audio guide being part of the experience. They visited with someone as well, but they wanted much more of a shared experience. So I guess a learning here is that audio guides can be detrimental to a shared social experience. So it's a real factor for consideration when you're thinking about what problem an audio guide is going to solve for your audience and who you're targeting. And the last example is a talk I attended at museums in the web conference in April, this year in Vancouver. So this is really interesting. This is a collaborative project between MoMA and SF MoMA. So these two museums collaborated on the same exhibition. They focused on the same artist. And they partnered with the same creative agency. So you might think that they'd end up with nearly identical audio guides. Well, they didn't. The audio guides each have a unique and distinctive point of view as the interpretation was developed separately by each institution. So MoMAs was much more formal. It focused on the artist's work. Whilst SF MoMAs was more informal and the focus there was on the personality of the artists. So what's key here is the opportunity to set a tone of voice within the audio guide, be it really fun or playful or more serious. And this is what we're going to explore more at Auckland Museum. So let's take a look at the audio guide that we piloted. It launched in February this year and it took a long time to get on the floor. So the project had already kicked off when I started at the museum in December 2016. And it took us from that point 14 months to launch this. There are a whole bunch of reasons why and we'll cover off some of these as we go through. So we ran it for four months and it finished in June this year and we ran it for that timeframe so that we could actually do some evaluation. The audio guide's still on the floor and it's still being used by visitors. And our target audience was really international visitors and those who brought in their children aged between 7 and 14 and mainly English and Mandarin speakers and solo independent adults. It's available in English and Mandarin and there's three versions. So the adults version is either a one hour or a 30 minute guide and the children's guide is 20 minutes and then the number of stops reduces by the length of the tour. So essentially there's six versions in total as each is available in English and Mandarin. So we decided that we weren't going to provide devices so visitors have to use their own device to access the guide. So it's a bring your own device experience and it's free. So part of the pilot has actually been to evaluate whether we'll charge visitors to use audio guides. They can purchase headphones from the store for $8. And we chose a local off the shelf audio guide product working with Glen Barnes and his team at my place. And just want to point out the reason the guide took so long to roll out the software has been really good. And we decided to use the web based app version so visitors access the guide via our website. We didn't want to make visitors have to download an app. We promoted the guide in our daily whats on so visitors can access it via URL or scan a QR code and we worked really closely with our visitor services team just to make sure that the hosts who are interacting with our visitors were briefed on all the components of the guide so they could really help visitors out and answer any questions. And our marketing was also a really core part of the team. They made sure our internal and then also visitor face and comms were all covered. We also featured it in our Mandarin daily whats on. So let's take a look at the actual guide. So visitors arrive at a landing page and just for the span of the pilot we asked them for their email address just so we could send them a survey and this was incentivised so they went into the drawer to win a gift hamper from the store. They then chose their language so English or Mandarin and the length of the guide 30 minutes one hour children's and then they started the guide. So there's a menu for the next stops if you want to skip forward or back and there's also a map to show where the stops are within the gallery and this is a view of a page within the guide. So for every stop and there's 19 in the one hour there's an image an audio file a title and a transcript and the same for the Mandarin version and then there was a separate version written for the children's tour. And actually one of the challenges we faced was content. So at the start of the project we worked with a company to write the content and they were based in Australia which was actually really challenging because the content was focused on specific Maori taonga at our museum and really our curators CNI developers and writers were best placed to write and produce this content. So fortunately our team did end up doing that which was the right decision in the end but it was a real shame that we didn't make this decision early on because it would have saved us a lot of time and a key learning here is more consultation with key stakeholders early in the project to make sure these decisions are being made right early on. And we commissioned Anna Coddington who's a New Zealand singer-songwriter to record the audio for the English Guides so take a listen. No mai haire mai. Welcome to the Māori Court. Here you can discover the richness of Māori culture through stunning taonga or treasures from magnificently carved houses to intricately woven cloaks to beautifully crafted jewellery and the tools needed for everyday life. So let's get started. Then of course there's in-gallery signage so we work closely with our in-house design and production team to create and place the signage for each of the stops and some of those were relatively simple. Others had to be on the outside of the case and in this case one on a plinth in front of the pātaka and one of the difficulties and challenges that we face was that after the audio guide launched we were made aware that one of the objects in the gallery had come off display. Not this one obviously. It was stop 15 and so unfortunately we found out after the object had been removed from the gallery so there was a bit of a scramble to update all six guides and it wasn't as simple as just removing a stop as each stop includes audio that leads you on to the next one. Here's Guy. Hey Guy. He's our digital experience manager. So he performs a magic to update both the English and Mandarin guides and you can see now stop 15 is sitting at his desk. But what it means now though for us is that the hour-long guide that includes all the stops one through 19 now goes from 14 to 16 which isn't ideal but it still works and it's another learning for us so for guides that include objects within permanent galleries what's your plan if one stop has to be changed or removed. So the app needs to be updated and the audio may need to be as well especially if that audio leads you on to the next stop. Let's take a look at what we actually discovered at the end of the pilot and I'll note about the evaluation. It was pretty tricky. We have a great visitor marketer market research team and we worked really closely with them but early on we realised we couldn't do any face-to-face research as we didn't know when people would actually be using the guides and we couldn't have somebody standing in the gallery all day waiting. So we decided to run surveys also in English and Mandarin and these formed the basis of the evaluation along with the analytics that we had running on the app as well. So over the four month period we had just over 700 users and of those about two-thirds used the English guide and a third Mandarin and we received about 60 survey responses so not a huge amount but it was enough to get a sense of what visitors thought and two-thirds of users were international visitors and they were primarily from the USA, UK, China, Germany and Canada and the uptake from Aucklanders was also strong which was interesting. So how did they actually use the guide? Well, not surprisingly most people used it alone and they used their own headphones and really only a really small percentage actually purchased headphones. In terms of which guides were the most popular well, interestingly there's discrepancy between the survey responses and what our analytics are telling us. So our survey responses are telling us that the adult 60-minute guide is by far the most popular but Google Analytics is telling us something different. So it's telling us that there's a really close split between the 60-minute and 30-minute for the adult guide. So this is likely to be a skew based on the small sample size of survey responses and so what we're going to do is we have an analytics dashboard that we're creating so that we can get deeper insights on an ongoing basis. One thing that both of these make really clear is that the children's guide has a really low usage and is not really being used much at all so we're going to retire that version. And then in terms of the qualitative feedback from visitors it's been great but maybe that's because you get more engagement from people who actually enjoy an experience who want to let you know that when they complete a survey. So 90% said it made their experience more positive they felt it was a unique experience it was really easy to use and they were extremely or very satisfied with the guide. They felt the content was interesting it focused on the right stories and it had enough information and unsurprisingly most people said it should be included free of charge with the ticket price they did tell us that they'd like more audio guides for other parts of the museum so that's good. And some areas for improvement we need to make it easier for people to actually get the guide so we need to improve our marketing for when people arrive at the museum they need clear instructions on actually how to use it the wifi dropped off for a few people we need to slow down the te reo narration and we need to provide English translation for some of those words and we need to help people find the stops better so good old stop 15 that's not there it's now or in our office so we need to fix that so the pilot really has given us great insights and feedback from visitors we're going to make changes to the Hetaonga Māori guide to make it better we'll remove the children's version we'll do research with our visitors to find out what other guides they would like to experience and after that we'll do more questions for Kelly this is the first session I've ever chaired in my life where we're well ahead of time that's fantastic that decision was made before I started I think it was identified that we wanted to capture tourists and see really what the uptake was from that market coming into the museum and that audience it's certainly something that we're going to consider for the next round so the kids to our stuff kind of interesting because I went on a school visit with my kids to the museum and we went into the marae but there's none of the interpretation stuff is kind of included is it possible to repurpose that content for school visits to increase the use potentially I thought there was something missing we have that content part of the decision to retire those guides is based around the fact that the usage is low which you would kind of expect anyway however to maintain and manage those and update those on an ongoing basis we just feel it's not worth the effort and we also are looking at what other experiences we can create for children that are more engaging rather than being talked to so something a bit more playful that they can explore around the museum I was wondering if a mobile charging station would increase usage of the app and perhaps headphone splitters at the shop would that help the sociability aspect yes potentially and it's something that when we do our research and VMR and ask visitors will certainly include those types of questions within that over the did you say two months the period four months and I was just wondering what was the roughly you know what the monthly visitation to the museum is trying to get a conversion rate it's about 100,000 a year a year okay a month sorry a month yeah thanks Kelly I just wondered what was if you can in a sentence tell us what was your main impediment to making progress more quickly you mentioned 14 months for production so yeah too hard Chatham House Rules it was it was a lack of consultation really on with key stakeholders and making sure that we were making the right decisions about who's producing the content and getting people lined up to do it and really good project management so yeah Hi Kelly I just wondered for each stop was there an average sort of time that you wanted the voice over to be or did it vary depending on the object she was describing it did we didn't want each stop to be too long but one of the things that we did find at the end was that for certain objects people wanted more time maybe listen again to the audio so there's a couple of stops I think that we need to adjust the timing and slow it right down and that was actually one of the key pieces of feedback from the children's guide is that they found the pace of that content too fast, too quick Kia ora, just following on from Amos' question did you actually calculate that conversion i.e. the proportion of your overall visitor numbers who picked up the guide we actually haven't pushed it so in terms of marketing it to visitors because it was a pilot we really wanted to keep it quite low key so that conversion rate is not really high but actually because we didn't push it a lot and a lot of other museums they have stands it's really obvious when you go there that this is a really key part of the experience we haven't done that, it's actually quite hidden so because we've seen that it's successful and it's working and visitors like the content we want to make tweaks, we want to make it better and then we're actually going to look at that on the floor signage so that when people arrive they can see and it'll be really interesting to see if there's much more of an uptake and what those conversion rates are Kia ora, thank you