 John Stack is digital director of the UK's Science Museum group. John joined the group in 2015 and he's responsible for setting and delivering its digital strategy. He manages the group's digital departments which encompasses the museum's websites, digitised collections, apps, games and on-gallery digital media. Prior to joining the Science Museum group he was head of digital at the Tate Art Galleries and during his time there he carried out a major strategy development which asked what would it take to be a truly digital organisation where digital was the norm and this work became a Harvard Business School case study. John's currently leading a project called Heritage Connector which explores the opportunities for computational analysis to build links at scale between collections. Though it's targeted at museums I suspect this will be applicable across other sectors as well and I'll be fascinated to hear about its outcomes. The global pandemic has brought the digital space into sharp focus for us all and the title of John's presentation, Managing Digital in a Time of Accelerating Change is I'm sure very central to the current experience of all of today's audience. So a little bit about the Science Museum group first of all. So the Science Museum group is five museums so it's the Science Museum in London, the Science and Media Museum in Bradford, Science and Industry Museum in Manchester, the National Railway Museum in York and the Alert Commission which is in Sheldon County Durham. So we have the National Museum and so we hold elements of the National Collection and the moment we're digitising all of that material and putting up online. I'll talk a bit about that later on. We're very active on YouTube and social media as you would expect. We also have a presence digitally on Google Arts and Culture and on our UK and we're in the process of uploading more of our content to Wikipedia so we can reach greater audiences there. So overall we're very digitally active. I've been there for about six years and a lot of the first few years we're really putting in place building blocks to an infrastructure to start doing more digitally and we're starting to see the kind of fruits of that now including obviously through the Covid-19 lockdown. So a little bit about the collections, very diverse collections. So if you said Science Museum group collection to everybody, they're immediately the first thing they would say is well scientific instruments and yes there are scientific instruments but it's a lot more diverse than that. So we have a large transport collection including Stevenson's Rocket and also the Flying Scotsman. There's quite a large historical collection, so going back to Greek and Roman times especially around the medicine collection. There's a lot of mass produced objects so cameras, record players, bicycles, all that kind of stuff. There's quite a substantial art collection as well and one of the reasons we're a part of our UK interactive there is we're obviously, no one thinks we have an art collection so that's a kind of route to those audiences. There's an enormous photographic collection which is estimated to be somewhere about sort of four and a half million photographs and we're really in the very very early days of digitising that material. There's enormous numbers of archival materials which range from posters to maps to prints and so on and through to sort of technical drawings. So these are some of Charles Babridge's drawings for the difference engine which we held in our archive. So really really diverse collection and which tells a lot of different stories and I'll talk a little bit more about storytelling a little bit later on. So thinking about this presentation, I read the digital shift manifesto which is really really closely aligned to a lot of things that we're working on and thinking about and the manifesto kind of touches on things beyond the research libraries into cultural heritage, which I believe I think probably will say beyond that as well. And so these areas around skills, scholarships, spaces and stakeholders and what the shift might look like. So all the headings that appear under here when I was reading it and nodding thinking yes, some of those things would feel like we're in good shape and some of them are still on a to-do list. But I thought the way I focused today is actually thinking about this list which appears earlier in the manifesto which is really, it says in order to fully benefit from the digital shift we need to and then there's 13 things. And so what they talk a bit to is the sort of conditions for change and that felt like a kind of good moment to reflect on what it means to be a kind of digital leader and what it means to kind of try and implement some of the things that are kind of held in some of these areas here we're just talking about and what it means to actually implement them within a large complex organization. So I picked out, there's 13, I picked out 10 of them and I grouped a couple of them together and these are kind of some reflections on what I think's, what's worked well for me, what I think the challenges are and kind of looking at them in a lens of sort of reflecting back on what I'm doing now and what I've been doing really for the last 16 years. Because I've been doing digital since it was just a bunch of scruffy people in t-shirts in the basement and now of course it's like all the trustees want to talk about. Okay, so firstly, a clear vision and strategy in direction for decade-long digital transformation and then kind of reliable foresight and horizon scanning capable to inform the above. So these are kind of four strategy documents that I've written a couple of takes, a couple of sounds, music and group and I don't think there's really a one-size-fits-all strategy for cultural heritage organizations but I think there's some sort of themes that emerge and I think each organization needs to think about its own readiness to address those themes, the urgency of them, the ones that and so actually the ones that are listed in the digital shift form document are really good as a framing exercise is almost like you could go through those with a highlighter pen and think about ones that apply to you or put them into some sort of logical order or prioritization. So the, but I think the areas that need to be included in these kind of strategy thinking there's something around the user experience so that the actual people who were at the other end of the internet cable across the desk or whatever using the tools and services and content provided. There needs to be something in there around organizational transformation and we'll talk a bit about more to that a little bit later but it's important that it's a recognition that any technology project also is an organizational change project. I think it's worth having an eye to what funders might are prioritizing currently and those could be public sector or they could be the heritage national heritage philosophy fund or research councils or even sponsors and other trusts and foundations and so on because they have a particular set of things they're interested in I think the strategy needs to have an eye to that how it's all going to be funded. I think it's interesting to look at changing audience behaviors and have that into the strategy document and then clearly there's something around the technology landscape and so across one of those things it feels like you've got to have an eye to kind of both short long and medium term considerations and so what I've done historically is bite off the strategy into sort of two or three year chunks which kind of feel deliverable because the very long term strategies tend to they have to be quite sort of nebulous because the technology changes so fast so I think the things in the digital shift forms a 10 year project is the right ones but I would break it down to something a bit manageable but actually like when I was putting this together I was actually reflecting and thinking actually even within these two or three year chunks of work the new things have come things have been deprioritized so there is actually a need to adapt quite frequently and quite adorably I suppose the other thing to say is like digital is like never actually finished it's a thing that goes on forever so then thinking about kind of like horizon scanning and adopting things I think there's a lot of being there not always being the pioneer but being the person that comes second or third is actually quite a good thing so these are a bunch of standards some of you probably know them triple I have an image format there's a couple of copyright and open access things there as well we've adopted all of these things but we were not the first we were the ones who we'd go to spend a couple of years going to conferences and talking to people and seeing presentations and case studies before we adapted these things so I think horizon scanning is not necessarily about being the most innovative and being the first person it's about adopting these things at the right time and actually there is there's a lot of reports coming out of funders and academics and industry the Nesta, the Arts Council the Horizon reports and so on so there's actually quite a good body of reports and case studies that mean that the horizon scanning isn't too onerous except for the fact that it's actually totally overwhelming because digital touch is everything and it's really difficult to be a kind of expert on everything so the executive summary in the report is your friend, read that so thinking about organizational structures processes, cultures that embrace and respond to change this is something I've been doing a lot of thinking about this is from a paper that was published at the museums and the web conference by Calty Price from the V&A and David James from National Museums, Wales and it's really looking at different organization how do you structure digital in an organization so the one that I've become really interested in and we've kind of adopted as a hub and spoke model which is that you have a sort of small centralized digital team but a lot of the digital activity is happening out in the organization so in our case as a museum within our curatorial departments or marketing or learning collections and so on and so it's very much a partnership between a sort of small digital center and lots of people around the organization doing all kinds of things so why is that a good model versus others in my view it's good because it scales you can do more digital stuff because there's just more people doing digital it involves everyone so it drives organizational change whereas it's not like all the digital people are in that room over there and they're doing all the digital stuff there's some risks and some things to keep an eye on because it risks fragmentation of the user experience or things not being done in a holistic way so you've got to have sort of strong relationships between that center and the outside and that can be done through different kinds of workflows or project approaches or certainly if you do things like training the other thing that's worked quite well for me over the years is doing things as kind of proof of concept or prototypes ahead of launching them and that can be there to demonstrate that this is would be a successful approach and that we should therefore put more resources towards it or to actually try something new so this is a like we wanted to produce new kinds of online storytelling tools for collections and so we actually built this website very very fast we built it on an existing platform it didn't really integrate with our search and we built it sort of off to one side and it had a different design and all that stuff but it was a way that very fast we could try out some of these different formats because that was really what we wanted to experiment in with the editorial not the web design there was a bit of that in there we wanted to build a prototype that said what does it mean to engage with the science and technology collection online and a lot of that is to do with storytelling in ways that kind of bring those audiences so bring those stories to life so a kind of prototype approach which means that you can do things sort of fast and in the end we decommissioned this website so we didn't spend very much money on it and then we decommissioned it moved all the content into our final approach and we directed all the links so skills and innovation and all the other things you see there yes this is like really really important and so this is not my team at the Science Museum this is part of my team when I was at Tate's part of the coffee cup in the foreground bit late now so but both at Tate and at the Science Museum I had like a really multi-disciplinary team that covers editorial design and user experience video production hardware, software development web coding data science analytics and metrics partnership management and those were the people at the centre so they're also you've also got other skills pushed out so this is like a really diverse set of skills there's no like digital person but you need to cover a quite a broad area which means that if it's a very very small team it's really hard to find people that can cover across all of those things and so we'd like lots of others like have lots of contractors and specialists and consultancies around the edge so things that aren't full-time jobs but where we have there are particular companies who we work with on that really specialised areas so accessibility audience inside some of the really really nautical, technical, databasey stuff we have companies that we work with and so it's a kind of hybrid model of some things in-house and some things with suppliers and I think that's pretty normal but I think in addition if you're going back to the question about how do you know how do you have that kind of digital innovation and change and pushing things forward and so on I think it's not just have you got all the skills it's actually do those people have the time to work on things that will push the organisation forward are they encouraged to experiment do they have the time to go to learn from others to go to conferences read case studies are they encouraged to look across funding opportunities themselves because ultimately what you want to drive innovation is to have more ideas than you can possibly deliver so that as funding opportunities arise or you're shaping things for the future you're always sort of thinking what could we be doing next I think the other thing to think about in terms of skills is some of the softer skills are as important if not more important than some of the technical skills so having technical people who can really engage with the curators who are often historians and humanities people by background or who can work in partnership with potentially external partners across a multi-partner consortium project so those softer skills I think are really really important and the kind of can the digital people learn and train themselves they tend to be curious people and so they're looking for space to exercise their curiosity but I think you're also looking for some people in there who can really be the people who face out to the organisation and out to the world we've done quite well working with things like projects and bringing in staff projects both in our own department but also in other departments around the organisation and having roles created through those projects which at the end end up being made permanent roles in the organisation because you get to a point where no one can imagine the organisation without them because they've demonstrated so strongly how important and their skills are so number five so think about sustainable investment in digital in a constrained financial environment it's sort of a number of different directions here so one of which is we've tried to just keep things simple and standardise things a lot so we have a standard set of like formats I guess they are into which lots of our content flows so learning resources these stories in the top right hand corner peer reviewed articles blog posts and so on and so we try not to build too many new things I'll talk about when we do build new things we try to flow content into these areas because one of the challenges is that every new thing you create it adds a layer of complexity and this is the most complicated thing which we do but it's an example of when you totally do want to jump into the deep end of the swimming pool and deal with the complexity so this is a mapping from our internal collection management system onto which fields appear in which bit of the website and what features and functionalities those drive so it's actually really really complicated you've got to like shunt data from internal systems there's a big search index some of those fields do different things in the search some of them are links online some of them are drop-down menus really really complicated but it's like the core of our offer so where some of the bits of the things that we do we have external suppliers you know ticketing systems we mostly work with external suppliers because the ticketing system is a ticketing system for everybody we have some unique things but nothing that's really complicated this is really complicated and sort of we need to own in-house and we're always tinkering with it and changing changing it so but doing things like this what you incur is like a lot of technical debt there's an enormous amounts of code in here there's a kind of there's a there's a lot of data complexity there's a lot of systems integration going on and so the more of this kind of complexity you build the harder it is to do new things within a constrained financial environment where your operational budget for your department is broadly speaking the same each year over time if you're doing more and more complicated stuff you've got to maintain the things that you've built in the past and so the ability to innovate and do new things slows down with the more of this kind of stuff you have going on so it's something that you know we have a lot of conversations about what do we how to approach things in a really sustainable way the other thing we've done over the last few years is really try to like own the code so where we're working with external suppliers to get all of the underlying software code so that's something that we own and that kind of prevents vendor lock in it means it's easier to change things but then you've got to scale up you've got to manage all this stuff you've got to manage all this code and look after it and all that kind of stuff but it feels like really increasingly important to us that as an institution one of the things that we do is make software in a way that you know even probably ten years ago institutions didn't really make software now we do we're in that business and then there's no way to say goodbye and so this is a very well often much used websites but in the end it just became it started to break the integrations were broken it's more than ten years old it's got lots of flash in it which is the thing that used to be cool on the web and is now the thing that doesn't work on your iPad in fact it doesn't work anyway now because it's been decommissioned as the first of January so we we decommissioned a lot of websites in some instances we entirely rebuilt them as they were in more modern technologies so we rebuilt games that were in flash and very popular we rebuilt them in HTML5 this one we lifted all of the content up and we moved it somewhere else in the shiny new design but actually I think in addition to adding being careful about adding new things there's also having an eye to a weekend archive this because we're government funded this is the UK Government Web Archive which is run by the National Archives so as we build things now we're actually quite careful about can they be archived or do they get archived really neatly so one of the things that you don't get in the web archive for example is things like search don't work brilliantly but you can build it in such a way that so thinking about okay updated flexible technology advanced spaces that meet challenging user requirements so when I was at Tate there's not actually very much digital in the museum there because it's primarily in our museum there's audio guides and a few bits and pieces at the Science Museum there's absolutely loads probably more than any other museum in the country and so this is not it's probably not the same thing that you would have in a research library but I think there's probably some okay so I'll just pick out some things that I think really might cut across the two and sort of reflecting on this point hopefully you can hear my dog barking okay so number one this stuff is really expensive and it dates really quickly so every child now assumes every screen is a touch screen and not only that it's a multi-touch screen and not only that it's going to be as responsive as their iPad is and yet it also has to be like battle-hardened in the museum because there's going to be 10,000 school kids brushing away at it and eating their packed lunch of it but it can give like a really high quality experience to use because you know the context you can customize the content and that it can be like a real wow experience especially if these are the kinds of technologies that people don't have in their own home so this is actually much bigger than it looks in this photo anyway it's a very very large touch table the size of a single bed and it's kind of amazing it's a four player multiplayer game but I think it's worth saying that the maintenance costs of hardware add up really quickly you need spares and there's a whole other set of skills which are not the same set of skills as building web development and building things online that's a really unique set of skills not just to build the thing but also to maintain it and also to understand the user experience that you're trying to have so we're quite careful about when we implement this we have a lot of conversations before we put like tons of digital into a space and we do do it sometimes but in other times we kind of step back away from it I think one of the really interesting things about COVID-19 has been the kind of bring your own device thing and the growth of things like QR codes and what of those behaviors will continue beyond the pandemic when the answer is we don't know but it feels like lots more people are doing things like scanning QR codes with some of the means like oh this could be really interesting in the future and I think it goes without saying really fast reliable wi-fi is always a good thing especially if you have teenagers so have a defined role in discovery and access in an open research content so how we define our role is absolutely tons of audience research and of all different kinds and so we're trying really hard to define what it is our audiences need so things like web analytics can tell you like an awful lot because you can see what people are doing where they're coming from and stuff like that you can also look at things like search times where people are coming from what search queries are they putting in but we're now increasingly starting to do things like much more sophisticated online surveys sometimes around specialist areas we're doing follow-up telephone interviews in order to understand user needs better we've done focus groups and we're now increasingly around the editorial content doing keyword analysis which is looking at how strongly do search terms related to the content we're thinking of making, how many people search those things and how well would we rank so to give you an example if we produced content on the moon landings the Apollo moon landings lots of people look for that content online but we actually wouldn't rank that well because there's an awful lot of content out there already and we have an Apollo the Apollo 10 capsule so the one before the moon landing the kind of dress rehearsal I guess in the museum but we've kind of slightly shied away from doing too much content around it because it's so densely an example at the other end is we looked at navies who were the people that built the railways and canals also and we don't have much content on them people are very very interested in the content online so if we produced content on navies we would rank really well in search engines and so we try to sort of understand the audience behaviors and our place in there our role in a couple of different contexts one is around content that people kind of like pull to them so that's primarily via search in search engines, Google primarily and so we've done lots of work around search engine optimization and search ranking then there's a question about like content that we like push out to people so there are so to give an example of the search lots of our work around learning resources is really clearly tied to themes in the national curriculum because teachers are looking searching by these terms around the pushing content out a lot of that's to do with things like social media so you're trying to position content into people's kind of stream of daily digital content that they're consuming and across both like push and pull content there's this question about partnerships and who kind of amplifies reach to a different audience or to a larger audience and then the kind of audience insight and the analytics and sort of understanding our changing role there's always more to learn and it's always changing it's a kind of on-going thing so it's actually something that we're starting to build a capability for in the museum much more strongly so a way to innovate in services despite an often ailing core technical architecture yep I know that story have innovation and continual service improvement embedded as a way of working so there's two like things here and they're sort of pulling in slightly different directions as you can see so I dug this out okay so this is I mean I don't think it's like this is the technical architecture when I was at Tate so at the bottom in pink pink and yellow are like internal systems and databases and stuff the blue is like various integrations and data services and all that kind of stuff and then the green the like services at the top and the gray I think were the ones that we were trying to decommission so what you see here is like similar to a lot of people a lot of legacy systems a lot of like legacy data so and data in the format that is really sort of not what you want you want something different but because it's in this system and other things are connected to it it's really hard to change things and these systems are big and complicated so we're often not in the business of changing them very often you seem to like review their collection management systems like every 10 years which is why collection management systems tend not to innovate very very fast there's some green shoots there's a lot of interesting conversations about collection management systems these days so I shouldn't be too rude so what we try to do, what I've tried to do is do something more modular and to actually accept there's always going to be this kind of complexity and actually having multiple systems and having them loosely coupled and having data services that push things around actually gives you some flexibility so maybe having all these systems is not that bad because you can kind of manage the life cycle of them independently of each other you can retire systems there's a lot of monolithic lots of, one big system doing lots of things it can be quite difficult to change things without breaking other things and so on and so actually having lots of different systems doing specialist things that are kind of lightly integrated with each other means that you can also run developments and changes in parallel a lot of the times that these things are kind of replaced they're done via projects so you end up with this kind of like sawtooth investment model where there's periods of large investment in one of these boxes and then periods where nothing happens to them and people are just kind of operationally keeping the thing running or making small configuration changes so we generally we've been trying to move to a kind of more operational I think some of those changes and doing more frequent changes and fewer of these larger transformations and we try to get those projects to their full contribute as broadly as they can it done lots of work on like open data and APIs so trying to get our collection data in a machine readable format which we then built all kinds of other things on top of I'll show you in a second and then we have a kind of lab approach which is a place in which to do experimentation and try things and so among the things that we've worked on we did the first kind of 3D scanning work and we were pretty confident that 3D scanning technology was like here photogrammetry and LiDAR scanning and so on we're pretty confident it would be good but we want to demonstrate to the organization this is the thing here's what we can do with it and interestingly the place it played out and got most traction was in learning resources for classrooms so teachers can pull these up on whiteboards those hotspots are which are the numbers on the 3D scan are kind of aimed at children so with the open data we've done things like hackathons to bring together developers so they can kind of and in part it's like a fun thing to do and let's build some fun stuff in the pizza but on that level it's also about us understanding how easy is this data to work with where do people get stuck so out of the hackathons various like games were built and visualizations and so on using our open data once we first released it and then once we had a kind of collections API it suddenly became a lot easier to do things once we had open data when people approached us, academics or researchers and so on it became a lot easier just to be able to provide them with a look here's the keys to the car go and play with the stuff so this is this is our like viral moment because on Friday what this does is it queries our API and it says give me a random object from the Science Museum group collection which has a total page view count of zero so if you click on the link you're like the first person to see that thing since it's been released online so we released it on Friday and since then it's had like 150,000 page views so it's kind of pulled us over a little bit but in a way and that's nice in a way the interesting thing though is when you look at it it's actually only about 200 lines of code and a lot of that is spaces so the code is really nice and readable because it's not all of the complexity is like somewhere else has been dealt with in the open data thing and so lastly the capacity and capability to meaningfully steer or at least engage with artificial intelligence so so spoiler alert artificial intelligence is here and we're really interested in it for a couple of reasons which we'll go through now so firstly a lot of our catalog is really thin because we're digitizing things at speed we're even by need to move out of our current collection store to a new one which means we're photographing stuff but the catalog records are really thin so we've been looking at what a computer is really good at and what can they add and so these are machine generated tags based on just looking at the image, the computer hasn't got anything else to go on so where is this technology now it's about 80% really interesting and 20% completely crazy and totally wrong so it's sort of where optical character recognition was a while ago and so this is a particularly good one the nice one I liked here was cobblestones oh that's really nice, our curators totally not put cobblestones into the caption but if you're interested in cobblestones we've got you covered now so that's really interesting it doesn't feel ready to present on the web interface yet but we've been looking at things, you can see what it's trained on, it's really good on photographs it's much less good on things like archaeological collections and things like that but it's definitely going to get better and you can kind of see what it's been trained on it's quite good on products and things like that especially the Amazon one because it's obviously been trained on all the products in their shop the Microsoft one is really interested in cake so it quite thinks things are cake this is kind of clustering which this was done by Kathleen at Nesta which took 9000 things from a particular bit of the collection and then clustered them by visual similarity and then the outline colours are decades and so again the computer doesn't know anything all it knows is it's just looking at the objects and trying to cluster them so here you can see it's pulled together like old style computer keyboards calculators mobile phones and so it's clustered these things together based on just how they look so again kind of really you can start to see interesting ways in which collections might be explorable in the future and actually a lot of these things probably would have been in the same categories but there are other bits where in a way it's more interesting the proximity of groupings of things is that it's got all the typewriters next to each other so then thinking about like clustering one of the things we've been thinking about is what does it mean if you can kind of cluster images by like how similar they are to each other can you do the same thing with kind of like concepts in the catalog so we have a grant from the arts and humanities research council for 21 months to look at analysing the actual catalogue itself and try and build links between things in the catalogue so by analysing the fields in the database but also the free text descriptions and so on and also associated material to start to build a kind of knowledge graph against initially our collection but seeing the V&A's collection to see whether like the computer can start to cluster together and infer links between things and so there's a sort of complicated set of techniques being used around matching entities between different systems using wiki data as a way to sort of bounce through wiki data and pull in other data sets looking at other things like gazetteers to bring in to help the machine so sort of spoiler alert on how that's going the it's really good at things like really structured things so stuff like dates and countries it's really really good at it's pretty good at people it's less good at companies because often company names have like ands in them especially older companies they tend to have like longer names and a modern company which would be much more sort of a short word brand but a good name but that so off the shelf with an untrained machine learning model so a model that's just you just plug it in and off you go it does pretty well so those results are just that's an untrained model so but what we're going to do next is work out where's the best place to start to train the model and what things will it get good at so in answer to the sort of question back here capacity and capability to steer or at least engage with machine learning I think the answer is that a lot of this machine learning technology is going to become a commodity in the same way that these days you would if you wanted to buy an email system or a customer service system you would sort of just buy one as a commodity and write your own and so it's getting pretty close now and actually in a few months we've got to the point where we're using off the shelf systems and we're starting to like work with the in a light way kind of train our own model so I think actually steering and engaging almost the same thing but it is a kind of specialist skill we have to bring in a researcher from outside and he's come from IBM and we are learning an awful lot from him it's really really interesting work and probably in the future these things will sort of be built into system and this is how like Google and Facebook and Pinterest and all these things work all those and Netflix all those discovery based systems are based on machine learning and it's not that they're doing magic on their own someone behind the scenes is kind of tuning them and building them and reinforcing the things but clearly there's organizations with enormous amounts of content this is something we definitely need to engage with. Thanks very much. Thank you, John. That was fantastic. First time anyone's mapped their presentation to the manifesto and it worked really well, thank you and I just love the mix between the really practical and down the dirt as it were and right up to the organizational wonderful. We've got a number of questions and again the questions reflect this there's some quite high level ones but also some really practical ones and I'll just lead off if I can just coming back to I noticed on your website you had a blog which it was a couple of years ago it was about implementing change in the digital departments and what really struck me was none of the how to's you gave were actually about anything digital and so the message behind that is really interesting and I'd like to link that with a question that came in with thinking about the user experience what will a digital organization or a digitally transformed organization will do better for them. I think the I think what people expect from the user experience is something that feels really seamless it sort of meets them where they are and so and the challenge in a big complicated organization is that there's a tendency for the user experience to kind of reflect the structure of the organization and so it's it's actually quite a lot of effort to always come back to what is the user going to experience how will this make sense and so to give an example one of the examples that's always given of like a really terrible user experiences university websites so at the top there's always like some like home page sunny day on the campus and then maybe there's like a quote from the Vice Chancellor about their impeccable research credentials and all the rest of it but once you get underneath it tends to be like a total mess because it's very devolved everyone's just kind of putting in whatever they want and so but in a sense maybe that doesn't matter because you know the philosophy department and the physics department are sort of next to each other in the directory but they haven't really to a great extent got the same audiences but actually when you think about a collecting institution what you want to give people is the is you put that in the search box everything you might possibly have wanted has come there we've thought through what you want to do so there's a kind of service design element and this is what gulf.uk have done spent a lot of time working on and which is why the government's website now is actually quite good because they've really come at it with a service design from a service design standpoint but that means bringing together disparate people from different places because it could be that the data is coming from here but the the other people coming from here and so there's a lot more kind of conversations to make great service yes okay so thank you so moving on slightly I mean one of the things that obviously strikes you and you you talked about high speed wifi etc everything's digital what about digital exclusion and this is something that's in the news at the moment and interesting to us because we are all UK's planning an event on digital exclusion poverty in the role of libraries later this year so yeah really important and so not everyone has high speed wifi not everyone has a laptop at home and so I think it's going to become more and more of an issue unless there's like a seminar in a couple of weeks from the end of the month study on exactly this subject which I was thinking yeah I must sign up for that so a plug for them I think it's I want to say Thursday the 17th but it's on like access to collection and sort of what does it mean when everyone doesn't have access so I sometimes don't have an answer but I agree it's increasingly being an issue and you see it with school kids who don't have laptops I keep the usual is going to do it when we get there moving again slightly further just one that's just come in talking about the discussion you talked about financial sustainability of digital what about the environmental sustainability and how is our organisations we can meet current user needs while also doing environmental support yeah totally and so in fact it was one of the other things I started to write on it and then it is one of the 13 in the initial list then I like whittled it down to 10 and was trying to get to time so I think yes so I think in part it's about the suppliers that you work with and so for example we've moved all of our primary website hosting to Microsoft and we didn't have in the ranking factor green credentials of the suppliers but I think that is the way to do it and if we had they probably would have come out top so I think it's really important consideration things like machine learning like if you train your own model from scratch it's incredibly computationally intensive so most of this stuff is now not going to be done in our buildings it's going to be done in the cloud so I think it's therefore really important to look at those cloud service providers amongst other things hardware again like e-waste is a massive problem so you know things in the museum thinking about their lifetime and their life cycle and where could they go after we finished them is also really important right moving on to the slightly more practical so there's a question around do you diversify between different user groups and services you build on the content? Yeah 100% and some of them some things are designed for multiple different user groups and some are really you know there's a website for teachers and there's we often talk about doing websites for children some things have to cut across them all the tricky one is actually the collection because that needs to be everybody from like a kind of 10-year-old who's suddenly interested in aeroplanes all the way up to a kind of doctor or student and everybody in between and so that's quite often the tricky one is how but yes we absolutely zone off bits of the content for specific audiences because they have particular needs Indeed And a really interesting one is I don't know you didn't really talk about the methodologies you use but there's a current tendency predilection for many organizations to use agile Yeah as a way of doing things and you didn't talk about it but Yeah we totally do with agile software development which in a nutshell for those who don't know it tries to well there's a whole bunch of things but it tries to accept that the more you work on a project the more you know about the difficulty of implementing things what's important what isn't so it's quite a flexible format and so it really lends itself very very well to especially software development so if we're building something and it's like an online shop we sort of know what that is and so we tend to just do a lot of upfront specifications because we know how to give that to a supplier when we're building something really complicated like a collection or something that's using computer vision we totally work in an agile way because everything is a conversation and the more frequent those conversations are the better Yes I've totally drunk the agile Kool-Aid Thanks I'm conscious of the time if we can run over a couple of minutes that would be really helpful there's a couple of points so linking with that and methodologies big difficulty we've got in academic libraries is we manage a lot of legacy software applications and as the question of comments all of which are critically important as far as the researchers are concerned but do you have any internal guidelines on when you rewrite code bases to keep them modern and alive without pain and you know what is there a link at all with the agile approach We don't really have guidelines what we do often do with some of these legacy systems is we just look at the data out into something else that we can work with in a much more flexible way so also we have a number of collection management systems and the data structure is very complicated and very rigid and goes back decades and probably even to card indexes before that and then ledges before that so some of it is probably more than 100 years old and they're really difficult to work with in their current formats so we try to get the whole thing out into something that we can work with so some other data structure or into an elastic search index and then we tend to build on top of that so we try to sort of abstract away from some of the legacy systems I know that's not really what the question is asking but it just means that we can work on top of things because those systems are probably not going anywhere in a major implementation project so different thank you and last couple you mentioned API an API and so other collections discoverable and searchable via an API yeah if you wanted to you could in fact the collection is open source so if you wanted to you could download the collection website on github put it on your laptop our collection API and you would essentially get our collection running on your laptop which you could probably do you could probably do it in an afternoon if you really wanted to but don't tell your boss that's what you're doing and so essentially the API the API is the public API is the one that we're actually using it is the live one and so yeah so you can fire search queries in fact there's things you can do on the API if you start to dig around in it there's features on the website yet so actually you'll see things there so for example the image tags are in the API but they're not used in the website search index currently that's interesting that's a nice challenge for someone yeah and the last question and coming back to users always the core I'm looking at the touch table the digital services generally how do you evaluate users reactions and the payback and user engagement for the level of resource of input you put in like in the museum itself is that the question so they're really integrated into the experience so where they are so we do there's sort of three parts to that so there's formative evaluation is done on is this the best way to deliver the content so you know what problem are we trying to solve is very very tightly defined upfront then there is a prototyping stage or sometimes even two or three prototyping stages and that can be a technical prototype running on a computer or it can be like a paper and cardboard prototype that just says what happened and then you do this and so one of the nice things about being museum and I guess the library is you have people there so you can sort of say why don't you come over and can you spend like 20 minutes with us we're going to talk you through this exhibit and you can try it and sometimes the thing changes quite radically at that stage it's not uncommon as you the unspressed to hear to find that things get really simplified at that stage there's a tendency to make things over complicated as a user experience they often get made more simple and then there's a summative evaluation which is observational so it's normally somebody we can be just watching and counting the number of people using things but there can be kind of shadowed visits as well especially if it's like a kind of for particular audiences so there'd be like a kind of shadowed visit of a number of school groups or families or whatever and then there'll be kind of interviews afterwards so there's actually a lot of evaluation science museums tend to be really really good at like evaluation because science communication is really complicated and so it's not necessarily about the learning outcomes are not necessarily about the facts they're often about things a little bit more I said that was the last question but I've had another one which I do want to ask you and it's about you so what are the digital projects or sites not necessarily cultural but any that inspire you and make you happy I think that's a lovely question I'm really interested in like sometimes I feel like all websites are the same these days so I'm kind of really interested in unusual design what else oh goodness I've usually got like 10 obsessions on the go let's think I saw a really nice thing last week from the British Library were they taking their sound collections or someone had taken their sound collection and made a kind of virtual it was all recordings in forests and they created a virtual like forest and you put your headphones on them with your computer keys you sort of went through the forest and as you moved around the different parts of the forest the different sort of birds sang to you which I thought oh that's a really nice kind of way of engaging with the cultural heritage collection it's kind of very experiential what else there was an audio experience that was done at the Banquetting Hall in White Hall I don't think it's there anymore it's called the Lost Palace it's an audio experience that you did and you put on headphones and you carried I think it was a thing and you travelled around the streets around and using like GPS and and so on it took you on a kind of experience of the palace that used to be there and again that was a really nice it played to all the strengths of digital and yet because it was in an audio experience you were still sort of like engaging with the world and it encouraged you to look up at particular buildings and said up there would have been the room of so and so and the river would have come to just here so some of these kind of immersive experiences I think are really really interesting thank you and one of the things you've talked about today has been the complexity and simplicity and just picking up on your I'd just like to say your own website you've got the text at an angle across the page which is really not normal I found that really disconcerting in a really good way so I think there's a trending website called Brutalist Web Design which is kind of Web Design which is supposed to be kind of like unforgiving and kind of well Brutalist so there's a bit of a nod to that yes and it worked it was really worked so thank you John that's been a wonderful session really fascinating and a great addition to a series of forums so really appreciate it and there's been comments in the chat about the outcomes of the AI project which as I said is really interested me and it's high relevance to libraries very much so and yes so the thanks are coming in well you can see yourself in the chat thank you John