 Alright, so how do you go about starting a crowd-sourcing project? Well, I started with a shawl. I was going through some old exhibition files when I found one that basically said, when Carrie McDonald was a young girl, she made this shawl with the help of her mother using some wool from their neighbor's sheep. How interesting. It's so rare we have something made by children in the collection. I wonder if it's any good. Here's a close-up of the shawl. That's a pretty intricate feather and fan pattern. I'm guessing this wasn't her first project. From the catalogue record, I found out that her mother's name was Harriet and her father's name was Donald. That's right, Donald MacDonald. I saw that the neighbor with the sheep was Mr. McKay and the donor of the shawl was Mrs. T. W. Dent. I was very pleased to find out her father's name because this meant I might be able to find out some more information about her. Canterbury Museum has a wonderful resource called the GR MacDonald Dictionary of Canterbury Biographies, or the MacDonald Dictionary, for short. I should note that there's no relation between GR MacDonald and Donald MacDonald. They spell their names differently. For those of you unfamiliar with the MacDonald Dictionary, you might be wondering why I wouldn't just look for Carrie's name in the dictionary. That's because the dictionary is mainly organized by the male heads of the household. The dictionary was presented to Canterbury Museum in 1964. This is a photo from the museum's annual report from that year. You can see a tall bespectacled George Rinald MacDonald reading one of the biography cards to a board member and archivist librarian John C. Wilson. George started the project back in 1952. At first, he just intended to write biographies for some of the portraits in our photographic collection, but the scope soon expand to include anyone he could find information about who lived in the 19th century in Canterbury. Twelve years later, and he had biographies for over 12,000 people. Some people had just one line, and others had biographies that span several index cards. There are over 22,000 index cards in the collection. And as you can see, they're originally stored in the metal drawers. I should mention that George volunteered his time for this project, receiving encouragement from our then director, Roger Duff. The dictionary was considered very progressive at the time it was donated because it included people from all social classes. But you have to keep in mind that MacDonald was writing about 19th century people in the 1950s and 1960s. So while people from all social classes are included, which is progressive, there are very few Maori and women who have their own biography card. In the early 1990s, when Canterbury Museum was preparing an exhibition to mark the centennial of women's suffrage, someone went through and counted all the women who have their own biography card. They came up with 64. No one's done account of how many Maori have their own individual biography card, but I'm sure it's equally as low. However, women aren't completely absent from the other thousands of index cards in the dictionary. They're just not indexed because they're not the head of the household or a public figure. And that's why I look for Kerry on her father's card. Here is page one of Donald MacDonald's biography. There's his wife Harriet mentioned twice on the first page. Since all the cards were written out by hand, George invented shortcuts to make it faster, such as M-A-R-R for married or D-A-U for daughter. Now this is where I got excited because I noticed that his daughter Flora married T-W Dent, which means she's Mrs. T-W Dent and Flora is the donor of the shop. On to page two of Donald's biography. There's Harriet again, and Flora, and Flora's husband. But who's this? Caroline Houston MacDonald. Could Kerry be short for Caroline? To figure that out, I could check some of George's sources, such as the family Bible referenced here on the second page, or at the bottom of the first page there are some newspaper references. There's a list that explains all these abbreviations. And using this list, I can see that these are newspaper references to the Littleton Times from the 20th and 21st of July 1906, and it happens to be Donald's obituary. Now the Littleton Times isn't digitized to 1906 yet, but while I was on papers past, I found Kerry's obituary in another newspaper, and I did in fact confirm that she was Caroline. Using the dictionary can be a bit like solving a puzzle between deciphering the handwriting and the abbreviations. In the past, people had to come to the museum to look at the dictionary, and they could look at the printed list of abbreviations and see the index with the list of women's biographies at the front. There was also a subject index, so you could find a list of all the people connected to city council or whatever it is that you were looking for. This was a good resource for the few people who could make their way to Christchurch and to the museum. Now we've put it online with Creative Commons licensing so that anyone with internet access can potentially look at the cards. So we've greatly increased our potential audience, but we've actually decreased access to the finding aids. You can still search for all the people that were indexed. That's under associated person now, but the list of abbreviations and the subject index isn't available online. Sure, we could put PDFs of these documents online, but that isn't really making the best use of collections online, and it doesn't really reflect how people want to do research today. They expect to be able to type in Carrie McDonald and come up with all the documents, objects, photos that are related to that person. Wouldn't it be better if you could just click on a link and go straight to Donald's obituary on paper's pass? It would be amazing. But then you remember that there are over 22,000 individual index cards and you get overwhelmed. But then you get excited because you realize that the McDonald dictionary is perfect for a crowdsourcing project. And a crowdsourcing project driven by volunteer contributions is very much in keeping with Donald's original project. I just hope it doesn't take us 12 years. Making the project a reality is where Chris and our project intern, Phoebe Fordyce, come in. So I'll hand over to Chris now. Kia ora koutou. Thank you for having us here to talk about this. So as Joanna said, we got excited about the possibilities for crowdsourcing some of this information and running it as a student project. And that conversation really began at one of a series of meet-ups that we have been running at the University of Canterbury for people interested in digital humanities and for those from the cultural heritage kind of areas in Christchurch and the region. We've been running these kind of monthly, except for the months where we don't run them. So they've been a little bit ad hoc throughout this year, but they have been a really fantastic way to kind of connect up people from different areas and sort of join up interests. They've been run by a couple of post-grad research associates of the Arts Digital Lab at Canterbury, including Kara who's pictured at the top here. She spoke at last year's NDF and she's gesturing to a cat meme here that says the future is meow, in case you can't read it. So we do some fun stuff. And this is really the sort of setting where Joanna and I kind of talked about in the McDonald dictionary as a potential sort of crowd sourcing project. We have a program called Professional and Community Engagement at the University of Canterbury, where students can do an internship that is both sort of underpinned by some academic work, but where they get to work with a partner from industry or the community on a specific project to do some practical hands-on kind of stuff. So we set up an internship project around this. But really we wanted to sort of ask, well, kind of critically examined, is this actually a good idea? What would we want to get out of it? What specifically would we want to transcribe? How would we want to make that available? And kind of answer some of those questions, as well as really answering how would this work as a teaching project? I sympathised or related to what Harkin will said about being a person who doesn't have very many colleagues around them who knows what they do. Sometimes I feel a little bit like that in teaching and doing research in digital humanities. And so we wanted to sort of see if this really was a good idea. And Canterbury Museum's interest in transcribing the many names of women mentioned in the dictionary also provided a natural, intellectual focus for the project and thinking about crowdsourcing both in practical terms but also in terms of feminist history and having a kind of a focused approach or a direction to our aim for our crowdsourcing work. So we found a great intern, Phoebe, and we made her do a number of things as part of the project. She had to transcribe a number of the index cards in full in order to get a sense of what this material was like, what the challenges would be with the handwriting, deciphering the abbreviations and so on. And we also asked her to then produce a scoping report that would kind of set out the kinds of things that we might do, how we might go about doing it, who would be involved, what the sort of considerations might be for Canterbury Museum in setting up something like this. So she wrote about sort of the suitability of the McDonald dictionary and kind of identifying the things that would be worth doing, what the potential issues might be, including thinking about sort of the any potential ethical or kind of copyright issues. She had to produce some sort of project management sort of scoping material and summarise what the options would be in terms of potential software that we could use for the project. She also had an academic side to the project. So she was taking a Stage 2 paper on feminist history, so she brought a lot of her interest from that into this and kind of asked what kinds of theoretical underpinnings would be most valuable to apply to the project through that work. We also, I should also acknowledge a lot of great work was done by our developer colleague of ours, Antoine, who is here giving us a dramatisation of web development in action. He's a digital project specialist in the Arts Digital Lab at UC and his main task was working out the data round trip, so taking material from Vernon and getting it into Pybossa, which is the platform we ended up using, which I'll say a little more about in a second, and then getting it back again. So kind of doing that plumbing work and setting up a web platform that we could sort of start to test some ideas out. It was sort of a difficult balance, I guess, that I was exploring with this project to do with how much could a student, in turn, realistically do here. How can we involve someone who's not done this before in a project team where they get a real sense of ownership and contributing something and learning new skills, but, you know, we don't want to do it for them. We want to challenge them but not challenge them too much, if that's a good way to put it. So we were trying to strike that balance and not having done a project like this before, that was really one of the things that I was seeking to find out how we would do that. So having Antoine's support to do a lot of the technical side of the setup and, you know, say no to things when I was suggesting maybe we should bigger the project was really valuable. So part of Phoebe's work with assistance from Antoine and interference from the others in the team was to assist different options for a technical platform, and we looked at three things. Scribe, which is a platform that's been used quite extensively by the New York Public Libraries for some big projects that many people will have heard of. I believe the old Weather project and measuring the ANZACs and some ones like that use it. And Pibosa, which is another Python based crowdsourcing framework, and Google Docs for a sort of more easier to get up and running quickly kind of approach. And there were sort of obvious trade-offs in the analysis that Phoebe was able to do of these. Scribe is very powerful, but we had some problems really understanding it, getting to terms of running properly. So it seemed like a bit of a racehorse in this kind of field, like it was perhaps a bit, had more features than we needed and a bit difficult to manage. Google Docs would have been very easy to work with, but would of course limit the ways that we could present the project and what we wanted to, if we wanted to build this into sort of more of a public facing thing, then we'd be limited. So Pibosa was a good middle ground in terms of the solutions that we looked at. It's open source, it was fairly easy to install, had reasonably good documentation that we could work with. They also have a platform called crowdcrafting.org that's powered by Pibosa that you can use to get up and running if you don't want to host your own platform. So that was quite good. So we made something that looks like this and started with Phoebe and Antoine, mainly started working through the issues of kind of how do we get this interface to work, how do we write instructions for people that the most people will understand, how do we make it clear for the widest audience. So we had a number of challenges getting the image big enough to do transcription. People are sort of going to be frowning at the screen trying to work out McDonald's handwriting. So we needed to make it as big as possible and we went for actually not showing the whole image at once, just having a wide image that could be scrolled in order that we could also have some form elements on the page and instructions on the page. Actually writing the instructions in a clear sort of direct way was a challenge. Capturing alternative forms of a name because particularly with the women listed in these cards often there are multiple forms of their names, they have their husbands names maiden names and name variants. So we wanted to be able to capture all of those if we were going to do this work and there's some browser testing stuff that we need to do. So Antoine did a great job of getting us started with a platform that we can use but at the moment it only works on Chrome. So yeah, future work there. We then did a bit of a trial to test our platform and this kind of expanded the scope of the project sort of a little more than I'd really anticipated when we started the earlier in the year. We did a Human Ethics Committee application in order to run this as a research project through the university and if any of you have done those sorts of ethics applications before that you know can be a fairly difficult process sometimes. We also initially targeted students to get students to participate in this and found that they are very difficult to motivate. So we sort of shifted tack and targeted colleagues and people whose arms we could twist much more easily to sort of get a group of people who we thought were an appropriate audience who would be likely to be motivated to participate in something like this for real and we wanted to find out really what they thought about the interface and doing the task but also why they thought the project was of interest and if this was something that they would do more of if they were interested in doing this because it had a feminist history perspective or for other reasons and in terms of our findings with this small study we got around 20 respondents who did the task, used the website to transcribe a card and gave us some feedback they found McDonald's handwriting difficult to read so we do need to manage the expectations of this and recognise that there are people who are very keen to do this and that's the hang of it so we want to do the best we can not to put people off early working with the dates is quite challenging I think our interface didn't allow people to just put a year date so they could put a month and a year or a day and a month and a year and so that's a problem that's sort of tied up with browser testing and other things but that was a challenge for us and something that users wanted to do better do more accurately interest in the feminist aspect of the project was strong there were a couple of people who didn't understand the question as we posed it but for the majority it was motivating or an interesting factor to be working on a project like that and really we need to do a bit more testing of our resources to kind of make sure that the clarity is there so there's quite a bit of future work that we've identified through doing this and as Joanna pointed out one of the things that we'd love to do here is to create links to other external resources it raises questions about exactly how we do that how we delimit McDonald's work as a historical document in relation to links out to other things that we might draw in that are relevant so there are a number of interesting related materials on papers passed that we could draw in interestingly the Littleton Times article that McDonald actually cites that Joanna mentioned to you earlier hasn't been digitised but you can find other materials that he didn't cite such as a bituary from New Zealand and carries a bituary from the New Zealand Herald so there's a range of other materials that we might draw in here and we'd be very interested to retest and refine the interface next year to be able to capture some of these references and consider how we can use these additional links that we might be able to create to papers passed and other such things and also think about the existing subject index and other finding aids that exist at the Canterbury Museum and three or four large ring binder folders and what we could do to connect those to the collections online as well so the state that we want to get to I'll ask Joanna if you want to say a little more about that so when everything's all done we hope that our collections online will look something like this you can see the list of names has been greatly expanded you can see all of Donald's family members including the neighbour that he brought some land from Mr. Jay McClellan the family bible that was referenced on the second page is held privately but you would be able to click on all these newspaper references here from the first page and clicking on any of these names will bring up anything in Canterbury Museum's collections online that relates to that person and hopefully people will also use this data to make collections past Canterbury Museum's collections online and all these names will become nodes in a network and who knows maybe that network will even end up resembling a feather and fan pattern thank you thanks it was a great presentation are you modelling the familiar relationships or just capturing names not currently in this run we did look at testing that but it proved to be very complicated and it's not always clear yeah Vernon has the capacity to store that data but would you use Pipe Oster again we're just trying to choose a platform now for some geocoding crowdsourcing and Pipe Oster sort of rose to the top of the list from my early research but good to get some feedback it seems to be quite stable quite easy to use quite easy to get up and running sorry yeah so it seems to be quite yeah quite stable quite easy to use and get up and running fairly quickly with we don't have a lot of technical expertise and we don't have a budget to kind of get a specialist to really dig into a new platform so we wanted to go with something that we felt was was going to be you know within our abilities and that seemed like a good match for us has it been around a lot? it's been a few years I don't know exactly when it started it's mostly developed by side fabric who do you know kind of crowdsourcing and citizen science provides sort of services on top of the platform I think they're based in the UK in Spain is it? all right did you find with the trial that you needed to have an editorial eye over the results like was there a need to kind of see what people were producing from the cards and make sure yep so we will be doing that we just haven't done that yet because we finished the trial right before the conference but we will do that and just another note the Linton times will be it is coming nice that fits perfectly I was just curious it's a great resource that you're doing here and I wondered whether there were thoughts to in future link through to you know archives New Zealand's references to those people or to other you know crush your city libraries references and that kind of thing there's definitely lots of opportunities and I think after we finish this first trial and get it working and out to a broader public for tagging I think that's when it's time to start thinking about other possibilities and those are definitely on the list you know with the scope of the cards that you had did you find it more less wine blowing bring that back down to a small portion of the woman aspect of it well we will be tagging all the cards as part of the broader crowdsourcing project that's just kind of like a theoretical focus for why we are approaching it and of course everyone's name will be tagged but many of those people are women as a McDonald well done hi I was wondering about recognition of the contributions from the crowd as a way like there's an outdated word at this point but gamification to encourage people to do it more so a leader board or levels that you can get to as a contributor yeah perhaps like a comment so the reasons people gave why in the trial that we did as to why they would participate were varied that sort of competitive aspect of it or the gamified aspect of it didn't come up but I don't think you'd expect that with the kind of question that we asked people talked about doing these kinds of projects because they find them because they're personally interested in the social history some people one person talked about they find it relaxing and kind of good for the clearing the mind to do some kind of this kind of work yeah Pibosa has a leader board kind of stuff that sort of built into it that we could make use of to do that so it's something we could look at at a future stage I was going to ask the same question as him but I'll now ask a different one what's the schedule for sort of public release or going live or announcing it to the world or moving to full function we don't have a schedule yet as you'll remember we finished the trial right before the conference so we have to after we get back to Christchurch we've got to go check that if anyone else has participated in between and very carefully review that feedback and then make a schedule and I guess I'd say we want to figure out how we can develop this as a teaching project so that's sort of been my motivation for and one of my motivations for doing this is working out how to integrate it with teaching and kind of keep it manageable and make work for the students and meet certain learning outcomes so we want to sort of design a piece of the next steps that could be done next year and maybe have two or three students work on it rather than one to be a little bit more ambitious about the size of what we'll do but that has to be kind of flexible that time line I think we're at lunch so one more round of things