 For our first session this morning, the title of our session is Digital Exclusion, lived experience of minority and marginalised people in York. Our speaker is Claire Boardman from the University of York and she is a research associate there and is currently finishing her PhD in Applied Digital Humanities. Today she's going to share with us insights from her summer 2021 Digital Exclusion research. So we move over now to Claire. Thank you Claire, we'll hand over to you for the rest of this you're speaking. Thank you very much Liz and thank you for inviting me to talk this morning. I'm Claire Boardman, I'm a white woman with brown hair and today I'm wearing a red shirt and a flowery scarf and I'm just going to start my talk in true tradition with a mini caveat. I just wanted to let you know that this work was not undertaken within the context of collections or indeed the cultural sector at all. It was actually commissioned to support the development of a local authority digital strategy and as such today it's offered really as insight and perhaps hopefully inspiration rather than any sector specific information or instruction. So I will just share my screen. There we go, I hope that everyone can see that. Okay. So from the early weeks of the first lockdown in March 2020, the extent of the digital exclusion in the UK, the so-called digital divide became apparent and impossible to ignore. During this time on behalf of the University of York's Directorate of Research and Enterprise and Screen Industries Growth Network, I was able to supplement my PhD research with a targeted deep dive study of the lived experience of digital exclusion in the city of York. This focused on six ethnic minority and marginalised communities who had participated in or were otherwise associated with my PhD research, including York's Islamic traveler, homeless refugee and socially and economically disadvantaged communities. I interviewed community leaders, mentors and mentors, members of the communities to better understand the lived experience, issues and requirements of the communities they served. A lot of great research has been and continues to be done in this area, much of it driven by the global pandemic. However, we can still see two common myths that dominate the digital divide conversations. The first is a myth that digital exclusion is a generational issue, mainly impacting older people and further an issue that will naturally resolve itself over time. Because of the second myth, that younger generations are all digital natives. But actually in my study, I found that while excellent with mobile devices and apps, many young people struggle to use PCs to successfully store entry files and grasp common applications such as Microsoft Office, resulting in many leaving school without digital skills needed for employment or in further and higher education. What quickly became apparent to me was how simplistic both this understanding and the solutions driven by it are particularly within the context of minority and marginalized communities, which I'll discuss further now via five interrelated key themes to come out of this work. First, a little scene setting to determine the overall scale and scope of the digital exclusion. Sorry, I'm just wondering if you can see that clearly. Within the boundaries of this study, all participants were asked the same opening question. How would you define the digital divide? While all participants defined the digital divide as a form of access deficit, which often but not always are indeed explicitly referred to online access, their use of diverse terms such as exclusion, difficulty, deprivation demonstrated perhaps unconsciously how broad and multifaceted the nature and impact of the digital divide is. There's also a clear recognition from the participants that while overall affordability was a recurring theme, a situation exasperated in a city like York, there is no single underlying cause for digital exclusion. Nor is there a direct link between specific issues and specific ethnic minority or marginalized communities, which contain a variety of digital skills and needs. So to the key findings, participants felt that given the variability of digital skills within their communities and their innate vulnerability, they themselves are best placed to identify instances of digital exclusion and to determine the type of support needed. They noted interestingly that the street saviness that they saw in many of their client and service users was not evident online. However, it was also felt that to be truly effective, this must also be delivered through collaboration between themselves, between the communities, and between them as a group and the local authority. A network underpinned by the ethos that an individual connected to one was connected to all. Participants, often members themselves of the communities they serve and often self-taught, acknowledged that their own digital knowledge and skills had limits, and that it was difficult, if not impossible, to keep up with rapidly changing digital world, let alone share this effectively with their communities. They all felt that they would benefit from access to dedicated expert central resources, providing timely and consistent advice. Participants explained that for some members of their community, complex issues and intersectionality may prevent or restrict levels of digital inclusions indefinitely, requiring an ongoing assisted access service and support, just to maintain current digital competencies. The fear being for them that individuals would knowingly be left behind by society. However, it was strongly felt that this should not constrain the potential of or opportunities for the community as a whole, and that long-term development programs are required to provide the advanced digital knowledge and skills demanded by today's workplaces and indeed the world in general. Participants acknowledged that the development of digital skills could and should not be separated from the development of wider life skills. Seeing managing personal finances and budgeting being key to digital, especially data affordability, as strong risk assessment skills are to online safety. Their experiences showed that the best outcomes were achieved by taking whole life approaches tailored to the individual, but taking into consideration the wider family and household situation. As one participant noted, when discussing unexpected or high data charges, if an individual goes into debt, the whole household goes into debt. Participants shared their frustrations with digital first strategies, especially in relation to essential state services such as universal credit and housing. Central to this is the remote development of processes and systems which do not consider the specific needs and limitations of ethnic minority and marginalized communities. Many of these communities already provide cultural competency education and would welcome the opportunity to co-design services and support programs. Turning now to briefly consider solutions rather than issues. What came out strongly from this study was the absolute determination to change the situation and as communities accustomed to not being the primary target audience, fully expecting that the change would need to come from within and be led by themselves. Many grassroots initiatives to counter digital exclusion were developed and launched in York over the last couple of years and I'd like to quickly introduce three here. Launched just last week, the public living room. This was launched by the York Travis Trust based on the public living movement from the international Camero Ados movement. Basically it's based on mutual aid. It's, as its founder described it, they're human spaces for folk to be alongside each other during the good times and the tough. There are places to meet new people, have a chat with a couple with no agendas, outcomes. A place where you have permission to be rubbish. It's amazing what happens when you come in without the intention of trying to fix people. People often have the answers themselves and here in terms of digital inclusion, that's exactly what has been encouraged. No pressure, you can come in spend all day in the public living room but they do provide laptops and they also provide support to access online and they also provide private rooms if you need to take Zoom calls etc. The next initiative is by an organisation called Restore. Founded in 2010 Restores, primary aim is to provide accommodation and support for individuals experiencing homelessness in the city of York for whatever reason and today provides accommodation and support for 39 adults in 11 properties. However, the attitude, approach and outcomes of Restore set their initiative apart. Taking a person-centered approach starting from whatever point in life each individual is at Restores aim goes significantly further than they're a resident's immediate safety and support although that is vital. Recognising a lack of positive social relationships and long-term social exclusion and isolation. Stable community living and learning including digital and other life employability skills. Recreation and contribution creates the foundation for whole of life development of individuals matched to their individual needs and at their individual pace and continuing until individuals have the skills and confidence to live and work independently. And finally one very close to my heart, the digital inclusion labs. Digital inclusion labs are run by an organisation called the Good Organisation which is a very appropriate name and initially it's sought to help homeless community during the pandemic by offering practical resources ranging from solar powered backpacks allowing rough sleepers to recharge their phones through to the provision of mobile devices software and basic IT training. However the initiative has grown and it now provides a supportive environment where people with lived experience of homelessness can acquire a variety of skills and knowledge related to digital systems from basic foundation level computer literacy to app development coding and the application of proximity based technology. So just to finish up just some key takeaways from this talk I just want to emphasise that the digital divide the problem is really social rather than technological that I believe success lies in community knowledge rather than technical knowledge and allowing communities to need and co-design services and initiatives and also to plan for and commit to long-term collaborations and holistic whole life approaches. So thank you. Thank you very much Claire that was that was really interesting. So we move on now to our second session and this is entitled by accident or design diversity inclusion and innovation in born digital collecting at the National Library of Ireland and we welcome Della Keating to give us this talk. So Della works as an archivist in the National Library of Ireland and for the last number of years has been involved in pilot projects to enable the library to collect digital content. Her talk today is an opportunity to reflect on the impact of these projects on the work of the library for inclusion and innovation point of view. Hello my name is Della Keating and I work at the National Library of Ireland. Before I start I would like to acknowledge that I'm talking very much from my own perspective here today about the work of digital collecting that we do in the National Library of Ireland. I recognise this is quite a specific interpretation in relation to diversity and inclusion and also that what we consider as our progress in this area may not equate to everyone's view of progress. I would also like to thank my colleagues Joanna Finnegan and Kiran O'Leary for their assistance in developing this presentation and to our department head Ono Karagon for his support. So just to give a little bit of background to the library the National Library is the library of record for Ireland. Our unique collections are permanent accessible by all and constitute the most outstanding collection of Irish literary and documentary heritage in the world. The mission of the library is to collect, protect and make available that recorded memory on behalf of the state for the people of Ireland and what you can see on screen there are two representations of our lovely reading room one of which is a Lego representation. The National Library has three collecting strands published special collections and digital and digital includes both digitised content and born digital material and within born digital we also we have a fairly relatively mature web archiving programme as well as the collection of unique born digital content and this started in 2017 in the form of a number of pilot projects and it is really that work that is that is what I am talking about today in the presentation. So today's presentation is an opportunity I suppose five years on for us to reflect on that collection of unique born digital content and has led to the title of today's presentation where I want to explore how collecting digital has opened the door to inclusion and innovation on a number of different fronts so you can see those fronts highlighted there so one is it has led to more diverse and inclusive collections the other is the impact on the way we work on those collections and finally how it has led to the development of the new forms of engagement and this analysis is framed very much against a backdrop of governmental initiative through our parent department in relation to gender equality a number of years ago now various national library diversity and inclusion initiatives over the last number of years and also the 2022 iteration of the national library strategy which is moving towards embedding diversity and inclusion across a number of areas and I suppose what's been interesting to me as someone who hasn't been directly involved in any of those initiatives but is involved at the cold face of digital collecting activities is teasing out to what extent the library's remit to build collections that reflect society has been influenced both by an increasingly diverse society as well as the collection of digital itself and to see what can be learned and shared from that analysis so I suppose as a national library we're very aware that we do not operate in splendid isolation over the last 25 years Irish society by any measure has become increasingly diverse and it is this change alongside the advent of digital that has provided us with the opportunities for inclusion and innovation in our work and what you can see on screen there is a screenshot from one of our web archive collections and it represents most but not all of the languages that information in relation to COVID-19 had to be translated into in order to reach the various communities it needed to reach so at a policy level these changes are visible an examination of the 2022 collection development policy which is due for publication later this year reflects a shift in from viewing our role as impartial actors in collection development to a more active role to represent Ireland in all its diversity there's also an acknowledgement that some communities are underrepresented in our collections and reference to involving these communities more in the choices that we make so from a policy point of view it's also important to reference the role of the diversity and inclusion committee which was set up in 2018 it's cross-organisational it's chaired by the director of the library and it is board accountable and in terms of practice I also wanted to refer specifically to the important role web archiving has played in helping us transition from a collecting physical only model to something different and I'm specifically not saying transitioning to a digital model here and this is because web archiving itself is at times a very reactive activity requiring quick action and responsiveness to emerging events where previously decisions in relation to collection development could be considered from a number of different angles in terms of current and future implications decisions in relation to web archiving challenge this model and help us pivot from ponderous consideration to a model where at times there is a need to document the now and to make the decision now where it seems for example that emerging issues and trends are put on the collection development radar at warp speed compared to how things happened previously and this has paved the way for a number of changes so changes in how the decisions get made there's more probably explicit risk analysis quicker lateral and upwards lines of communication it's paved the way for on the fly expansion of collection criteria incorporating new voices who are very comfortable online and who use online to get their messages across but are blissfully unaware and don't care about the lack of permanence of the environment within which they are operating format wise it's also paved the way for responding to emerging ways of communicating so I'm very conscious that 10 years ago when the web archive was starting out we captured blogs and blogs were used very much back then as a way of getting more personal opinions and messages out there in the same way now that that is captured by social media for example and in the digital world of course there are no geographic boundaries or borders and this is something of importance to a national library where our legal mandate is to collect of Ireland so by 2017 we had engaged in a number of pilot projects to collect unique only digital content and I can only give a flavour of those collections today our first acquisition related to Marion Keyes an Irish author of international renown and a recent author of the year winner at the British Book Awards we recognize Marion not only in the literature sense but also in the digital in terms of how she interacted with her substantial online presence across many platforms and some of these we could capture in different ways such as through our web archiving program for example and her physical output through our legal deposit we visited the donor at our home we photographed her workplace her laptops very much trialling the enhanced curation model of the British Library and all of this is done with a view to providing the user with as much additional contextual information as possible in this new space where the object is not tangible and you can see some of what I'm talking about there on screen at present the collecting areas of the library are very much organised on format lines such as photographs in prints and drawings and there's clear division between visual and text material but our newly acquired born digital collections have thrown that concept into disarray and our unique digital collections have shifted us into a position of needing to manage multi-format collections as an entity as a single entity so here are some lovely images from a digital photographic collection documenting the 2015 marriage equality referendum where Ireland was the first country in the world to legalize same-sex marriage by the popular vote with this digital only collection we could build on work already done by the library special collections department and as you can see there on the slide we hold the most extensive LGBTI plus physical archive in Ireland other collections acquired since 2017 include Waking the Feminists which documents a grassroots campaign for equality in the Irish theatre and there's a quote there on screen from Lee and Bell who really spearheaded the campaign and as you can see the campaign from the photograph you can see that the campaign itself did receive international recognition we've also acquired a collection called Senses of the Heart and this collection contains English and Irish language spreadsheets containing survey data that in the year of the 2016 census so we had a state census in 2016 and running to parallel to this this survey was conducted asking people more about their emotional responses rather than the the clinical hard research data of the census so their responses to issues such as homelessness and health and so on so from where we stand looking back over those collections there the barriers to inclusion are constantly being wrackled and the notion of what constitutes the collection was and is constantly being challenged but collecting digital has also impacted on how we work after a few failed attempts at getting the collection of digital up and running the library made a conscious decision in 2015 to set up a department devoted to the developing managing and preserving the national library's digital collections both born digital and digitized and the makeup of this department was different to other library teams for example the hiring of software developers had started prior to 2017 so it had started at a time when the library's digitization workflows had started to mature two years ago we had our first digital preservation hire digital preservation was already implicit in how we did things but this post was an explicit recognition of the incorporation of digital preservation in the end-to-end life cycle of a digital object the team has also expanded to incorporate a recent graduate intern who brings up-to-date theoretical knowledge to the team and a shout out to Connor and line it there I hope he's listening in we also have librarians and archivists who've moved from working with physical collections to digital and the building of this team has led to a diverse skill set all in one location all learning from each other and the original aim of putting it together in this way was to minimize the bureaucratic barriers that sometimes exist when trying to get time to work together and progress various initiatives and the irony was from where I'm standing was that once these barriers these bureaucratic barriers were minimized our interactions have actually been more formal and with a more formal project management type on the curatorial side we have moved from a more considered unilateral approach to a more iterative on-the-fly process in relation to our work we found ourselves absorbing the iterative practices so beloved of developers also not being afraid to fail and not feeling the need to get everything right first time and it shouldn't have come as a surprise but it did that we regularly need to have check-in conversations in relation to developing shared terminology and shared assumptions in order to progress our work so moving on collecting digital has also allowed us to become more innovative in relation to how we engage so there's a lovely image there of our yes equality donors Grania Healy and Brian Sheehan over the last decade the library has expanded its model of user engagement beyond what was previously the primary point of engagement and that would be a reading room and also through exhibitions and something this is something the library did and does very well like many other institutions and this model frames user engagement at the end of the life cycle where in a world where all processing work is completed but work on our pilot projects has taught us that our donors are major stakeholders in the collection process we now have a new model of interaction with them with an ongoing relationship throughout the life cycle the various stages so this is meant that the donor is front and center in every activity whether that is conversations about access about collection content about sensitivity issues about methods of transfer involvement and transfer description information and of course the frustratingly never ending and sometimes circular conversations about rights it is as Jesse Lawyer put it in a recent article where titled collections are our relatives it is very definitely about people and not stuff and also as a result of publicizing our new born digital acquisitions we are in receipt of requests for access to these collections and this is something obviously that we are still working on and still working through so these requests are often a source of turmoil for us because as a rule we do not provide access to unprocessed collections but the desire to share and develop new ways of interaction as soon as possible in the workflow is strong and I suppose it was these requests that provided the impetus for us to get creative and collaborative and this manifested itself in the form of participation in an Irish research council grant funded grant where we are working with a PhD student Maria Buckler to investigate what academic research requirements in relation to digital literary acquisitions might look like so specifically the collection the Marian Keys mystery of mercy close collection there and the implications these requirements might have not just on the reading room experience but also further upstream in the workflow and this approach enables us to address issues at a more appropriate point in the life cycle and give us the opportunity maybe to incorporate workflow changes before ships have sailed and we view this process as a way to innovate collaboratively in areas where we have not yet figured out solutions and and obviously in areas where we may not it makes no sense necessarily to figure out the solutions by ourselves so we do not want to think of our users as waiting possibly at a virtual desk at some point in the farthest of future when all processing is completed I feel very privileged to be part of the digital collections team and we know that we are on a journey the benefit of hindsight or reflection such as today also has an impact on how we tell our story to ourselves as well as to others but what is clear to us is that collection digital has provided opportunities to become more diverse in terms of what we collect how we collect it and how we engage and it's also clear that foundations were being built to help us avail of opportunities that came our way and I would also say that by design rather than by accident that the explicit outputs of the diversity and inclusion committee has provided me with a lens through which I can question how and who makes the decisions so I just like to finish with this quote there on screen it's from a 2017 Australian conference it was kind of one of their taglines and it really it kind of sums up for us maybe where we might want to get to where our next horizon might be and the thoughts we need to bring us on our journey so I just want to say thank you for your time today and to the organisers for the opportunity thank you very much that was really interesting very stimulating so we move now to our third presentation final presentation for this session which is entitled crowd sourcing the history of slavery the benefits and challenges of public engagement on sensitive topics and our speaker now is is Graham Jevon from the British Library Graham currently works for the endangered archives programme at the BL and he was also one of the library's 2021 Coleridge fellows we've conducted the agents of enslavement research project that he's going to discuss with us today so hand over to you Graham for our final session hello my name is Graham I have brown hair I'm wearing a blue check shirt and thank you for attending this panel today so in June last year I launched a crowdsourcing project titled agents of enslavement and today I want to reflect on the benefits and challenges of citizen science as a research method on a topic that contains offensive language and upsetting content before I proceed with the presentation I do want to warn people that this project relates to the history of slavery and source material therefore contains content that is now considered discriminatory harmful or offensive I've tried to limit the extent to which overtly offensive material is included in the presentation but I want to warn people that there may be some references to such content first off it might be useful if I provide some background to me and the project so three years ago I joined the endangered archives program or EAP for short at the British Library fellows unfamiliar with EAP our program funds the digitization of endangered archival material around the world and this small map in the bottom corner gives an indication of our geographic spread today we funded more than 400 projects and made more than 10 million images freely available online and the very first project that I worked on in 2018 was a collection of newspapers digitized in Barbados by the Barbados archive department which you can see here and it's very apparent that these late 18th and early 19th century newspapers contained a wealth of information about the history of slavery and there have been a number of attempts to engage with this content the Barbados archives themselves run a workshop with local people which you can see here but I felt there was an opportunity to use digital methods to open up these newspapers to research and interrogation and thankfully I was awarded a short research fellowship that gave me the freedom to start doing this and I should say that my background as a researcher has been to use non-digital methods but I recently completed a postgraduate data science degree and this project was really the first opportunity to put these skills to use in a research context the main thrust of my work during the fellowship was to use crowdsourcing as a means of creating datasets for analysis so we have here a screenshot of the crowdsourcing homepage and you can see that the main name of the project was twofold so firstly to identify the ways in which colonial newspapers facilitated and challenged the practice of slavery this was the overarching research question and secondly to create a database of people places and events found within these newspapers and the main objective of the crowdsourcing was to extract the data from these newspapers that would populate the database which in turn could be analyzed as a means of answering that research question but it's also hoped that the database will act as a resource for people researching their family history and I know that there were people who got involved in the crowdsourcing because they were keen to find a mention of their ancestors um now to create the crowdsourcing site I used the the Zooniverse platform um anyone can use the platform to set up a crowdsourcing project and promote via their own channels but if you want to tap into Zooniverse's pre-existing community if you want Zooniverse to promote your project and make it easy for people to find you need to go through the official Zooniverse review process and that's what I did now part of the review process involves having members of the public test the project and then answer a feedback questionnaire and I think the feedback question that was of most interest was that when people were asked if they read the additional information pages the overwhelming response was no and this suggests that people were diving straight into the tasks and for many projects I don't think this would be a problem but for this project the additional information pages contained content warnings and contextual information about the language and history of slavery and with this type of project it was important that members of the public were not unwittingly confronted with offensive and upsetting material and that when they did they had resources to help them understand and deal with this um yeah that's why we we used the additional information pages to provide contextual information including a language of slavery guide and some advice on how to cope with upsetting content and on that you know it was encouraging if if that's the right word to see people post in the discussion board that they were taking our advice and having a break from the project you know sometimes after seeing something that particularly upset them sometimes after a general accumulation of consent and it was also nice to see these people come back after a period away but partly because of the feedback results we also made two decisions based on the assumption that many people would not read these so firstly we made a content warning appear on every page this is the blue bar that runs across the top of the screen um there was a little bit of pushback on this with some people commenting that the content warning restricted the space available to complete the task and here you can see it on the task screen um I therefore made some adjustments to the size of the bar but to affirm that it remained necessary because the project was continually attracting new users and the second decision made on the assumption that people would not read the additional information pages was that when promoting the project on twitter for example we never directed people directly to the crowdsourcing site this is completely countered to what we might ordinarily do but we chose to direct people to a blog post about the project the blog post then directed people to the crowdsourcing site but it also contained contextual information and therefore helped act as a buffer to the content so it's probably time I started talking about the content and the tasks um now there were two phases to the crowdsourcing project now the aim of the first crowdsourcing task was to create a data set of adverts and notices relating to slavery um to that end people were asked to identify four types of advert or notice that they were for sale adverts uh wanted adverts fugitive or runaway notices and captured notices so people were asked to look at a full page image of a newspaper like you can see here on the right and they were asked to find adverts that fit into these four categories and to draw boxes or rectangles around them um now the community were presented with about 12 000 full newspaper pages onto which they drew more than 90 000 rectangles and after I had aggregated the results we ended up with a data set of more than 25 000 cropped adverts across the four categories now the second stage of the crowdsourcing project was to extract the content of these adverts in order to populate the database so the 25 000 cropped adverts were then fed back into the crowdsourcing site for transcription via a workflow of questions that needed to be answered about each advert and as these two screenshots show um some of the questions provided multiple choice answers others enabled three text answers now when this data is fully inside the database you will be able to analyze the data at scale and make connections between people, places and events um to give you an insight into the kind of connections and stories that can emerge I want to follow part of one particular story uh briefly uh first I just want to warn people again that the next few slides will display adverts from these newspapers that that may contain offensive or upsetting language so this is the the story of golly bridges and it begins on the 27th of january 1807 when john and harry at lagoos posted an ad bar in the bar bedos mercury seeking the capture of dolly bridges and her three children who they claimed as their party now a few days later another notice was printed in the mercury and you can see in the bottom right that the person who posted this notice was dolly bridges who described herself as a three black woman now responding directly to the previous notice seeking her capture dolly asserted that the public are respectfully informed that if any person whatever should have any doubt respecting her or her children being otherwise but what they are bona fide free people that referenced to the secretaries and marshals offices in this island will fully explain their legality so dolly was asserting the freedom of her and her children and resisting enslavement this altercation appears even more striking when you go to the full page image and see the two's notices side by side a reprint of the legacies claim to dolly bridges in the right column and dolly bridges rebuttal on the left a few months later the legacies posted a marginally revised version of the same advert which suggests that their first attempt to claim ownership of dolly had failed and again in the next edition dolly rebutted their claim immediately this time she challenged them to take her to court so in what seems like a rather mocking tone she asserts that the pretended claimant to herself and her children had better seek a more legal way by a course of law than by a newspaper claim as she quite clearly seems to be mocking the legacies for using a newspaper rather than a proper legal process to state their claim so it's almost like a 21st century twitter spat with newspapers in this instance providing the platform for a very public dispute now one reason that I flagged this particular story is if I go back to the four categories of advert that people were asked to look for this is a good example of where an other category might have been helpful now dolly's notices do not technically fit into any of the four categories people were asked to look for and so technically they should not have been identified but they were at least one crowdsourcer decided to break the rules I had set and in this particular instance I'm very glad they did there is more to dolly's story but I don't have time to talk about that today except to say that over the course of the project you know several people noticed dolly's story you know some people would flag the only advert that they saw some people would see more than one advert no disconnections and one person even started the hashtag go dolly and this highlights one of the key benefits of this project I think beyond any research aims and that is public engagement and I think that keeping public engagement in mind is important when considering the benefits and challenges of crowdsourcing as a method and this is what I want to do as I bring this talk to a close you know if I go back to the beginning to the crowdsourcing review process when it was first tested by the public while the response was overwhelmingly positive one member of the public answered no to the question is this project suitable for zooniverse and they gave the following reason it just seems like we're doing someone's research for them this was just one comment and at the time I thought well yeah that's kind of the point of crowdsourcing but I'm now in two minds about this and I'll explain why as I reflect on the process so looking back I think the first stage could not have gone better if I'd taken a traditional research approach and trawled through the newspapers myself I could not have identified and cropped 25 000 adverts in three weeks I was delighted with the contribution of the crowdsourcing community I was pleasantly surprised that my machine learning algorithm worked and this task you know can now be very easily replicated for a potential new project you know I now have an efficient template that can be repeated if I was to make one change I think I might be inclined to provide another option to give people an outlet to highlight things that don't fit the expected mold you know crowdsourcing is quite a rigid method you want to give people a very clearly defined task the downside to this is that you are potentially restricting unexpected discoveries and that leads me on to my my biggest concern about crowdsourcing as a research method so traditionally I have engaged in research by visiting archives crawling through documents and making notes you know either researcher and personally immersed in the collections in this project either researcher have been quite distant from the collections you know I worked on setting up the project and processing the results but I did not get immersed in the newspapers that task was outsourced to the public and as a researcher I think I lost something there this got me thinking about whether I should have done the transcription tasks myself maybe to test this I'm currently transcribing one of the four collections of adverts directly in the database myself with the other three having already been transcribed by the public I don't have concrete statistics on this yet but my instinct suggests that this particular project would be perhaps more efficient when I do the transcriptions myself rather than using crowdsourcing but I'm still wrestling with the question of whether crowdsourcing is still the preferable method because of the public engagement benefits because crowdsourcing did not just generate data you know it generated interest and it generated debate it led to articles in the observer newspaper and who do you think you are magazine you're crowdsourcing for attention to the project that would not have been possible if it was just me and my computer crowdsourcing created a community of nearly 4 000 registered volunteers and countless unregistered volunteers who completed more than 120 000 task workflows and engaged in more than 5000 comments on the talk boards you know the method of crowdsourcing connected people to these collections allowed people to interact with them and immerse themselves in the history and this would not have otherwise happened you know the newspapers have been available online for two to three years but have not been viewed anywhere near as much as they have in the last few months and so on reflection you know might have been selfish of me to deny people this opportunity by hogging the research process to myself and this fact is brought home when I reflect on some of the comments that people made in the discussion board you know for different reasons people expressed their interest in the project when it launched you know in the bottom left a lecturer tweeted about using it as an archival training tool for students on the right hand side of the slide you know a lady was hoping to find information about her ancestors you know some people were just grateful for the opportunity to help uncover the hidden stories of people who have been oppressed despite the serious nature of the topic people had fun with the project you know they enjoyed getting lost in other articles and not simply limiting their reading to the task itself you know people had fun trying to decipher this coded advertisement on the left and several users found this 18-19 article about a smallpox epidemic and the need to vaccinate all persons very topical but these were transgressions and while people had fun with these little asides the community were conscientious and thorough in completing the tasks and engaged empathetically with the content you know at times people found it hard and these two comments on the left both came from people who felt the need to take a break from the project but as I think I mentioned earlier you know these people came back but it was hard to know that people really felt like they were doing something worthy and the quote in the bottom right is just one example of someone who felt they said they felt proud to be helping and there's comments like this that reassure me that crowdsourcing was the right way to go with this project or at least for part of this project because having other people do my research for me did at times make the project more efficient but crowdsourcing can also increase the workload but that doesn't necessarily mean it shouldn't be done because central to any crowdsourcing project I think should be the question you know what is the crowd getting out of this project and with that thought I'd like to finish thank you thank you very much Graeme a fascinating project with some disturbing elements obviously in terms of content so could I now all our panelists to actually set themselves onto video so we can see the see you all that's fantastic thank you very much and we've had a really interesting selection of questions come through in the Q&A please don't hesitate to set more questions as we go through some of these now and invite you to post more questions so perhaps I could start with Della if that's all right so we've got a comment that's really interested in your stakeholder engagement with donors and wondering if you could say a bit more about how that process works so is each relationship to manage differently or is there a standard model for navigating that process and very interested that actually now have I guess collections which could continue to grow from that individual donor so perhaps you might reflect on that element as well yeah thank you very much Liz just checking you can hear me okay yeah grant and so I suppose it has been a wonderful experience working with the donors I guess we've considered ourselves very fortunate but also very aware that not everybody has the time to work with donors the way we have been able to on this project but actually it became a necessity so maybe for some people the disappointing news is that there is no one size fits all it's been a very bespoke kind of setup and I guess very structured around our initial foray which was in terms of the relationship with the donors which was very much built around what we called a survey a donor survey and this was a kind of very detailed document that we were very nervous about broaching with the donors at the start because it was so detailed there's two page long list of questions in relation to their collections but we hadn't held them through that process and actually it's paid dividends and I think as you highlighted there Liz the relationship isn't ongoing one so we thought initially that once we had the survey done that we kind of promised the donors that we wouldn't pester them as much after that but actually they're happy to be pestered because this is their baby in a way but we have had to continue to engage with them but none of them have a problem with that but the levels of engagement so I would maybe just highlight I'm not a particularly techie person myself and some of the donors are quite techie and some of them aren't so there's been a full kind of spectrum of engagement there so it's been really interesting and a wonderful learning experience for us I hope that answers the question in some way I think it does thank you Della Claire perhaps we could turn to a question for you a reflection that whilst you said your talk wasn't about collections and everything was a great amount we need to learn from all your research I guess the question arises from how to become visible to those communities so that they know there are potential possibilities for us to get involved and to assist and do you have any ideas about how we might most effectively make ourselves known to those communities that's a great question from personal experience it's about becoming a trusted partner and that is a meticulous and slow process and necessarily so so the short answer is really you know reach out and and and say hello but the the reality is that going going into this go in the long term and go in to being prepared to move slowly go in to listen first I think a great way to build trust is to perhaps go in with the idea of give first we're very lucky and very privileged we have many things to share you know whether that's resources such as equipment learning venues even if it's just time so yeah so do reach out in my experience all the communities that I've worked with I'd be very very happy to hear from people and don't expect anything to happen overnight but I will say that once you are a trusted partner and a trusted ally that is enduring and also you will realise how many underground networks there are and how these communities are interrelated and work very closely with each other and you will you will be known very quickly thank you um question for you Graham um around how accessible is your data now to perhaps other researchers who might want to use it to pose other questions is it is this now a reusable data set or is it really bounded within the areas that you're particularly researching um so it it will be it says not available yet it still needs to be work done on it but I mean that that's one of the key things about Zooniverse so if you do have a Zooniverse project one of the requirements is that you make sure the data that you come out of that project the crowd community create that is open access so so yes the at some point that the data will be made available in a kind of in a raw sense so people can just you know start working with the data as it came out of Zooniverse and but also in a curated sense you know where I'm cleaning the data and putting it into database so people can hopefully um uh get into that data at a level that's suitable for them you know right at the beginning or you know that sort of curated versions but not yet but definitely work fantastic thank you um perhaps um Della another question for you um particularly around the Marianne Keys um material and did you ingest potentially sensitive personal data from Marianne Keys and is this one of the reasons um that you're delaying access to unprocessed material to kind of mitigate against any risks that might be occurring there um so in this instance no I saw that question popping up uh because we worked very closely with Marianne at the start and asked her to kind of go a long way towards deciding what you wanted in and what you wanted out of the collection so these are the four collections I showed there are different pilot projects so we're kind of testing different things with each project so there is information like that coming in in other collections and we haven't quite got to exploring uh areas of redaction and how we're going to deal with that but we're we have them teed up if you like to explore so I'm happy to chat offline with anybody who wants to chat with me about that because we're always looking to learn from how others have done this that's a really interesting point actually is there a sense that you are collaborating with the other universities or other libraries across Ireland um particularly I was reflecting on the the normative of um web archiving and whether you're actually working in collaboration and pick up particular areas and pass others on to other collaborators um I guess uh at the moment um the National Library is heavily involved in web archiving and we lean on other libraries and other organizations to feed into the content um and you know whatever the universities and that are doing it's largely not necessarily collected connected to our own web archiving program which is very specific to of Ireland um but we're always looking to lean on and work with um other universities other libraries other communities across Ireland and also beyond you know we've had huge help and input from to many many organizations across the world who are involved in web archiving that's really interesting thank you um another question for Graham do you think crowd sourcing would work as a way of getting undergraduate history students comfortable with primary material and if so why and if not why not uh yes I think it would uh and yeah there was a there was an example I think in the in the in Twitter where someone's used that and I've and I've had other examples where people have mentioned to me about using crowd sourcing for their undergraduate students so I know that people are interested in that um I guess if if I had to say why I think it would be good I think because there's that level of interaction it's um um it's not just reading if there's a task to go with it so someone's got a clearly defined you know reason for for doing something so they can sort of okay so you think that the crowd sourcing hopefully you can see what's going on there's a question there there's a research question there's a task and you can hopefully see that the link between what's what's going on but also seeing uh let's say um uh how a dataset can be can be used so you know just looking at one single document you're looking at you know it could be 10 20 30 documents over the course of an hour and hopefully you can see that you know the difference how one document has value and how a document apparently doesn't have value so I think you know and there's probably lots of other reasons why it's probably a good idea and probably better reasons than than I just articulated um but yeah so I certainly think that um yeah the crowd sourcing as a as a means of interaction and in the way of a means of control because a zooniverse project for example so mine was was very public project but with zooniverse you could just use their site and not make it public uh so you basically just give it to anyone that has access to the url so you could sort of contain classroom environment but you you use that platform as a way of uh sharing material uh and presenting it randomly to people and seeing what they what they come up with so yeah I think it it could be a good a good tool for that purpose yeah I think to a degree that relates to another question that's come up which is around your decision to actually pursue data science and I guess from my perspective looking at it I think it would be a great way of actually encouraging history students to think about new research methods data science but what prompted you personally to take that decision to actually move to that data science course um I guess I mean partly it was so I've been interested a while of moving to sort of digital methods because I'm working in a in a sort of digital archive there's lots of digital things that need doing so not just from a research perspective but just from a general day-to-day archival practice um uh this is you know just things that take a long time that could be quicker if you if you automate that process uh and so some of the things that I did in the course for the project for example was just a way of um one of my particular problems in work was uh generating archival references catalog references for the for a large date of a large date set a large collection you know protect days in excel then they create this sort of little python program that did it in a few in a few seconds so really actually you know starting point for wanting to do that was just to make my life easier in the in the in the workplace um but also I stick quite closely with the the digital scholarship team at the British Library and there's always hearing about new things that are happening and new new methods and and and new and you know exciting ways of interacting with collections and presenting it to communities and so on and so yeah all of those kind of things it just um yeah it was good to try and um be more involved in that and to start contributing to that and sort of understanding where you know archives are are heading because you know a lot of material is either being digitized or as we mentioned today you know born digital as well so yeah sort of moving forward with with where archival collecting is is going and and making things um accessible as you know um as people heard from Claire's um presentation as well. Thank you. Della perhaps another question for you um we've got a queer about could you expand on how you handle language barriers in your engagement and I guess you've got various languages going on yourself in terms of curation and archiving you have your techies now and you have your your donors and perhaps other audiences so it looks like a quite a complex landscape so how do you manage or any tips for that language issue? Yeah well the first tip is to approach it all with a massive sense of humor because you could end up going down a hobbity hole so I guess again this the full mix of terminology that I would be familiar with um that I I would have assumed I would have been on the same page as the tech people so words like file folder directories it turns out they mean different things yeah different groups and who knew um so there's that and then there's the more technical terms that obviously we have to take on board um you know as a result of the work which is all great and you feel like you're learning something so that's like your digital primitives and your derivatives and all that sort of thing so the we were very explicit we we once we realized there was a barrier there we were very explicit and said you know what we have to devote time explicitly to this so kind of press pause on the work that we were doing and figure it all that stuff out before we could proceed and it it's obviously built into the documentation now to help us so that you know it's a reference point really and um you know I remember one of my colleagues sending me uh uh there's a a pop song by the group the primitives you know so these are the ways that you get a bit innovative and these are the ways these things stick in your head but it has been very explicit that's fantastic thank you um Claire I'm not sure what the topic of your phd is but I was wondering because some of the examples you've given in terms of um interventions and agencies who are trying to move into this area in York are really interesting and I was wondering if you're actually following through to understand the impact of some of those initiatives whether that was part of your work on your phd um yeah it's interesting my my phd was quite unusual within so I'm I'm looking at broadly the area of digital heritage really um or applied digital humanities but what I was really interested in was to see if we could use um archival content so images images really or just items um from archives and collections at a very local level with within neighborhoods to shift sense of place especially for uh minority and marginalized communities um and migratory people as well in those places so I've I've been working in these areas and with these groups for a very long time now or quite a long time since um 2017 um so I am able to sort of have that that long-term view and I'm still involved with with several of the groups now as in a trustee or an advisory um position so yes I I'm sort of seeing these projects come through now and um these initiatives and again it's it's that sort of a privileged position um able to see the interconnectedness of it all whereas and and before this the doing the phd and doing the research I I was very much like many people um obviously I knew of these communities but I didn't know anything about these communities really um I'm not sure if that answers your it does thank you I do look forward to reading your thesis um when it finally appears that would be fantastic good luck thank you so we I'm going to try and pull two questions together it's it's you Gray I'm I'm going to start at these two and it's it's really about the the crowd sourcing process and your volunteers um um so first of all starting about did you know much about your volunteers as they appeared you know about background and diversity and what your the overall body of your crowd sources looked like and then were they actually able to track those stories that they'd been working on um through to their end and what kind of control that process gave the participants in their journey through the material I think the short answers to the two questions are probably no and no so so in terms of who so I like so I don't have access to the personal data of the people so really I have any access to that the the username so that that is stored or gdpr purposes with the universe so um the only information I have is is information that people might reveal in a comment or a question um but I don't have any real statistics of who people are what their background is um I mean I would say in general the people that were involved in the project weren't necessarily subject specialists it's just people that are interested in crowd sourcing so they might be doing stars one day and then this project you know at the moment but they're not necessarily either a new to the topic a new to the to to the period um although even then I'd say that there's I'm relying there on what I know about people who are posting comments because some people you know some people get involved in special powers but a lot of people they just do the task and you never hear anything about them and so yeah I'm I'm fixated on those people the the second question about tracking materials so the the collections of the images that people are working on the tasks they're presented randomly so they say you can't suddenly say I want to see this newspaper or the next page that's not possible some people want to do that you know there were people that were experts they'll kind of try and piggyback on the research and go I want to you know use it for my own purpose can I just look at the next page and the next page um they can do that by going to the web to the to the dislar guidance but they can't control what's presented to them um but on the crowd sourcing site what they can do is when you are presented with an image you can favorite it or you can add it to a collection so you can come back to it as a later point and see that image again if you start seeing things if you start seeing connections you know if you saw the you know two pages that related to that dolly story you could face with them but you couldn't guarantee that you would see anything or that you would see anything that came came later so there's not much control only the only control um saving something they've seen that they couldn't control what they saw on there that's really interesting and is there any sense of um understanding people's satisfaction with the platform is there any feedback you are solicited or or getting in terms of that crowd sourcing experience from the user perspective yes I mean that that came particularly towards the beginning of the project so partly in the the feedback question there but also actually more so when after that point when we went live with it and and suddenly get a wave of new new people you know thousands of people coming coming at the site um and different people would have different questions or different um issues with the site so like I mentioned in the presentation some people thought that the the content warning got away of the of the task you know other people would say like the to the task of throwing a rectangle um you know one of the issues you talked about diversity and inclusion um and this kind of prioritizing with what Claire's talked about um you know um there's a lot of digital exclusion and and and so some people were not some people are very tech familiar and they can they they're familiar with the task and they can just do all the things easy other people were really new to it they came to the project because they've seen in the newspaper they weren't really familiar with technical things and some they found it difficult to walk that rectangle from one corner to the next example um and some of this some of this might be accessibility accessibility issues some of it may just be that people just you know they're just not familiar with using a mouse um and so the other issues like that were people saying oh I'm struggling to actually complete the task um so yeah there were issues that people would and some of these things I could you know fix uh some things I can't you know it's just that's the way it is and uh and then you might feed it back to the Cozooniverse and they might put enhancements but yeah there were small teams they you know they're constantly enhancing the project but they you know they can do so much