 Hi everyone, thanks very much for coming and thanks Diane for the introduction. I've been working on this research for a little while now. As many of you will know I've had some really valuable feedback and contributions from practitioners along the way right from the very beginning actually at my proposal stage. So it's a privilege for me to stand up here today and share some of the findings of my work and I hope it'll be useful for you as you consider crowd sourcing as an option or perhaps as you continue to tinker away at projects in progress. I also just wanted to shout out to Trevor Owens and Michael Luskarides who are joining us here at the conference this year. They've been a really big inspiration for me over the last few years in their work on crowd sourcing so it's really a buzz for me to to be contributing to the same kind of conversation. So let's crack on. When I set out on my PhD adventure I had a few questions buzzing around in my head. What are the aspects of design that influence participation and contribution quality on websites for crowd sourcing cultural heritage? Which aspects of design are more influential than others? And how could websites for crowd sourcing cultural heritage better support participation and quality contribution? These are just some of the questions that my research has sought to answer and today I'll be reporting back on what I found. But first I'll just back up a little bit to set the scene. So these are some of the potential benefits for crowd sourcing. They're the reasons why an increasing number of Glam institutions are crowd sourcing the processing of cultural heritage assets. So crowd sourcing labour intensive processes such as transcribing historical documents, recording personal histories, tagging paintings with keywords, correcting OCR text, cataloging cultural heritage collections. These are better enabling our institutions to create or enhance digitised data for public use and research and engage the wider community. But along with these potential benefits come a number of challenges. So crowd sourcing is still in an experimental phase and projects that involve cultural heritage collections haven't always been cost effective. Crowd sourcing is also an umbrella term for a really wide variety of initiatives. And while the success of any crowd sourcing project relies on sufficient participation and quality contribution, the design of these crowd sourcing systems needs to reflect the particular approach, the type and the context of each initiative. As Diane mentioned in the introduction, there are common project constraints such as limited time resources and expertise and the guidance available for designing and evaluating websites for crowd sourcing cultural heritage is somewhat limited and fragmented. So with these potential benefits and common challenges in mind, I've developed a set of design principles for supporting participation and contribution quality. These design principles are now freely available on my website, nonprofitcrowd.org. If anybody's actually, I released them the first version in March and a slightly revised version in July. And if anybody's using those or has had a bit of time to just digest them, I'd be really interested in your feedback if you want to catch up with me over the conference or after. So this new set of design principles builds on a foundation. It draws on existing guidance for highly interactive websites, online communities, crowd sourcing in general and crowd sourcing cultural heritage in particular. The principles also incorporate the findings of my study which I'll be talking about today and the results of some website inspections and a questionnaire. I just wanted to throw out a couple of caveats here. As I've explained in the report containing the principles on my website, these design principles are intended to supplement, not replace generic principles for usability. Also while they appear quite prescriptive, the idea is that they are one tool among many and ideally a project would also be going through a rigorous requirements process and ideally user testing along the way as well. So I'm going to touch on some of the principles today but I'm not going to run through them all as many of you will often be familiar with them already and those of you who aren't can catch up with them online. But what I did want to talk about is which aspects of design I found to be more influential than others on participation and contribution quality. So to answer this, I collected data from over 250 former, current and prospective users of websites for crowdsourcing cultural heritage. These included people from New Zealand, Australia, UK, USA, Europe and elsewhere. Some of those were affiliated with the cultural heritage sector and tertiary institutions but when you were from other professions as well. Thank you to any of you in the audience who participated in that questionnaire. I really appreciated the support. I got a really good uptake. So based on the results of the questionnaire, I was able to order the rank, the 21 design principles according to the likely level of influence on participation and work quality and the idea behind this was to help project teams prioritise aspects of design and optimise their available time and resources. The literature suggests that in many cases project teams would be working to requirements but would run out of time or funding to implement all those requirements. So the idea behind ranking these design principles is so that they can focus, they can order work through them in order of influence to some degree. So this brings me to my next question which is how could these websites better support participation and quality contribution? So to answer this question, I evaluated a sample of 20 websites using the new design principles. My sample encompassed multiple host types being the institutions driving these projects including galleries, libraries, archives, museums, research institutions and collaborations of these institutions. The sample encompassed 10 common process types including transcribing, recording, creating content, tagging, correcting, contextualisation, cataloging, commenting, critical responses or stating preferences which you might be more familiar with in terms of voting, geo-referencing, linking and mapping. The sample also encompassed six common asset types so cultural heritage assets being text, image, ephemera or intangible cultural heritage geo-spatial assets and cultural heritage assets containing numerical or statistical information. So I'm going to talk you through some of the results from this study hopefully to illustrate how the design principles might be able to assist with designing and evaluating this type of website. I've also organised the principles into four categories to represent the key themes that appear to me to be underlying this user experience. So websites for crowdsourcing, cultural heritage support, participation and contribution by informing users, supporting and engaging users and nurturing and sustaining the user community. So websites can effectively inform users by providing clear, concise and sufficient task instruction showing how project output is freely is or will be freely accessible to the public, keeping the website current, prioritising key information, presenting reasons to contribute, displaying project progress and conveying the credibility of the project. Now I found that the primary purpose of informing users is to support participation based on the results of the questionnaire. These principles were found to be moderately to very influential on the decision to contribute and to continue contributing. So overall the results of my website inspection suggest that the website sampled are effectively informing users but I did uncover what appears to be a common design weakness. Respondents in the questionnaire rated the provision of clear, concise and sufficient task instruction is very influential on both the decision to volunteer and contribution quality and in fact the most influential of all the design principles that were presented to them. However only 12 out of the 20 or 60 percent of the websites that I inspected fully complied with this principle. So how could task instruction be improved? Well to start with terms abbreviations and interactive elements should be clearly explained we should be avoiding jargon. Task instruction should be concise and easy to follow so as not to overwhelm our users but sufficiently detailed to enable our users to complete the task efficiently and effectively. New users should be able to start contributing within a short space of time and contributors should be able to work independently with confidence. A diverse group of users with varying levels of skill knowledge and available time should be supported by instruction delivered in various formats. For example task instruction might begin with an overview of task workflow using video tours of the task interface, instructive graphics step by step tutorials or demonstrations, incorporating step by step instructions or hover pop-up instructive text into the task interface can also support new contributors. More detailed instruction in the form of written guidelines and help documentation, FAQs, screenshots and examples, knowledge bases or forums these can all support contributors who require additional guidance. So as you can see here the website ancient lives which provides an interactive tutorial with step by step instructions as an example of an effective task instruction. So websites for crowdsourcing cultural heritage can effectively support users by minimising user error and effort to contribute, enabling users to review contributions, clearly identifying tasks, providing task options and simplifying the task. The results of my questionnaire found that the design principles in this category are at least moderately influential on participation and or contribution quality. The results of the website inspection suggest that the websites that I sampled in general are effectively supporting users but I did identify scope for improvement and five cases sustained participation could potentially be improved by minimising the effort to contribute. This was rated by questionnaire respondents as very influential on the decision to continue volunteering. So how could this aspect of design be improved? Well websites should minimise the necessity for users to provide the same information more than once and not demand excessive effort when aspects of the task could be achieved more efficiently by the system. Enabling users to perform tasks effectively and efficiently encourages our new users to continue contributing and it encourages established users to make larger and more frequent contributions. So some examples of minimising effort allowing users to contribute without registering or making registration optional or simplifying that process by incorporating existing accounts for web applications such as Google or Twitter or Facebook. Other examples of minimising effort are prioritising components or modules of the task which allow the users to contribute just to meet minimal requirements if they've only got a short space of time to contribute. Other examples are autosave functionality, automatic completion of data fields that are based on their previous contributions, automatically directing users to the next step of the task and enabling users to save and return to their work in a new session. So I found that some examples of non-compliance were an unintuitive sequence of interaction, confusing or interrupted workflows, an absence of the interactive functionality that users are likely to need and expect such as image magnification, requiring excessive effort to successfully manipulate digitised images or text and in some cases unclear or difficult input formats. The example of Europeana here is I found to be a good example of minimising the effort to contribute. The site allows contributions to meet minimal requirements through the use of mandatory data input fields and then the rest are optional. The site enables users to save draft contributions and then return to them in a future session. Websites for crowdsourcing cultural heritage can effectively engage users by being attractive, acknowledging participation, encouraging users to engage with the collection and conveying a sense of fun. Now the primary purpose of engaging users I found is to support participation based on the results of the questionnaire, the design principles in this category were found to be slightly to moderately influential on the decision to contribute and or sustain participation. The results of the inspection suggest that the website sampled are effectively encouraging users to engage with the collection and that most websites are attractive to users. However, I did uncover a common weakness. Only 60% of the websites that I sampled fully complied with the principle of acknowledging participation and this could potentially be negatively impacting on sustained participation. The sample also rated relatively poorly for conveying a sense of fun with only eight of the 20 websites fully complying with this principle and I did just want to point out that based on the websites inspected the nature of the collection doesn't really have any bearing on compliance with this principle. The websites that I sampled that can effectively convey a sense of fun range from those which focused on art collections, menu collections, ship logs, natural history catalogs and maps to war diaries, photos of the titanic rescue and even personal stories of 9-11. So what does acknowledging participation involve? Well this design principle explains that user participation might encompass registration, completion of steps in the task, completion of the task as a whole, submitting the task in cumulative contributions, feedback on user participation is always positive and it might take the form of text or visual indicators. The idea being that this encourages new visitors to complete the task and encourages established users to continue contributing. Examples of compliance with this principle are thanking the users for the tasks completed and then it gives you the opportunity to invite them to contribute more, acknowledging the user's contribution to the project goal or updating individual progress indicators. In the example here your paintings tagger provides a good example of acknowledging participation. Websites for crowdsourcing cultural heritage can effectively nurture and sustain the user community by conveying a sense of community, supporting community interaction, publicly recognising contributions and supporting content sharing. The results of the questionnaire found that the design principles in this category are slightly to moderately influential on participation and or contribution quality. The results of the website inspection suggested that there's scope for improvement among the website sampled in relation to nurturing and sustaining the user community. Only five of the websites inspected fully complied with all the principles in this category. I found it interesting that despite community interaction being enabled on 15 of the 20 websites only 11 actually successfully conveyed a sense of community and incorporating design features to achieve this could potentially increase participation and contribution quality. Furthermore the 10 out of 20 websites that don't currently publicly recognise contributions could potentially increase participation and contribution quality by doing so and the eight websites that aren't currently supporting content sharing could potentially increase participation by incorporating this functionality. Let's take a closer look at the principle of conveying a sense of community. The project community is comprised of the contributors or users, volunteers and the project team and it might include users of the project output. A visitor's decision to contribute may be positively influenced by the prospect of belonging to a community and by the presence of other people and this also raises expectations of project success. Users who are motivated by being part of a community may submit higher quality contributions due to the sense of commitment and return to contribute more. So some of the methods used to convey a sense of community include emphasising the collaborative nature of the project requiring contributors to register and displaying contributor names or handles, contributor profiles and evidence of community interaction. Some other examples include displaying welcome messages to new contributors, publicly acknowledging new contributors, publicly displaying community announcements such as project news, progress updates, new website features, that kind of thing and linking to related crowdsourcing communities such as you can see in the Zooniverse projects. So a website that does not convey a sense of community to users despite employing techniques that you would think would achieve this is still an example of non-compliance. So this is a very subjective thing when you're evaluating a site. Old Weather successfully conveys a sense of community. Just as an example, the site emphasises the collaborative nature of the project by referring to contributors as crew. So overall the results suggest that if we're to take the sample and to go out on a limb and generalise websites for crowdsourcing cultural heritage are effectively informing and supporting users and now these encompass the most influential aspects of design on website goals. However there are opportunities to support participation and contribution quality in all areas particularly in relation to engaging users and nurturing and sustaining the user community. So thank you very much for listening. I hope that's been helpful again for the full set of design principles. They're available on the site along with some other crowdsourcing resources including a rough and ready website inspection report if you want to take these for a bit of a test drive and again if you've got any feedback on them I'd be really keen to hear from you. Thank you.