 I hope you've had a really good day-and-a-half of sessions and of time to catch up with colleagues and to meet new colleagues. I have to say speaking for myself, this seems to have been a particularly good meeting for reconnecting with people that I hadn't had an opportunity to see in a while. I hope that's one of the things you've all been able to do here. Just before we move on to our closing plenary, there are a couple of things that I do want to do. Let me remind you that in your packet there are a number of bits of paper that have dates of things on them. That includes our fall meeting, and I hope to see just about everybody here at our fall meeting in December. That will be in Washington, D.C. There is information on our joint meeting with the JISC, which will be in Edinburgh in early July. I hope at least a few of the folks here will be able to join us for that. There's also information on a couple of other things, including how to get on CNI Announce if for some reason you're not subscribed to it. I would urge you to do so and use that to help keep track of some of the things that we're doing here. I'd like to take a moment to thank a few folks. Notably, I'd like to call for a round of applause for all of the presenters. We had a tremendous number of breakout sessions here, and at least the ones I sampled was able to sample were really super. I think I speak for all of us when I say how much we appreciate people who gave their time and expertise to make those happen and that we'll appreciate them even more if they remember to share their power points or other materials so that we can put them up on the website and those who couldn't get to the sessions can also share in that work. Please join me in a round of applause for our speakers. I'd also like to thank the CNI staff for all of their work in making the meeting run smoothly. There's been awful lot of logistics, and the best kind of logistics are the invisible kind of logistics. I think that mostly these logistics rose to the invisible, and please join me in thanking them for that. And now I get to make an introduction of our closing plenary speaker who will, I think, bring our session to a flamboyant conclusion and send you away with an awful lot of things to think about. I believe at least some of you have had an opportunity to meet Liz Lyon over the year. She's the director of UKOLN based at the University of Bath and has been the author of a number of pretty important reports about eScience and in particular some of the data management issues around eScience which have circulated in the UK and in the US and elsewhere over the last few years. She's been one of the real synthesizers of development in that area as well as running some innovative and pioneering projects herself or being part of the teams that ran them. One of the things that I think it's helpful to know about Liz and I'm not going to give you a long bio of Liz's contributions is that she actually is trained as a research molecular biochemist. So she really has an understanding of what goes on, you know, in laboratories that isn't theoretical. And that's very important I think when we talk about eScience and especially when we talk about the realities of eScience beyond a few kind of marquee projects and really start thinking about how the lives of working scientists in all disciplines are likely to change and the interactions between professional scientists, amateur scientists, librarians, data curators, information technologists and others get rearranged in this brave new world of e-research and e-scholarship. I think that Liz is superbly positioned to give us some wonderful insights into that and please join me in welcoming her. I'm going to talk primarily about this report, the open science report, that I published last November. It was commissioned by the JISC and it looked at a whole range of open science issues. This as I say was last November and it's now April 2010 so things have moved on. There is a set of slides from e-research Australasia here which actually keeps quite closely to the report but the talk that you're going to hear now is a completely new one and builds on the information in the report. So please if you haven't read it, download it, I think it's quite a good bedtime read. So I'm going to look at four areas and I'm going to draw on some experience of some projects that I've been involved with as Cliff has mentioned and I want to start off by thinking about scale and we've heard quite a lot in various presentations about the data deluge and about issues of data scale and I want to look at a particular perspective and the first one that I want to look at is from the perspective of well let's think about a scientist in the lab and in this case it's a chemist. So probably many of you have got chemistry faculty back at your institutions. You'll have chemists working at the coal phase and this chemist happens to be a crystallographer or she is quite a sort of switched on chemist because they're using a mobile device to record their laboratory methodologies and their results so it's like an electronic lab notebook and in the open science report there is a little section on electronic lab notebooks. So the crystallographer in the local lab works, they have to think about the chemical structures of the crystals that they're creating and they want to analyze them, they want to synthesize them and they want to characterize these crystal structures and most of the time that works fine but sometimes they'll have a crystal structure that's particularly difficult, it might be a particularly small crystal structure and so at that point they need to use a different service so they might go to a national crystallographic service and this one happens to be in Southampton. So I come from Bath so let's say there's a researcher at Bath at the local lab and they've got to take their sample to a crystallography service a national service at the University of Southampton and so they've got different hardware there, different equipment bigger-scaled equipment and most of the time they handle the crystal structures it's fine but all of a sudden they may also get a crystal structure that's a little difficult to characterize so they decide to take it to the synchrotron. So this is the synchrotron, the diamond light source in Oxfordshire and the national crystallographic service has got beam time on one of the beam lines there and they have a whole different set of equipment there and the sample goes to Oxford and goes through a pipeline process there. And what we find is that there are different workflows in each different place your lab probably has a different workflow to the workflow at Southampton or at Bath and so what we've got here are far from seamless data integration issues and we've got a new project and I'm starting with some quite sort of nitty-gritty stuff here a new project called I2S2 which is funded by the JISC and we're working with a range of partners to look at these data integration issues across a range of scale and as you've probably gathered from what I've said so far we're looking at data integration across geographical boundaries and also across institutional boundaries and we're looking at some technical metadata issues but we're also looking at cost-benefit issues so we're working with Neil Beigrie who's developed the Keeping Research Data Safe model to look at where, at which points, intervention will realise cost-benefits and I'd like to share with you some early results so we've been involved in the requirements gathering phase trying to understand what's actually happening at these different points and so I'm not going to go through all of this but if you have a quick scan through there's some interesting points here which I'll pick out so the poor old researcher back at the coal phase has to fill in multiple proposals, multiple forms in order to get their sample through these various channels whether it's at the coal phase, whether it's at the national service or whether it's at an international global service like Diamond we have a fairly well-established metadata schema as part of the E-Crystal's national service at Southampton but there's variable standards, different metadata used in other sites as I say, the workflow varies from lab to lab some of it is reasonably formulaic, being crystallography but some of it's quite complex and unrecorded and for my mind, worst of all people take their data samples, their crystal samples to these facilities, they get data off the beam line and there's millions, billions of pounds invested in these services but we take our data off on a USB stick how safe is that? Or on a laptop? So this whole project has raised a lot of issues that we're starting to address with the hope of streamlining and overcoming some of these challenges of scale so that's crystallography so what I'd like to do now is to move field slightly and think about some bio stuff and this time I'm going to go up to the coast I think my geography serves me right to the Broad Institute which is part of MIT and Harvard which is a genome centre there's a very nice display here in the atrium of the Broad where you can actually see the data flying off the sequencing machines and at the moment they're sequencing the genome of the African elephant so they've got 117 sequences, sequencing machines each churning out 96 parallel streams of data let's get the right button this time just to give you, I don't know if you can you see that all right in the back it's coming out a bit strange here but there are streams of data here coming out so you're probably aware that the human genome was first sequence back in 2000 and they were using first generation sequences then and the sort of sequences that they've got at the Broad are Illumina sequences which are second generation and you can probably see from the table up here from nature that the costs are coming down but now they're thinking about third generation sequences and there's a whole range of new companies coming into the market with new technologies to try and sequence faster, quicker and cheaper than before and this is resulting in a technology race to the market and the real goal is to sequence a human genome for under a thousand dollars and some of the companies are promising this within the next few years as well so what does this mean for us who have to think about managing some of this data and providing cyber infrastructure to manage the data and services associated with that well there are a number of requirements for data storage that we need and I've listed some of them up there they're fairly obvious really it has to be cost effective it has to be secure but equally we need to be able to port data to and from different services easily we need to be able to analyze the data so we can get new knowledge from it and ideally what we'd really like to do is move genome sequencing out of these big data centers and maybe you can sit at your local coffee shop with your laptop and do a quick bit of genome sequencing at your leisure so enter the cavalry of cloud services and I've sat in a few sessions that have been talking about cloud computing and there's quite a lot of hype a lot of media coverage for cloud computing services and all the big players are there Amazon, IBM, Microsoft they're all there and they are developing services relatively rapidly to help computing services and data services manage their data not all I should say of the hype and the media coverage is good as you'll see from the bottom right hand corner there and this table I hope you can see it in the back but this table from Nature Biotechnology shows that there is a real ramping up of the companies that are providing cloud storage for bio for pharma companies and there's an increasing range of services being supplied and an increasing range of clients as well using those services so over here on this column we can see all the people that I've just mentioned the blue chip players a range of services being offered and here on the right you can see some of the big farmers and the smaller startup companies that are actually buying into these services so Eli Lilly, Johnson & Johnson, US Department of Energy they're all there and they're actually using these cloud services to manage bio data and interestingly I think this is probably the first if not one of the first published articles that describes sequencing and in fact it's looking for single nucleotide polymorphism in other words DNA variation in a sequence using cloud computing and this came out in late 2009 and you'll see that in fact this has been published by colleagues up the road in Baltimore and it's using Amazon Elastic Cloud services and some open source software called Crossbow so it's actually happening and I mentioned that the first human genome was sequenced back in 2000 so we're now a decade on and you may have seen that nature's just produced a really nice special issue commemorating the first decade post genome and you can see from the nice graphic here that the cost as I say is declining but the number of human genome sequenced is increasing and there are now about 24 published human genomes and over 200 that are unpublished they've been sequenced but they're not published yet so real growth so what does this actually mean well this is Lee Roy Hood and he's the director of the Institute for Systems Biology in Seattle and he's a man with a vision and he's thinking about how the advent of a more widely available genomic data is going to affect the treatment of disease and how we approach health and therapy and he's got a concept of P4 medicine so he thinks that it's going to be predictive personalized preventive and participatory and this is radically different to how medicine is right now so he envisages a world where every patient's genome will be sequenced your genome will be the basis of your medical record and this means that once you have the patient's genome you can start to predict what sort of diseases they may be susceptible to and you can actually treat the patient on an individual basis rather than treating the disease as a whole that's radically different to how we approach medicine right now and for those of you that are in an institution that has a medical faculty or some bio faculty this will impact on how you start to think about information support and research data management support and there's an article just come out in science which is from his lab his colleagues where they've sequenced the genomes of a complete family of four so both parents and boy and the girl to start to identify traits and dispositions to particular disease so this is a real sort of first if you like so thinking about scale which is where we started we need to think about genome scale biology and what that is going to mean for all of us providing information services and data service support to faculty in that area and we have to think about genomic data as a commodity which really radically changes how we think about data in this particular context now this is a slightly different project this is sage bio networks and I've been privileged to be a part of this new initiative this is Stephen friend and he used to work for Merck pharmaceuticals big company and he left Merck to set up sage bio networks which is a not-for-profit enterprise and he's also a man with a vision and his vision is that a whole load of clinical data genetics data genome data will be openly available for sharing between scientists in the sage commons and he's thinking about cancer data obesity data diabetes data all freely available in a way that it's not at the moment for scientists to share to mine to interrogate and to gain new knowledge from it and there's a congress in a week or so's time in san francisco where some of the developments that we've been working on for the sage commons will be shared and very interestingly just this morning in the wall street journal today is wall street journal there's an article written and with an interview with Stephen called my data your data our data and it's all about the sage initiative and some associated initiatives and I just like to read to you the last paragraph so I'll just get somewhere where I can see many of the scientists agree it will be increasingly difficult for their colleagues to resist cultural forces that insist on more sharing my son is four by the time he's 15 his genomic data will probably be on facebook so what what they're saying here is that the way people's leisure lives are conducted is actually affecting scientific research so if patients demand their data be shared scientists almost have no choice so again if if you have faculty in the medical area or in the bio area these sorts of trends will affect you and there are already a whole bunch of people who have shared their personal genomic data and the numbers increasing this is one of the more high profile gentlemen who have shared their data this is archbishop desmond tutu and there are also increasingly a number of personal genomics companies you may have heard of 23 and me decode me where if you spend about 400 dollars uh send it off to 23 and me you get a kit uh you send them a saliva sample and they will analyze your genomic data from that sample and people as customers are sharing their data on the web uh in a very very open way i think it raises some really really interesting questions of ethics of privacy that your ethics committees in your institutions will need to start thinking about if they're not already but of course as we know only too well many researchers do not want to share their data they will not share their data and there are a whole bunch of reasons for that and some of them i i cover in the open science report and some of them are perfectly valid it may be because uh the the data isn't ready to be shared it may be because it's sensitive data whatever but also as this rin report the research information network uh report from last year where they did a a number of case studies from the life sciences um researchers are also extremely reluctant to use other people's data they don't trust each other they won't share their data because they don't trust each other and they won't use other people's data so there's some real interesting cultural issues to overcome there okay on we go so what can we do to um help people to share their data and to make it worth their while and in the open science report there's a section on um rewards on incentive credentials around citation and attribution and i just want to say a few words today about that so in recent uh issues of nature biotechnology uh nature cell biology there have been calls for action around um how to make uh data citable and how to be able to uh attribute uh ownership and uh provenance to that data and and in the sessions that i've been to at this conference i've also heard a lot of discussion about citation of data sets and associated supplementary information and um in class computational biology uh last year there was an attempt to develop a metric that took citation of data sets uh blog postings wiki entries into account as a new metric and they called it the scholar factor a new metric to uh help to attract contributions and participation and data sharing and if we start to think about um citation we can think about levels of granularity and we're we're used to metrics for um uh journals plos talks about article level metrics but in the sessions i've been to we've been hearing about citing data and of course it's hugely complex we have very large databases we have data sets we have data elements in tables so there's a big question for me about what level of granularity do you need to be able to provide citation services do you need to be able to cite an annotation so many of these genome sequences are annotated do you need to be able to cite an annotation and then there's an initiative called the concept web alliance where they're thinking about um how a concept can be described in rdf by a number of triples and uh beren mons and jan velcrock talk about nano publications and nano citations and micro attribution so what level of granularity do we need to have functionality to be able to cite at a level which um delivers the uh services and the attribution that we need and i went to the um presentation about data sites i've heard reference to orcid there are a number of different services that we can start to draw on which are providing persistent identification to enable these services to be delivered but the complexity challenges are considerable let me give you an example i'd like to go back to sage so sage is all about sharing network models and i talked about network biology and steven friend uh envisages the sharing of multiple data sets might be genetic data might be clinical data environmental data which are interrelated to see what the variation and the implications are from one data set to another which affect a patient's uh disease prognosis and these are displayed as network models as visualizations if you like using cytoscape which is a visualization tool and that's a sort of typical um network so how do you cite that this is the work that we're looking at we're just beginning it it is very much preliminary and the first thing we have to do is to understand the workflow of the biologists of the clinicians that are creating this data and understand what their requirements are in the citation process so this work as i say is just beginning we're hoping to um talk to some of the modelers and to some of the scientists in san francisco at the sage congress uh and um hopefully we'll be able to share some of those results with you but i think this conveys the complexity of the citation requirement and the attribution requirement so sage is very much a participatory initiative and what i'd like to do now is to look at some of the different ways in which uh participation for data curation uh is being addressed in the open science report i try to get a handle on what we mean by openness and so far in this talk i've been thinking about access and data sharing which is what you see along the x-axis here but openness can also be about participation and that's what i've tried to um display up the y-axis here so we can think about the lone scholar not always in the humanities sometimes in the humanities and we can think about um volunteers and interested amateurs being participating in some sort of uh science initiative and we can think about citizen scientists the public being involved in science and there are a number of different projects which are presented here is just a selection if you like which we can position on this curve on this continuum depending on how open they are but it's not just about this here's a different graph if we think about um the amount of work all the functions all the services all the effort all the tasks that we need to do in order to manage the research data that's coming out of all these new experimentation procedures i've listed a few on the right hand side there we need services for annotation audit data cleansing metadata schema ontologies and so on there's a whole list of stuff that needs to be done who's going to do all that so down here on the x-axis we've got some notion of capabilities so the scientist the professional scientist is the subject expert they know their data inside out but they haven't got any time so they've got the capability but they don't have any capacity they don't have the time to do what's needed and over here we've got people out there the public galaxy zoo great example who've got real interest real enthusiasm and they have the time but they don't always have the skills so we've got this dilemma which i i've tried to characterize on this decaying exponential graph of how do we balance the requirements of capacity and capability to help us to curate and manage the data that's that's being thrown at us today so as i say let's look at some examples some possible approaches so this is wiki pathways so this is a wiki based initiative where biologists are sharing their metabolic pathways so it's a absolutely traditional wiki you can edit it you can comment on it you can add to it and it's growing there are i think about a thousand pathways metabolic pathways being shared and about a thousand users that's pretty much a handful but it's a start let's look at another example so we go back to crystallography here and this is chem spider which is a service from the royal society of chemistry which manages chemical structures and they're very much into bringing the community the crystallography community the chemists to help to curate their own data so they have a manual to help people a guide and you can log on as a beta tester a curator or a depositor and as such you can help to curate the data so you can edit it you can add to it you can tidy it up and they have a notion of master curators who have oversight so they check all the work that's done by the curators so this is quite a nice example i think of community curation could it be ported to other disciplines is it being addressed in other disciplines in this way i think it's a nice example and you know if you've got a iphone you can do it on the move there's an app that allows you to do it on the moon so you can do it sitting on the bus and also quite innovatively i think they're using gaming approaches to attract people to participate so they've developed this spectral game which allows you to if you like take part in a sort of semi competitive way and they score they've got top ranking scores and it's a way of attracting people into participating so just moving on if you remember the open continuum at the top was citizens citizen science and in the open science report i there's a whole section on citizen science and one of the aspects that i particularly liked was a comparison of citizen journalism and citizen science and what can scientists learn from citizen journalism and since writing the report there's a nice blog post from this guy martin boland and he's a journalist but he's also a historian so he's citizen historian and he's got this notion of the curation gap so he characterized professional journalists and then he characterized citizen journalists and i liked this and i thought well we could apply this to scientists so you can characterize professional scientists in a very similar way and you can think about enthusiastic amateurs if you like the citizen scientists a whole relatively untapped resource so let's look at some examples and see what the issues might be so this is a project which is being led by intel in berkeley in california and they're asking the question what if every mobile device had an air quality sensor okay so just picture this you've got your mobile phone and it's got an air quality sensor in it so anywhere you go you can take a reading and they're developing that hardware there you are you can see the very preliminary i think hardware there but their vision is quite exciting so they envisage that you'll be able to use your mobile device and send in readings from wherever you happen to be it'll go into a server they'll synthesize the data and they'll send you out text messages to tell you where the difficult areas are in terms of air quality in the particular city that you happen to be in and then they also envisage having signs up in our urban areas telling you which way to walk so you go in the healthy areas okay and they're calling this participatory urbanism i think this is quite interesting it it reminded me of google street view in in terms of privacy issues you know if you're if you happen to suffer from asthma and and you have the the data sent to you and so on some interesting privacy issues there but of course one person's view on privacy is not necessarily the next person's view on privacy so there's some interesting areas there to explore i think as well for our ethics committees who are going to be very busy in the future i feel so here's another example slightly different example of citizen science so the bbc has been quite active in this area with an initiative called lab uk and the bbc has been looking for particular topics scientific challenges that are suited to mass participation and it's got some quite high profile scientists as you can see there's robert winston uh promoting this approach and it's starting to have some real success and one of the things i like about lab uk is it's working with professional scientists to develop these initiatives and it publishes a list of criteria so that if you've got an idea as a scientist you can decide whether it's worth submitting it to see whether they'll take it forward but better still they assure that they will publish all the results from these citizen science experiments in peer-reviewed journals and indeed they are doing that and and david attenborough you can see him on the web promoting the results so the results of sex idea have been published uh in a peer-reviewed journal so my question really is how will this go down with your faculty what are their attitudes going to be to working with the public in this way i think there's going to be some interesting um discussions and uh deliberations to see whether this is something that your institutions will want to take part in so that takes us on to the the final section that i want to talk about and to think about how our institutions are responding to these changes that are coming up and there is a section in the report which starts to look at some of these issues and this is uh the university of edinburgh's informatics building uh so this is one way of responding uh to the growth in informatics i don't we're all lucky enough to be able to have a beautiful new building like that in the open science report i i developed a very very simple checklist as i called it so uh an open science institutional readiness checklist so it's 10 very quick questions that you composed to your senior management team to see whether they're on the ball and if you have a look it covers strategic planning it covers structures policies and it covers library and information services as well and how engaged they are with data informatics and there's a whole section on data informatics in the report and in this talk i've covered uh some of those areas and i want to draw on some specific examples in this last section to show you some uh ways in which some institutions are tackling some of these challenges so one way uh that some institutions are tackling high throughput biology is to concentrate their initiatives and their efforts in some new centers some new institutes for high throughput biology and so UCLA has a center university of british columbia has a center and johns hopkins has a center just up the road for high throughput biology so this is thinking about gene sequencing microarray experiments and concentrating that effort we heard some talks about cloud computing and i've said a few words about it well this is quite a nice initiative uh so this is from the triangle universities north carolina who three of them are collaborating to um develop a new cyber infrastructure for data which is based on cloud computing so they're going to share their cloud services and develop policies and practice collaboratively for the benefit of all three campuses so it's a kind of regional solution if you like i liked that another thing that some institutions are doing is completely restructuring their information services so that this is uh Queensland qut in australia uh did a presentation last year at e-research australia presenting this new structure and if you have a look they're bringing together people like visualizers high performance computing people as well as data librarians liaison librarians into an e-research support structure so again this is something that your directors of information services or you yourselves may want to consider doing now in the title of the talk was the word constellations and you're probably thinking well she hasn't mentioned constellations yet so here we are so at rpi in new york they have the notion of research constellations which i rather like so the idea is that they're bringing together the stars in different disciplines together to work in particular areas so areas like bio computation and bioinformatics tetherless world and integrative systems biology so multidisciplinary groups team science but what i would like to suggest is um that doesn't quite go far enough because given the what we've just been seeing what about graphic designers to help us deal with these these increasingly uh complex visualizations animators to help us use gaming technology to attract punters in to help us curate our data i've listed a few ethical and socio-cultural issues we need social scientists in the team to help us with some of those cultural differences and i know that there are going to be some ipr issues associated with all of this so we need some legal expertise as well university of michigan so they've got an nsf bit of funding to set up an open data program and they've got an open data faculty and if you have a look it's drawing on people from again different disciplines and they've actually got an open data faculty so this is again thinking about how their institutions are supporting open science and this article that i referenced earlier from genome biology which is produced from colleagues here locally if you have a look at where they're from and i'll have to step closer so the first author is from the department of biostatistics at johns hawkins second author is from the centre for bioinformatics and the third author is from the i school at the university of marieland that's a really nice combination so one of the last points that i want to make is about the changing role of library and information scientists and in the uk we have an i school now the university of sheffield it's the first uk i school and they run courses in chemo informatics to help the researchers and the new graduate trainees to understand the issues in chemistry and then illinois their i school has a course which specializes in data procuration so these are all initiatives to help to develop either faculty or to develop the up and coming information scientists and librarians so they have the skills when they come into our libraries and into our institutions they have the skills and the knowledge and the know how to help the researchers to manage their data so a few take homes i'm almost at the end we need to think of some fairly pragmatic solutions to help the researchers to change their ways and they have to be embedded in the workflow to help them to share their data so that this prospect of open science can really happen and i think critical to that is to be able to demonstrate attribution for a particular data sequence for a particular data set and we have to work with the funding organizations and the research assessors so that this is built into research assessment frameworks it's not in the uk research assessment at the moment and i don't know that it is in the us either i do strongly believe we need the crowd we need people to help us we need effort we need capacity and we need to provide tools interfaces and guidance and assistance so that we can get the community participating and get citizens out there helping in the way that galaxy zoo has been so effective you all come from a range of different institutions different characteristics i hope you'll take back to your institutions some ideas to talk to your colleagues and your peers about how you might address supporting open science because it is going to change the prospects i think are very exciting they're slightly frightening as well but they are transformative and my last slide is a completely shameless plug for the international digital curation conference which will happen this year in chicago in december and i very much hope that i'll see many of you at that conference so thank you for listening thank you so i don't know whether anyone has questions i'm happy to answer a few if you have some i have a question about the citizen science angle that you talked about because citizen science you promoted that as a concept that we need to leverage and you hear about that a lot these days i know the nsf is very interested in it as an approach for data curation but the scientists that i talked to are incredibly skeptical about that and there are a couple of good examples that get mentioned every single time you know identifying star systems and things like that but they really don't think that it's going to work for the particular type of science that they do so you is it really worth pursuing that angle if the scientists themselves won't accept the contributions of amateur scientists because they're not really you know vetted what do you think well i think we need to do some work to understand what's involved in citizen science successful citizen science and think about which disciplines it works for and which disciplines it perhaps isn't right recognizing that there will be some disciplines sub-disciplines where the complexity of the of the laboratory just doesn't lend itself to someone out there with a web browser contributing but but equally i think there are other areas where there's real potential so the open science report really is throwing this idea open and seeking comment and views from both the information science community but also from the research community from the funders and and i know certainly from some of the uk research councils public engagement is is high on their list of drivers to show impact and i don't know if that's also true of u of us funders but what better way to demonstrate impact with the public than to get them involved in your research and and you know school kids get them involved right from the start so they understand the process they understand the skills that they're going they could usefully have and it becomes second nature to them that they they contribute they manage data i i think it's quite an exciting possibility but it won't suit everybody and as with many new things we we need to understand the perspective of the professional scientist who's well established and and see where they're coming from as well i think there's a big discussion to be had in this area and and i hope it will begin now okay i think i'll hand over to cliff thank you thank you very much for that wide-ranging talk i hope it's left everyone with a good deal to think about i'll just mention a couple of quick things one the pointer to the report that liz was referring to is on our website and is also in the in the meeting material that you got and i do urge you to have a look at it we will put a pointer up to these slides on the c and i website and as you probably noticed they're dense with material and urls and should be a helpful reference and i do also want to point you once again to this international digital curation conference that's happening in chicago it's the week before the c and i meeting and c and i is indeed acting as once again as we have with all of these as a co-sponsor so it's an excellent opportunity for digging further into some of these issues please join me once again in thanking liz for making the trip over and giving us all of this to think about and with that we are adjourned i wish you safe travels perhaps some of you will have a few moments to enjoy baltimore before you go and i look forward to seeing you all in the coming months and certainly in december thank you for coming