 Good afternoon, I'm Alan Wolff and hi, and I'm Jan Cheatham From the University of Wisconsin Madison We're going to be talking about the process that we went through in in establishing in electronic lab notebook service for campus We're relatively early adopters in the eye of Service at an enterprise level for electronic lab notebooks although certainly there are small installations on probably many of your campuses Some of which you may know about some of which you don't we'll talk a little bit about that as part of our strategy The other thing we wanted to start doing as a community is start a community conversation about tools like electronic lab notebooks and other tools that will support the ongoing management of the scholarly record as Scientists social scientists and humanists start working more in a digital environment We think it's going to be very important what we're finding with our graduate students Across disciplines is that many of them want to work in a digitally born environment and from there they get They get frustrated by the fact that they're expected to work in paper Because that's the requirement that their lab has so we're trying to head off some of the people going off in their own directions So I'll begin by talking about what electronic lab notebooks are Of course, they have a lot in common with paper laboratory notebooks Which are centuries old tradition for recording the the discovery process along with the data But there are a few key differences One of them is of course electronic lab notebooks are software So they're not generally a device at all the although they can be used on many different kinds of devices They're collaborative which is I think a big change from paper notebooks because it can it can be really hard to Keep track of what someone is recording if you're the principal investigator You may not get to see the lab notebooks of of the people in your group very often at all and so Electronic lab notebooks have a lot in common with some of the file-sharing platforms and the collaborative tools that we use every day like Google Docs and Box and Dropbox and so forth But like the traditional paper lab notebooks Electronic lab notebooks are a place to record the thinking process The observations the data the conditions under which the data were collected the conclusions Etc. They also can be used as tools for compliance Right now the main standard that most Electronic lab notebook manufacturers are sort of pitching towards is the 21 CFR the code of federal regulations Which I think speaks to the Comparability of electronic records having them count in the same way for legal purposes as paper records There may be other compliance kinds of standards coming in the electronic lab notebook market But lab notebooks in general are considered a tool for protecting IP intellectual property And many many patents defenses have involved producing laboratory notebook records just a very quick history of the idea of and Implementation of electronic lab notebooks, so it kind of looks like back in the 1980s a few people started thinking about them It looked like that it's it was chemists who were thinking about them quite a bit sort of seeing them as a tool for excuse me a tool for Pulling together some of the chemical structure databases that were in use Also thinking about the automation of labs a lot of people envisioning the lab of the future would be one where Instruments would be hooked up to software and people would be recording Their their observations and in the same record and of course that that is happening more and more not so much I think on on higher education Campuses and in higher education labs, but I think evening we're increasingly seeing more sophisticated software And people are collecting some of the output of instruments in in tools like MATLAB and and so forth It kind of looked like some folks at a chemist at Virginia Tech and a Chemist at Procter and Gamble were sort of the original Visionaries there then in the next decade There there were more and more people using them so it looked like In 1993 there was a full day devoted to discussion of electronic lab notebooks at the national meeting of the American Chemical Society a group of the collaborative electronic notebook systems Association spun up in about 1997. They started looking at architectures for sharing information across lab notebooks The 2000s saw a lot of people creating open source La electronic lab notebooks so using some of the existing platforms in the top you see an example of the Pacific Northwest National Lab Notebook looks like that was based on Java that I think had some uptake for a while But it looked like it was discontinued so you can no longer Get support for it. You may be able to get it still Another one was my lab book which came out of University of Houston Drupal based You can still find it on their lab site, but it doesn't look like they're providing much support So a lot of these open source things sort of came and went there are still a few things though like the Open wetware has a tool that you could still use a wiki-based tool and more recently lab trove if you've heard of that one Which is based on? blogging software also available but at the time when people in the Academic section session were working on open source tools There was a lot of uptake in the private sector of commercial tools So that by 2012 a survey that was done of companies Researchers and companies showed that about 40% of the respondents to that survey were using electronic lab notebooks in at their company and Almost that many again were said their company was looking into it at the time. They took the survey Even so there was a lot of consolidation and change in the ELN market at the time a lot of companies buying out other companies and mergers And about that time a little bit later on some universities started to sign site licenses So I think some of the earliest ones were Caltech and Cornell Yale and More recently, I think Ohio State has at least they're almost they're almost at the point of signing a site license And of course University of Wisconsin signed a site license back in July of this past summer Okay So about the time that we started thinking about electronic lab notebooks on our campus, which was about four years ago We had been talking to a number of researchers and hearing from them about how difficult it was to keep records that were sort of half paper half digital so Data is digital and the the traditional notebook is paper and they were they were losing things When you can't keep the data alongside the documentation of the data Things get lost people were losing the data themselves or they were losing the documentation People were using things like Evernote or one note to keep their the expository part of their record To keep that part of it, but they were losing the data We were also hearing that some people were starting to adopt electronic lab notebooks on campus And there was some concern that we would end up with five or more different systems on campus That wouldn't be operable That people were exposing themselves to some risk by signing agreements individually with companies and agreements that didn't protect their IP or the data itself So there was some concern there and our patent office was kind of a big driver in this process So they've always been the authority on campus for good notebook practice And of course that involves keeping a Complete and continuous record of the research process one where there aren't gaps So if if you If you made a change or use you didn't record anything in your notebook You usually made a note of that in the notebook. So it was pretty clear when something was missing Notebooks also were a place to keep track of who did who did what when and a Lot of a lot of people's even sign and witness notebooks too So the idea so provenance is is a really important thing Even though the patent laws changed recently There's still good reasons our patent office said for keeping a good lab notebook There could be disputes that arise you might need to demonstrate that you were using a process before a patent was filed for among other reasons and Keeping a good lab notebook is a pretty good way to protect yourself against any accusations of research misconduct a lot of a lot of the research that you see a lot of the Retractions often go back to people not being able to show that they had kept a complete archive of the data and so forth Around around four years ago our campus established a data Stewardship policy which I'll describe in a little more detail And of course funders were coming up with data stewardship requirements and of course we're all thinking about the the long-term the long-term archiving of research records and PIs themselves and and researchers themselves really wanted to have a single digital record that was searchable That was kind of their bottom line So all of these things kind of added up to our business case for pursuing electronic lab notebooks which we kind of embarked on a Few about four years ago, and I mentioned our campus data policy. It has a few key provisions It's kind of a basic policy in these four areas one of the main things It it says is that the PI is the steward for the research the PI is responsible for setting up a way of organizing the data and Providing access to the appropriate individuals and explaining their access to them The data should be retained for a minimum of seven years sometimes longer and There's their rules about being able to take data with you when you leave if you're if you're the If you're a graduate student who's leaving you may with the PI's Permission take a copy of the data, but the original must stay with the PI on campus If you're a leaving PI then if agreements are signed with the new institution The PI goes to then often the PI takes the data with them and So the reason I brought that out is there were some implications for the electronic lab notebook Implementation that we did on campus We very much wanted to To have this be a system that would align with that policy and that kind of put some constraints on us the Software we like the software. It's great. It's not quite where we want it to be with administrative functionality and We so we have had to kind of work with adopters of the system to ensure that the the PI is Really in the stewardship role and that's meant making sure the PI is the only one with the role of notebook owner owner is basically the highest end level of rights in the software the PI then determines the roles for the other individuals that are in their lab or in their Collaboration group and provides access to them through those roles and sets edit view rights Etc. The PI also has the ability to Determine who can export and share from the system and if they want to implement some kind of signing and witnessing Witnessing isn't doesn't really work in the software very well They're kind of some workarounds to make it happen. The PI can also set up templates And processes for data management The seven-year retention is something That exports can make possible out of the ELN system But that's actually one of the main things that we want to talk about today is the thinking about the archival from the system And how it's kind of a work in process as Alan alluded to We'll talk he'll talk a little bit more about transfer. I think I mentioned also when people leave So of course at one of the other elements that has been driving our adoption of tools like electronic lab notebooks Are some of the funder requirements and obviously that's very important to us at an institution that a Big chunk of our annual budget is based on our research dollars So the requirements that you keep an accurate record of your research are not new All of the funders have said that but they're becoming increasingly Proactive in trying to manage what you choose to do how you manage that record So while they're still trying to allow the disciplines to define what that means The introduction of data management plans first by NIH and then by NSF and then basically everybody else has Really started some new conversations on our campus and many campuses across the country and Then the the the next wave of this with the open access public access policy by our funders is another thing that's driving What kinds of tools can we provide that will help support our researchers? So we start One of the first questions we received once some of our new associate deans had heard about the public access policy was well Will the ELN help us meet the public access policy and the answer is yes for many of the poly for many of the policies it's going to help us maintain more accurate research records and The tools do allow for public sharing now one of the things that we have done because of a limitation in the current product That we're working with is we've turned public sharing off because it's a it's a binary switch we can turn on public access for everybody in the on campus or have it off for everybody on campus and that's a problem in that our Intellectual property people are very concerned about premature sharing of research data by someone accidentally clicking a button The other thing that this is really going to help with and has And not that this is our most important driver that that our administrators are happy about but our administrators are happy about the increased audit ability of Having things in an electronic lab notebook things are time-stamped So it makes it easier to find out who has access to the data who had access to the data and when it was accessed so For the human for the intellectual property our human subjects people are very excited because it means that they They're hoping that they can that human subjects researchers will be able to can store the entire human subjects protocol record including the Consent forms and other things in there in part because we're moving to a system in which they're going to be audit They're going to be auditing researchers to make sure they're in compliance with human subjects policies So having a way that they can do it without interrupting the researchers is something they're They're they're hoping to do Lastly for export control they're interested in being able to Monitor who had access to the data and have very fine-grained controls over who had access but Just as a this is a very small snapshot, but the We've got examples of the product that we're using lab archives as a public sharing platform So we hope to work with the vendor to make it possible that the PI will have very fine-grained Controls to say what can be publicly shared and what is it so that accidents don't happen and we don't have premature release of the data so a Little bit about the the elements of compliance that we've been able to address so far so far. We haven't we're working on having this Available for protection of HIPAA data and our HIPAA compliance officer is is looking at it actively They the company we're working with is a cloud-based vendor But they have signed the appropriate paperwork and we're now working it through our legal channels our human subjects officers Will allow people to use the electronic lab notebook now with with the appropriate Description in their IRB protocol similarly with other health sciences human subjects data and The the one group that we haven't worked with so far our select agents So if you're not select agents are things like highly contagious when influenza virus There are computer codes that aren't allowed for export those things are something that we don't know that will ever allow in the Cloud but something we've negotiated with this particular vendor is the ability to host this on our own campus and put it within our Select agent infrastructure so that for those people who really need who want to use electronic lab notebooks, but can't Use a cloud-based version. We'll be able to run it locally so One of the things that we said we wanted to do is find out from you what's going on and start a community conversation about electronic lab notebooks so Maybe by show of hands does anyone is anyone else looking at like does anyone else have electronic lab notebooks on their campus that? They know and help support What Kentucky Caltech? Oh, yeah, yes. Yes, we know We worked with with your people on the the net-plus service. Oh So so the other lab archive adopters anyone else working with it Mm-hmm, and one of the things that one of the reasons we Moved a little more quickly than we might have in terms of waiting for the product to be right Was because so many of these especially the cloud-based vendors are offering a free individual License so anyone can go sign up for an account and start putting data in and that made Many of our PI's nervous it made our institutional research policy people concerned and That we wanted to head that off by providing an alternative to them How many of your campuses have data stewardship policies like ours Or or maybe you don't know That was something that we had done It was a campus committee of our faculty our research faculty who established this and it basically was to set the The the chain of responsibility for managing research data in in a code that the university supported it doesn't necessarily mean that our researchers fully understand what that means and One of the things we're trying to do is develop tools that align with the our research policy so that we can It's a way of guiding people towards a better practice of managing their data So how many other has anyone else heard on their campus about people using electronic lab notebooks? yeah, and So it's certainly one of the things you'll find too Yeah, so I would look in your chemistry departments chemists have been way out ahead in the use of electronic lab notebooks So that's a place to look for both expertise and for people using it if you're if you're interested in finding out Who might be using it but I think one of the the things that you may want to think about taking back to your institution is a question about Having graduate students going out and Putting their data into these systems without the appropriate agreements and without the PI being able to get at that data That is probably the biggest concern and one of the reasons why we'll talk about the Implementation that we used on our campus to make it harder for them to do that So One of the things that we began with was a set of pilots and actually I think pilots have gone on at a number of Institutions, they're sort of a cautious approach. I think to testing out a tool and I think Prior to these pilots what I had heard when I spoke with other people was that Even some places where they offered electronic lab notebooks For example, I think the chemistry library at Stanford Pretty early on had a license for a Cambridge soft product And invited their researchers to come and use it and they got very little uptake And I think the the main story was that if it you know if it isn't broken Don't fix it. Most people were pretty conservative about changing to an electronic system since paper was working pretty well So we weren't quite sure if we would have a lot of interest on our campus, but we started talking to people We put out a call for participants for a pilot. We got a lot of responses I think we got about a hundred or more people who were interested and we ended up running a pilot with about 65 individuals We asked them to use one of either of these two products ecat or surf as the primary means for recording their research for three months But we also cautioned them to keep a backup copy of it on paper either printing it out from the electronic lab notebook or keeping an analog lab notebook at the same time And they were from pretty pretty broad spectrum of disciplines And so the what we found from that pilot was first of all people hated one of the products But we can we could say that now that product is now defunct. So the the surf products not because of our pilot but Other reasons probably just the turn in the market related Kinds of reasons. They did like the product called ecat though quite a bit The the main conclusion was that almost everyone who participated in the pilot even if they use the surf product Said that they would like to continue using an electronic lab notebook. So that was pretty good reason for us to go forward with a Procurement process We we went through a fairly lengthy one. They take a long time at a state university And at the end of that then we signed a site license with lab archives I thought I'd tell you a little bit about our RFP our request for proposals It had a lot of requirements for functionality Of course user interface is very important with a software tool that researchers are going to use as part of their research workflow We we also looked a lot into things like identity management security compliance data retention disaster recovery So we we had a good group of people looking over the products that we got responses from and and vetting many of these things and Then oh, that's right We wanted to share with you at the bottom there if you'd be interested in looking at our lengthy set of requirements and questions for vendors if The URL for that is there Very shortly after we signed a site license Internet to became interested in doing a service validation actually Cornell was the sponsoring institution and they we did a along with Cornell and and Caltech and Ohio State We did a service validation for a net-plus service and again that involved looking at many of the same things that we had looked at in our RFP so functionality We had a multi-institutional security assessment legal assessment came up with pricing and Also out of that spun up an advisory board. So we're working with the vendor on their roadmap. So net plus Now has a service offering for lab archives That's been around actually I guess since October and Just a few notes about our service implementation So this is a product that is designed to fit with SAML 2 The SAML 2 identity management the shibboleth integration, but because of our La our campus data stewardship policy, which says that the PI needs to be the steward and provide access to individuals According to what what permissions they want to give them We wanted to take it another step further. So we wanted to be able to Have the PI decide who has access and so that led us to a sort of a specialized integration using a tool called grouper that inner interfaces with SAML 2 so that that kind of gave us an additional Step of having to authorize individuals. I know at Cornell and I think at Caltech Pretty much anyone who can log in through your campus identity management can use the tool But we've sort of put a bit of Firewall around it for that And I'll explain a little bit sort of what's what that's meant for us We also spun up a service team with representatives from various units across campus Libraries the patent office the office of the CIO and IT staff and schools and colleges We have our help desk providing very basic support and a few knowledge-based documents So so one of the things of putting this extra firewall firewall around the system has a sort of necessitated is that we Spend some time kind of in high-touch situations meeting with lab groups and helping them decide the roles and rights for individuals and also Setting up accounts for individuals. So we we've kind of referred to this as lab onboarding So labs labs adopt the system rather than individuals and This is meant for us that we we've been going to lab meetings and meeting with entire lab groups Which actually has been great So it gives us an opportunity to talk about the campus data policies talk about data security Help them decide if they have the kind of data that's appropriate to use in in the system Help explain what the roles and rights are in the system how to set up and organize notebooks How to set up rights for sharing and printing and exporting and then this great It's just it's great opportunity to talk about data management that we don't often get with researchers on campus And people ask questions that really relate to data management And so we can talk about things like file naming and and keeping track of different versions and Linking linking out to large files that are too big to be incorporated in the system We can explain to them how nothing gets deleted from an electronic lab notebook That's one of you know one of the key provisions of of a tool like this and get them thinking about their backup and migration strategy out of the tool so it they're a number of Export opportunities and and kind of helping them think about which are the appropriate ones. So obviously Since these lab meetings, we usually schedule them for an hour and sometimes we've stayed for a couple of hours This is not all that scalable as you might imagine. So we're thinking a lot about how we can scale that up One idea and we've done this a little bit is to meet with Departments, so I know we did this I think in our dermatology department, which is a smaller department have a number of PIs and lab groups kind of get onboarded at the same time We've also been working with librarians and IT staff in units on campus. We've had a number of train the trainer events There's a there's a bit of a learning curve for learning the software So anyone who wants to be giving advice on how to use it to lab groups really kind of needs to know how to use the tool themselves And and so there's probably some coordination That that we will need to set up as as we scale things up a bit more The vendor has been great about providing lots of training and are willing to set up all kinds of customized training They'll train individual lab groups So that's been really helpful and we've also found that usually in most labs You'll get someone who's kind of the early adopter who gets all fired up about using the software learns all the you know the nuances of it and Makes it happen for the other members in the lab and that's been really good And we've also had some situations where kind of word-of-mouth has spread through different departments some early adopters who let everyone in the department know that this great system was available and and Why they were using it? Suggesting that the others sign up So just very briefly we so we launched this at the end of September We had a major communication out on campus about it in the first few weeks of Adoption, you know, it looked like we were sort of headed for sort of a logarithmic Curve but then things kind of slowed down the holidays came and we've we sort of put some of our communications on hold We're we're kind of looking at the ongoing funding for this. That's that's kind of a big challenge And so we slowed down on our communications and the adoption has sort of slowed down a bit But we're pretty sure that once we start communicating again that that we'll see an uptick in that again so Once you get labs into a system one of the things that they always ask about or as they become interested in A system like this the first thing that comes up is how do I get out? At least it should be and it should be something that you talk about with your people as you do this So some of the places for thinking about what we are doing Includes how do you archive things? Even if someone wants to maintain their lifelong record in a system most systems can't handle that very well so thinking about how you Have a strategy for archiving at least pieces of the lab notebook either for checkpointing a project so one of the things that some people have talked about in terms of a Data management plan is in their period and their annual reports being able to share results out So the idea that you could export data as an archive at annual periods at the ends of projects Of course it when PIs retire, that's when we want we don't want to just have their materials Sort of disappear We want to have an active project for a process from for bringing their their research scholarly record into our archives when individuals leave labs whether it's a Graduate student or a postdoc with the appropriate permission of their researcher Being able to take a copy of the research that they did while they're there is very often something that they want to do and our people move around a lot so Being able to export a copy of the record so that they can take it with them Is also very important and the nice thing about something like an electronic lab notebook It makes it relatively straightforward because the campus can maintain a copy as it's required to do as the owner of the research data But transfer it also to the new institution. So then you actually have multiple copies so in looking at what what we're looking for in archives is both human and machine readable formats So that we can maintain that continuity of the record so it's someone can go in and look at it But also as Technology becomes better and we can start reprocessing data looking for new things being able to to get access to it What we have now is less than perfect Right now there are two major formats that we can export Easily export from we can export PDF versions of the pages that includes most of the information but not all Or static HTML Representations so that includes a pretty complete Element of the record it also has some gaps and those are things that we're working on the vendor with the nice thing Is they do the company lab archives as a publicly available XML schema For their database and that's where we're hoping to do more work with them To customize the exports that we need to make what we're really looking for is that machine and human readable form That in that includes the complete set of provenance. So as someone makes changes That's important records in a lab in a paper lab notebook when you make a change in your paper lab notebook You line it out in a way that it's still readable and you make your change Lab notebooks are great because you can make those changes and look at a history of those changes But the archive has to reflect that The other thing that we're in the archiving as Jan mentioned the eln our eln was not designed for large files So if you're dealing with files that are a Each person has an individual quota of 100 gigabytes for some people. That's a lifetime's worth of data for some people That's a day or less So the ability to safely link to other to large data sets is going to be really important Something the vendor hasn't quite worked out yet, and it's something that we want to drive them towards as a community But then also how do you keep that record active? And how do you make sure that the file that was in the large in in your archive your large file archive is the same one that You that you was recorded in the eln is going to be important. So how can you check link check some that file? So again managing large Large files the other element of this product even though we're our primary interest is as a tool for research support There's a classroom addition. So And that's more actively well, it's actually very actively used on our campus. We're just it's not something that's part of our group But certainly Cornell is using it quite actively as in the classroom version But that raises a whole nother set of archiving and export needs in terms of students being able to export their data and have Long-term access to the data they they created as students and to maintain that record as Part of the classroom record is another important thing that we need to work with the vendor on The other thing that we want to do is really work with the vendor on integration with other data management tools I'm envisioning lab notebooks as just one of a Collection of tools that will help our researchers with management including integration with instrumentation data processing and Analysis tools so that we can set up data workflows a big thing that Vendors have not addressed yet is migration So when we want to move on from lamb our lab archives to the next best thing how do we get our data out and into a format that's usable and Let's say everyone says these ELN. We're just not ready. We need to wait another five years How do we get our researchers out or a researcher decides? They don't want to be there anymore. What do we do for them? Migration It's very much reminiscent of the learning management system Issue that it's not completely solved yet But is part of the solution or is better in a better place than ELNs is How do you move from one LMS to another? We have strategies for doing that with learning management systems that are some They're painful but can be done. We don't currently have that with learning management systems Right now to our knowledge and if anyone knows different we'd love to find out There are no products that make it easy to import and export in fact one of our runner-up vendors in the in our RFP said Well, what could we have done or what can we do for the next RFP? And I said write an importer for lab archive So that we can move if they make it easy to move our data They're obviously going to be a better choice for us in the future The other thing is very few of them have any capability of exporting a subset or even the whole thing or a subset of data For import into a different instance of the product So if someone does move from one lab archives institution to another The ability to easily migrate that data is all reliant on the vendor and I don't like that and That means access to all kinds of data sharing tools are going to be problematic in the same way So part of it is what institutions will choose to do with that open API and what can we get the vendors to do? Just as an example, we had we had several users of the ecat software Which is now a company called research space, which you may see if you ever do a ELN RFP We had several active users who had More than a year's worth of data in ecat and their solution to migrating data was to export as PDFs and import that into lab Archives the nice thing is lab archives indexes PDFs just fine So they maintain the searchability of their record and it had it put them in an okay position in terms of keeping that That complete record searchable so When we think about the archival formats that are available and this is where we're turning to you as the experts on some of this archival On how we need to move forward on the archival nature of archiving ELN is Right now We're looking at PDFs plus attached files That in some ways that makes sense because for a long time the way that a researcher could Demonstrate the legal aspects of their of their electronic lab notebooks is by printing them out and signing them and maintaining a paper copy But that's probably less than ideal. So some of the standard models for archival formats are Something we need to look at with people who understand them better than us the We should say Jan and I are signed came out of the bench sciences. So we maintain paper lab notebooks. So we're Looking for your expertise and trying to help us figure out what what would be a good way to archive this Perhaps the most interesting thing that we're we're seeing is some work being done by the University of Edinburgh on It's actually a collaboration with the other one of the other vendors that went through our RFP called research space on integration between lab notebooks and their their institutional repository and We're hoping that they have some better solutions for us and then Trying to figure out what is an appropriate metadata standard for archiving electronic lab notebooks is something we need to think through Starting with the standard things you would expect Dublin core There's a European standard called Sarah that is about maintaining the content the Contextual information about the research who was involved what publications were made out of that research What facilities were involved with instruments? But that that doesn't get at the hard part and that's that detailed information and how much does an ELN have to maintain? very specific metadata for a discipline and how much of it can be generalized to a large audience so at this stage We're interested in finding out how you're helping your your people archive the research data now and How would you envision using tools like ELN and what do we have what do we have to do to convince the vendor? So I actually am having dinner with the vendor in a couple of months or a couple weeks and not my whole goal is to pitch fork him into forward with Some of these things that we really need to protect our data for the long term. They of course want to keep us in their product forever But we know that that's not if I've said this to him several times that the best way you can show us that you would that That you're a good collaborator is showing us that we can leave That you have a mechanism for us to get our data out in a format that's useful And that probably is that's the best sign to us that what you're trying to do will help us going forward So at this stage, I'm hoping that you have comments or questions We can talk about the what we're doing with the ELN We can talk a bit more about the product if you're interested. I don't know if our colleagues from Cal Caltech or Cornell know much about the ELN But maybe they have any if they have any input that'd be interesting for us I don't want to keep you longer, but we're happy to answer questions Or if you want to find out more about our lab notebook or other things. We're happy to talk Throughout the rest of the conference Thank you