 Hello. We're going to get started in just a minute here. If you could grab your seats for the first session. Thank you all for coming. Welcome to our first Carnegie Mellon Open Science Symposium. We have the organizers, myself, Anna, Melanie, Jojen, and Eric. We're really excited to have you here today. I want to make just a couple of logistics announcements. The first is that the Wi-Fi code printed on the back of your program for those of you outside of CNU is incorrect. Please. Despite my best efforts. And now, this is secret two. Sure. Well, I'd say I have it here. R-D-Y-T-Z-2-3-V. It's coming back. Alright. There we are. Take a picture of that Wi-Fi code. There should be edge-a-roam. Oh, we're having a lot of technical difficulties. Like that. Okay. That was announcement one. Wi-Fi code. Come on. Maybe it's on the table. The location of the restrooms, which are not immediately nearby. However, there are restrooms on both this floor, the fourth floor, as well as on the third floor. You exit the library the way you came in. There are directions to both of them. You can take stairs or elevator down to three as well. Ask us at the break if you have trouble. And without further ado, I'd like to introduce our introductory speaker, Rebecca Dirge, our Dean of Mellon College of Science. Good morning. So I'm just excited to be here. So I've had my car breakdown. I called AAA. I've had a tow truck. I've had the car towed. It's at the garage. And let's all just hope that I do not need a new car. That was the instructions to the mechanic. I don't want a new car. So welcome to the first open science symposium at Carnegie Mellon. This is the future of Mellon College of Science as far as I'm concerned. So I'm super excited that we are having this symposium. And I'm super excited to be opening this conversation with a lot of the right people, let's just say. So about a year ago, the Mellon College of Science received a gift from the David Skafe Foundation. And it's a charitable foundation and it's a local group. And they support foundational science. And they basically said, Rebecca, here's a bunch of money. Do some exciting things with it. And one of the exciting things that we did was to fund conferences like this. So we're not really, we're interested in very forward thinking conferences that can bring the right people together and start the right conversations. So we're very, very grateful to the Skafe Foundation for helping us reimagine science in the future. And so throughout the course of the next day and a half, I'd really like to have, I'm going to be around my offices just across the lobby. I'd really like to have conversations about the future of science and your thoughts about open science and data sharing and how we move forward through this into the next 25 years. So at Carnegie Mellon, we believe that interdisciplinary science is a secret science, a secret sauce to science. In my opinion, what I refer to as single sciences lab work is gone. We can't solve world problems and have an impact on our own. And so when I think about the future of science, I think about hiring new, young, not necessarily young, but assistant professors, that was my hesitation using as a young assistant professors who no longer have their own lab. So the days of sort of opening a door and saying, here's your lab now, wait a year and a half for the renovation. Those are gone. That we'll be using centralized labs run by professionals who help us get the best possible data. And then what we do with that data forward and how we share that data is one of the topics that I know that you're invited to speak about today. So today and tomorrow, you'll have a chance to participate in workshops that cover pipeline development, data sharing, reproducibility of data publication results and things like that. I should probably have a disclaimer. I'm the statistician by training. So when I hear reproducibility, you know what the light goes off and when I hear sample size, another little light goes off. I'm trained as a mathematician, turned statistician, started in human genetics probably before all of you were born and then switched to agricultural genomics. So my research group chases technology and figures out ways to analyze the data because no one ever worries about experimental design or how to analyze the data. These companies just make really cool technology that generates more and more data and we have to figure out what to do with it, how to share it, how to share the results, how to combine the results and things like that. So tonight I understand that there will be speed dating. Where is there speed dating? Speed dating, which I'm actually, I think that this is really how to do things and I encourage you all to participate in it and the students in the room. You have to learn how to message your science in three minutes or less so that people can understand it. So really if you're not participating, which I actually encourage the students and the postdocs and the young people to participate, if you're not participating, just really pay attention to how important this is because you usually only get three minutes of attention. You guys, the one thing you're going to remember from today for me, my car broke down. That was the first 30 seconds. Everything else you're just, you're already off the model and it's the human brain. So let's see. So before we start, I need to thank lots of people. So the David Skafe chair of the foundation, I actually took a picture of this and I know the guy who runs the foundation and I texted these pictures of these conferences and say thank you, you're changing science and how it's going forward. I also want to thank Fickshare, CodeOcean, CMLH, protocols.io, BenchSci and of course the Carnegie Mellon University Libraries. We have Keith Webster here who is the Dean of Libraries. We're very grateful for Keith's forward-thinking ideas about libraries. He is a computer scientist by training and he's the only librarian I've met who doesn't really like books. We spend a lot of time together and I encourage you all to meet Keith because he's not your average librarian. I also want to thank our conference organizers. Ana was here before. Ana from the CMU Libraries. Eric Itry from Biological Sciences who has been here less than two years but has had a significant impact. I want to thank you all, your young people, you're leading the future of science at Carnegie Mellon. Everyone else, please network, make friends, if you need anything, yell at one of the organizers or me. I'm around all day. I'm going to try really hard to be here tonight so I'm crashing the conference because I'm super interested. Let's turn this into a discussion where we can all meet each other and learn about these new things because this is your future for sure. So enjoy yourself, have fun. I'm the core psychology and brain sciences as well as the program director for Open Science. I'm really excited to present the first four guest speakers that we're going to have this morning on our panel titled Open Science in Research. So the way the sessions are going to work today is that each session will have three or four speakers. They'll all give short talks. After their talks we can take a few minutes for questions, especially any clarification questions that you might have so you don't forget them before we move on to the next talk. Our sessions should have enough room to accommodate that. And then after all of the speaker's talks we'll have them all come up to these bar stools. We'll pretend to have a pub in the morning here. And then we'll have some time for a Q&A and I can help moderate that but we also really want to hear from you guys. So be thinking about what questions you might want all of these people to take a stab at and we'll let them chat it out. So first I'd like to present our first speaker who is Josh Siegel who is a scientist at the Allen Institute in Seattle. Josh. The employee of the Allen Institute which is a institute that's very dedicated to many different approaches to open science. There are a number of different things I could have talked about but today I'm going to talk about what I think is the importance of open source tools for doing science. So when I was a graduate student at MIT I helped develop an open source platform for extracellular electrophysiology. So these are experiments in which you are inserting dozens or even hundreds of electrodes into brains and you need to extract the signals from those electrodes and filter them, digitize them and send them to a computer for processing visualization. And when I was a graduate student there were not very many options for open source versions of this type of experimental hardware and the things that did exist were not well documented and not easy to reproduce. And so I worked along with another graduate student, Jacob Vogt, on creating a system that was first of all suitable for the types of experiments that I wanted to do but also had the potential to share it with many other people. So from the beginning we did all of our development out in the open, sharing all of our code and designs and documentation through github. So all of the tools you see here, the designs are freely available and doing this out in the open and presenting at conferences and stuff helped us find collaborators and rally people around the cause. There was a lot of enthusiasm for this at this point in time because a lot of people felt that the existing commercial hardware for doing these types of experiments was just too expensive for the features you were getting. But more importantly, it did have the flexibility to do the types of groundbreaking experiments that scientists really wanted to do. So Open EFIS caught on, these tools are now being used at over 140 institutions in 30 different countries. I think it appeals both to scientists who really want the ability to hack their own hardware and do things that just no one has thought of before and are only possible if you have access to the low level details of the hardware and software that you're working with. But it also appeals to scientists who may not have the funds to afford some of the more expensive commercial hardware. These are either scientists in labs that don't specialize in electrophysiology, so they don't have funding set aside for buying these large commercial systems, or there are labs in places that don't have that much funding for science in general, and this is the only way that they can do these types of experiments. Another impressive thing I think is that our software for data acquisition has been forked almost 350 times on GitHub, so that means people are copying the source code with the intent of making their own modifications and then hopefully sharing those modifications back with the rest of the community. And so it's really become a nice community development effort. A lot of the support questions that come in are just handled by people around the world, not necessarily handled by the people who originally developed the hardware and software, which is really cool. So I think the three main reasons why we put in the effort to develop these open tools were affordability, flexibility, and reproducibility. And I want to touch on reproducibility because I think that is hopefully something that will be brought up a lot during this symposium. So during the course of a scientific experiment, in this case recording electrical signals from the brain to the mouse, you typically want to treat your data acquisition hardware as sort of agnostic to the system under investigation. You kind of assume that you have this experimental subject that you want to work data from, but it shouldn't matter what specific type of computer you're using or what data format you're using. What really matters is the results that you get and how you make those public through your publications. But there are other types of experiments where the experimental hardware becomes part of the system under investigation and these are closed loop experiments. These were the type of experiments that we really wanted to do and which were difficult to implement with a lot of the commercial hardware just because they require so much flexibility. And in this case, whatever algorithm to use, whatever hardware you use to actually stimulate the system, so you take some measurement from the brain and then use that to trigger feedback to the brain, which is a really powerful way to study the system because you can do types of interventions that would be impossible otherwise. But here, if anyone wants to reproduce what you've done, they need to use the exact same, not necessarily the exact same hardware, but they have to use the same algorithms and very similar setup because everything is so tightly integrated. And so with Open EFIS, we were able to create plugins for the software that did all this closed loop processing and those were super easy to share with anyone else who used the system. And I think as closed loop experiments become more commonplace in neuroscience, which I hope they will be because they're really powerful, we need common interfaces to be able to share the algorithms that we use for these types of manipulations and having an open source framework for that makes it a lot easier. So from the beginning, the Open EFIS software was designed to be as modular as possible, so people could add their own data sources or filters or goes through processing algorithms or visualizations and then once those were built once by one person, they could be shared with the rest of the community and so we could reduce the need for redundant development efforts. A specific plugin that I worked on when I was in graduate school was one that could detect the data oscillation into the campus so this is a ring with a rhythm that's associated with learning and memory and trigger stimulation at specific phases of data because there were hypotheses that you have different phases of this rhythm that are involved in encoding and different phases that are involved in retrieval and without the ability to stimulate the hippocampus at specific phases of data we never could have addressed these questions. And so we were able to show that we could improve mouse performance on a working memory task just by stimulating the hippocampus at specific phases of data and now other people are using the plugin I developed to do similar experiments around the world. Our current focus at the Allen Institute in terms of using Open EFIS is to adapt it or use with a new type of experimental hardware called no-pixel probes so these are high density silicon probes with almost a thousand channels on them and you stick five or six of them in the brain at a time and so it's this huge influx of data that we have to deal with in real time in order to be able to visualize it in order to know that our electrodes are in the right place I mean to save it on to disk so we can analyze it offline and so this has been a big challenge to be able to handle all this data in real time but Open EFIS has been great for this and I'll do experiments like this one where you see spikes from almost a thousand neurons recorded simultaneously across different regions of mouse cortex so with the current hardware we can interface actually up to 16 no-pixel probes with one computer and then we've developed new visualizations that allow you to see the activity across all the different layers that you're recording from this particular recording had electrodes in cortex to the canvas and colonis and these types of experiments are going to allow us to answer questions about interactions between different parts of the brain that would have been really hard to interact with before so before I wrap up I just want to talk about some of the lessons that we've learned in the course of building this community around open source tools Open EFIS has been around for around eight years and we've grown from just a single lab to over 140 labs around the world and so I think some of the things that are worth mentioning is that first of all open source on its own is not enough to facilitate community development you really need to think carefully about the ways in which people will be able to modify your tools and keep them nice interfaces to add their own functionality it's super important to share anything that you do for science but if you really want to make sure that you can build a community around it and people will contribute back it needs to be really obvious how they're supposed to modify the things that you built and so along those lines I think having well-designed open interfaces is more important than being fully open source I think there's some cases in which it is important to have those source tools in order to be able to protect intellectual contributions so some examples of these are the Intan ships these are these tiny integrated circuits that we use for data acquisition in open EFIS and the chips themselves are closed source but they have really nice open interfaces for getting data off of them that allowed us to build this open source ecosystem around them and the neurobosses probes which I just talked about are themselves closed source but they have really nice interfaces for getting data off of them and so I think getting neuroscientists to agree on opening interface standards is really difficult but I think it's really important for moving the field forward to be able to have these shared interfaces both on the hardware side and on the software side and finally I want to say that maintaining features is even more valuable than building them it's really easy to say oh I'm going to add this cool feature on the weekend but if you're not really dedicated to making sure that it stays up to date when other parts of the software change or it breaks down in some condition that you hadn't anticipated before I think it's very easy to fall by the wayside so I think making sure that you have people that are dedicated to maintaining all the features that you add to your open source product is really, really crucial so I just want to thank some of the people who contributed to this Jacob Votes wouldn't have been possible without him we have a support person in Spain, Aaron, who has been doing great work opening interface board directors we have people in Lisbon who are handling all of our hardware distribution which has been really great this fraction of the people who have contributed go back to the opening interface software and help make sure that the community continues to grow can you repeat the question? so they're wondering about the... this was, I think, a useful part of the opening of this GUI I knew that the low-cost hardware for many people would be the most appealing aspect of this but also that hardware becomes obsolete quickly and we're a period in which there's rapid technical advancement and so I wanted to make sure that we had software that was agnostic to where the data was coming from and so this was actually built on top of a library for processing audio data because audio is very similar to electrophysiology data in terms of its sample rate and bit depth and a lot of features of the audio processing library made it really ideal for processing neural data and so without that this would have been really difficult to implement but because we had this audio processing infrastructure already it was pretty simple to build this on top of that sure so what prime example is we participated in Google Summer Code two summers ago which is a great opportunity for open source projects to get interns for the summer which are funded by Google both of the students that we mentored made really great progress during the summer and then never quite finished what they were working on and then once the internship finished they left and became more or less unreachable and so we've had to very slowly like try to pick up the pieces and fix what they've done but the features that they implemented never became fully functional so really just like having people that are dedicated to making sure that the features remain functional after they've been built is really important. Thanks a lot. So I'd like to introduce our second speaker for this session, Von Cooper, who's an associate professor of microbiology and molecular genetics at the University of Pittsburgh. I think some really thrilled for a chance to share my passion for pre-printing in the life sciences with you so I'm just across the sort of across the street you'll see in the school medicine at Pitt and I'm here as an ambassador for a movement or a foundation, a program called ASAP Bio that is directed by Jessica Polka who I believe was originally invited to come here and she graciously said you have an ambassador across the street why don't you give him a chance to sort of shop our wares and that's what I'm here to do. I also co-founded and direct a new center launching tomorrow for evolutionary biology and medicine. These are two very different cultures, one being more early embracing of open science the other not and I stand at the juncture. So just a quick introduction, I came here three years ago after spending 12 years on the faculty at the University of New Hampshire which is a relatively underfunded state institution our libraries were certainly under resourced you know no taxes in the state means no very few books and so we were really keen for access to journals and we were as a teacher who was often prescribing you must read these articles go find them we were always running into paywalls that were slowing our progress and so I hold that kind of angst near and dear even now that I'm here at this really wealthy institution and while resource institution school medicine at Pitt I'm actually down by the river on Second Avenue in one of those biotech buildings in case you want to come visit. So anyway these are two different research environments in terms of their mission and their resources one is really a teacher scholar model and the other is we want to advance academic medicine across the board. And so I think along with our angst it became really clear as a community that the traditional way of sharing science is really slow and so the need to get access to journals and the need to learn about resource or learn about findings sooner are sort of one in the same that is you know when you are not at a wealthy place or simply if you're time limited you can't go around and hear talks at conferences all around the world and so they come to you late often a year past the paywall gets dropped if at all and so you know there's a very long process where the best work goes through and even not the best work goes through peer review and cycles revision and rejection and so on and only after some period of time does community feedback and ideas and data sharing come around. So the notion that we should post our data our results ahead of publication is not new archive has been doing it in physics and mathematics and other sciences for more than 20 years but it is relatively new for the life sciences and so bio archive is our primary resource but it is not the only one. So preprints are where you post your results ahead of well at some point some people post it right when they submit first publication others like us submit it when we're getting ready to submit for publication and that's been incredibly valuable because we've been getting feedback from authors reviewers potential reviewers and even engaging them say hey thanks for your comments would you be willing to be a serve as a reviewer if I mentioned this to the editor when we submit it goes to a actually a reasonably rigorous screening before it gets posted not everything makes it as a preprint and then it's open and then combine the social media your research is immediately available and so this process of community discussion and feedback and idea sharing can be immediate and I'm really active on Twitter many people who use preprints are also active on Twitter and we see these dialogues happening in real time and they've been shaping the way we've been doing our science so just for the record preprints are permanent they are versioned they are citable they are open to the public and free and so they meet some really key demands that we had had coming out of this as I say sort of this under-resourced environment and again you're not at one many of you are not but think about who you want to read your research and how quickly you want them to have access to it so as I mentioned I'm an ASAP bio ambassador and I basically became an ambassador by saying hey I really like preprints I like your mission what can I do to help them and they've been very generous in sharing the materials in fact most of these slides come from an open drive that I get from the organization so it's biologists driven non-profit organization working to make life science communication faster and more transparent it is not exclusively about bioarchive although that association between that place for preprint that sort of site for preprinting and ASAP bio is fairly common so I just want to go through some concerns about preprinting because they are many really they are three but they are big and the first is I'm afraid that by posting my research ahead of being formally published I'm going to get scooped and as the founder of archive has said and has many fields have adopted that can't happen because archive provides a time stamp or date stamp sort of statement of priority and it is up to you the scientific community to review whether that statement is worthy of being accepted as a major funding the second is that it's against journal policy and the great news is this is happening this is changing really quickly so there's an enormous list of big journals this is just a tiny subset of sort of name brand journals that accept articles that already are appearing on a preprint server and that list is easily accessible at Wikipedia and at the Sherpa site which talks about publication policy and then the final thing is that oh well maybe there's just too much information how do I know whether my preprint is worth reading I'm going to see it later in a journal I'm going to see it in my society journal but I mean let's be honest we typically see about see the major findings in our research at massive post assessments anyway or even sessions like this you're getting new findings and research here in advance of print so we're sharing our work before peer review is common practice in fact it's sort of the first phase of peer review isn't it not so preprint use is really increasing greatly in the life sciences I will say not with my new collaborators here at the School of Medicine in Pittsburgh so we have a long way to go so I will say you take a look at all the papers that my lab has been affiliated with over the last three years only about a quarter of them have actually been preprinting because frankly my colleagues in infectious diseases don't know anything about it and those are big barriers to overcome so even though I've said we can get through this it's not a big deal there's a big cultural shift ahead these numbers help and I think there's sort of a crowd mentality that started to percolate particularly among called the under 45 generation I'm just past that so just to give you a sense these numbers really are exploding and so most of that growth in life sciences in that green group bio archive another great motive for preprinting which we see a lot again in these new faculty who are trying to hustle and get their first grants is that funders are encouraging us to preprint our work as a record of progress so it's not cooper at all in prep it's cooper at all bio archive here's your citation go take a look so what are some benefits you can gain visibility combined with just simply distribution from the servers all often have twitter handles that are specific to disciplines some of them have blogs you can move your data around sort of advocate for your science yourself you can get feedback to improve your paper that's happened to us many times often starting with a direct message on twitter hey I read that I think this is really cool I've got a problem here let me tell you about it and I thought about showing trying to dig up those direct messages but I don't want to bore you you can find and often there are journal editors that are now scouring the archive servers looking for good material for their journals and some of the major journals that are listed on there have a person dedicated to that role that's really pretty exciting you can find collaborators earlier I know we haven't quite gotten to this point from my lab that's why it's not in red all these others have helped us some people have found new colleagues and new resources through preprinting you create this record you demonstrate productivity for your jobs and grant proposals you can accelerate your discovery and I think last and probably most important it's free to the taxpayer so a lot of our work is taxpayer funded and this gets that result out right away another new and I think pretty exciting set of movements that really are still developing are these commenting venues that are sort of organized ways organized communities that take a look at preprints provide almost journal club level dialogue about those preprints and then can actually submit those reviews along with the preprint to a journal so I'm one of the associate editors of the journal evolution one of the biggest journals in my field evolutionary biology and we have a policy of accepting these packaged sort of preprints plus peer reviews from PCI evolutionary biology it tends to streamline the submission it's actually helping us as a journal to reduce the time from between submission and acceptance in some cases it's also just turning us allowing us to provide immediate editorial feedback as an associate editor I can say this is not going to make it at this particular bar but I'm going to refer you to a second every journal and so if you're interested more there are great ten simple rules and this is in cost computational biology about preprinting which is outstanding so I was just mentioning about high visibility here's an example of bio archive the twitter channel for evolutionary biology for microbiology note it's got a decent number of followers and there are folks journalists who are watching these channels and here's a really great recent example of a story that appeared in the New York Times citing the bio archive article in advance about actually coming out in science down the road about how how sort of the physiology and biochemistry of a beetle is providing broad insight into evolutionary biology so I just want to give you a sense though that all is not a panacea this is a real life and I hope you can read this in the back debate in my lab this is a slap message thank you between two of my really terrific postdocs who are both on the job market right now so I wrote hi Chris and Caroline so now that the article is submitted this is Caroline's major article from her postdoc how do we feel about bio archive you know I'm in favor but I won't move forward unless we're unanimous my policy Chris I'm in favor Caroline what do you see as the advantage of putting on a bio archive thus far I've put all my papers on bio archive and in my experience is that there's some real drawbacks people have been reading the flawed version two people pay less attention when the actual paper comes out three less likely to be important I've had difficulty where reporters want to write about the paper but the journal won't let me talk to them and then she says if it gets closer to job application season I would see the benefits out as outweighing the disadvantages but right now it seems to me that the disadvantages outweigh the advantages and then Chris outlined some of the advantages that I've already talked about to give you the punchline we actually didn't preprint the article we submitted it to an open access journal in our field and it was accepted pretty quickly which is great news and so it got right out but I want to respond to her comments now that we made that group decision one you can update your preprint it's got version control and you can respond to that feedback on the fly two when you submit an article and actually gets published bio archive updates your publication link your actual peer review publication and three you can see prior to your Times article this barrier is eroding pretty quickly which is great and that's really community driven so there are lots of resources available in ASAP bio if you're interested as I say I'm just an ambassador holding the flag and I just want to thank you for giving me the time to talk about it and I'm glad to talk to you more about promoting preprint America awareness presentation very interesting in my lab but also so I have a couple of questions which is what happens to the work that actually gets peer review but never gets published or that people never follow through and then what is the process for editors if you were saying now dedicated like an idea just identifying you as an editor who goes straight out okay I saw your publication yeah great question so my first response is we actually have an orphan one my lab is one orphan sort of preprint out there that actually went through two rounds of peer review and my PhD students said I have better things to do and he did and so there it stands and it's been read you can see that it's being accessed from time to time but it's probably never going to be in a formal journal and I think it's open to debate what good that is but there's a lot of papers a lot of preprints out there there's a time stamp this was put 10 years ago I got it easily seen when something does get published correct it didn't used to be quite so good about that but now I've been finding that that gets updated actually before it even hits PubMed you can see a little red text the thing now published in X with at least what we've been seeing your second question about how does this work from the journal side yeah so I've gotten contacted by well basically staff professional staff from the journal looking to to see whether we send that article to their journal and so I know E-Life has one person who's got a part time part time of their effort is dedicated towards looking at these preprints and I'm sorry one more you said something really that there's a way that one could identify potential reviewers yeah so people go there and they comment hello so my last question was that one could find potential reviewers for a paper who have I guess posted comment to identify themselves and then associate editors or something can just reach out to them and say hey you want to read the reviewer right so that's happened sort of in two ways one I've gotten feedback directly by tweeting out my recent preprint and folks have said hey looks interesting and then send me a message with comments I have an offline sort of e-mail dialogue about that to clarify the comments talk to my authors paper and response we post a new version and then when we submit it we say I ask in advance would you be willing to serve as a reviewer as I send it to the journal of course that's up to the editors of whether they take that recommendation that's happened a couple of times the second model is actually really a whole other exciting work that I can't really comment on which are basically journal clubs focused on preprint reviewing and they're organized in various organizations sometimes they're online at a particular school or in a particular online community they will review the paper and then they will provide share that feedback with the author the author can take that that sort of preprint peer review and package it through one of these channels PCI evolutionary biology for example package that say I am sending you peer review and an article at once if you guys an associate editor can take a look at that and make a decision sort of a go vote no go decision right now so that's pretty advanced but within a year so less than 12 months I will let you one potential issue you didn't mention was better discoverability if you're not famous or Twitter famous it is lost in the ether can you say something to that I mean it's nice to be famous Twitter famous if I put it on there and two people read it it's almost as good as not you can say the same thing about any article though I mean so I'm neither of those my Asian dex is mediocre I've been doing this for a while I'm average but yet our work is being found more often in my sort of anecdotal experience when I pair that with preprinting now you know there's no substitute for it some of the stuff that didn't wind up on preprints is getting plenty of attention because it appears in shiny journals with really you know well renowned co-authors so there's no kind of see a it's a fair point though I mean often preprints go unnoticed for you to cite and then it's just a it's a quite a benefit really quickly I want to point out that preprints are being indexed in Europe PubMed Central they're not in PubMed now but they are getting the discoverability is increasing yeah that's right they are and they get sucked right up into Google Scholar and sometimes there's an issue that sometimes that doesn't that when Google Scholar doesn't get replaced and that's a bug that lots of people know about but yes they're easily discoverable and Jolie have a comment just quick question have you had experiences where reading the preprints have actually changed what you do in your lab because you see something that came out and you say is that with methods that people use or results that they use yeah it's a great question so we have something we're dealing with because we're sort of partially scooped and so we're in communication with that that team to talk about how do we sort of slice this up and they have priority they've sort of established primacy on some part of that method and it's at least accelerated the dialogue I mean there's sort of two ways to handle it like a Russian trying to beat them but that's not the spirit of it we definitely see methods there's a fabulous method right now out for single-cell analysis of yeast that's still caught in that morass of peer review at a big journal but I have it on bio-archive and I'm in correspondence with the author to implement those methods in advance thank you great alright our next speaker is Denise Kai Assistant Professor of Neuroscience at the School of Medicine, Mount Sinai thank you so much for coming thank you so much for inviting me to be a part of this exciting dialogue so I'm going to be sharing about our miniscope project that started at UCLA and continue UCLA and we just and we just talked about extending the miniscope team to the east coast so let me first start off by acknowledging the people and really talking about how this came to be so when I started my postdoc I was a postdoc at El Sino Silva UCLA and I really wanted to study how is it that our brains are able to code all of these different experiences across time and we thought you know would it be great if we could peer into the brain and look at the neural activity that represents these different memories across time and at that time there was a technology of miniature microscopes that was just not accessible to us we just couldn't afford it it was too expensive and we were celebrating in Paymongo Shawnee's lab over some champagne and a lot of good stories started with some liquid courage and you know after many bottles of champagne Paymong said well why don't we just make the technology ourselves and share it with the world and that was kind of the spirit that we continued to carry on through this project so this project was really founded by Paymongo Shawnee, El Sino Silva three faculty members at UCLA that came together and for what I thought was a pretty brilliant team Dania Aharoni who was a physicist at the time studying dark matter and we joke that we brought him into the light of neuroscience in collaboration with Tristan Shuman who is an electrophysiologist so he's used to recording neural activity in live mouse brains and then myself, I'm really a psychologist and you know I like to think I'm a mouse whisper and can get mice to behave in interesting ways so the three of us came together and worked together really well and I really want to highlight kind of this very collaborative nature that we all bring in our different strengths we complement our different strengths and the way that we worked together Daniel would make the miniscopes Tristan and I would break the miniscopes and then together we would fix it iteratively over many bottles of scotch so the goal of the whole thing was to be able to as I said peer into the brain of a mouse while the mouse is behaving learning, interacting with the world and look at the neural activity that represented these mental computations so what you see here is the very first version of what we call the miniscope which is just a miniature microscope which I'll get into in a bit and what you see on the top right is the neural activity and this is the wrong way and the way we're able to look at this neural activity is we infect cells with a genetic modifier which when the cells are active and they express green fluorescence in which we can then capture it onto the cell camera which I'll describe in a little bit more detail we apply very simple math to align the frames and to pull up the signal and here you see about 600 cells kind of encoding the memory of this novel experience okay so this was what we started out with actually a group in Stanford invented this whole thing so we didn't even invent it so Mark Schnitzer group at Stanford what they did was they took a large tabletop microscope what most of you guys are used to using maybe in your high school biology classes or college classes and what they essentially did was they took out all the non-essential components and only kept the important one so through that all the way and then they took all the control electronics and took it outside and miniaturized the imaging sensor which is essentially that little camera chip that's in your cell phones the light source which was just LEDs and some optics some blow filters and they miniaturized it and put it on the head of an animal and so this was published in 2011 in Nature Methods and then they formed a company called Scopic in which they commercialized it and now sell it to anyone who wants to spend $150,000 on this system but for many labs that are not HHMI or rich or labs like ours we just couldn't afford it we thought this technology was just so transformative that we wanted access to it that we thought okay let's just develop our own version and really inspired by Obany Fizz and Allen Institute that well what if we open source this and maybe 10 labs would want to use this okay so I also just want to give a shout out we're not the only ones developing this miniaturized microscopes for open source platforms so Sheena Jocelyn and Paul Franklin's lab in Canada also by archive and now published in I think her biology protocols as well as Tim Gardner's lab has developed one for miniaturized for songbirds and 10 lab that NIH has also developed a scope so we're not the only ones out there but I will talk about our project so this is our UCLA miniscope and we really like to think about this as kind of two parts of the project and so we think that we developed the entire system with all the hardware and associated software for the use but we also spent a lot of time developing this resource and I also just want to comment one thing about what Josh was saying that we had to have something maintained and ready to put it out there and we had to spend a lot of time thinking about at what point would we share it we would just share everything as we were going along and we decided that we wanted people to be able to use it and so we had a test and so Paymonga and Sheena Jocelyn why don't you guys recruit some high school students if they can read through your tutorials and watch your YouTube videos they're just ready to share it with the world so that was kind of our testing bar so we did only use high school students we also used undergraduates as well they were all able to do it so we thought this was at least ready for the next step to sharing so in addition to the head mountain scope we also designed data acquisition hardware and we just used USB 3.0 you plug it into the computer the computer just reads this as a generic webcam and so that way that it will hopefully be compatible with every Microsoft office upgrade that you'll ever have to do and then you can also pair it with the behavioral camp any kind of webcam out there our data acquisition software reads webcams and time syncs the videos together so you can record both neural activity as well as the animals behavior now what I think is the coolest part of the project is all of the resources are provided on a wiki backbone, miniscope.org you do have to register to access to access the website but we basically accept everyone and so we think this is really aimed at at least two different types of users so there's the end user the people who just kind of want plug and play and to be honest it's not quite plug and play you still have to assemble but people who don't necessarily want to innovate and develop they just want to take the system ask your biological question know what the answer is to the brain there are also developers who maybe feel like Josh or someone else with an engineering background or computer science background take the source code or take the design files and they can innovate and modify and add all of these features and hopefully they also then communicate those versions back to the website and so this has become a really active sharing discussion forum and then there's everyone in between so we work very closely with our machinist who helps machine a lot of these products for our users and he actually showed me a picture someone was sitting at a coffee shop drew a new design on a napkin took a picture and sent it to the machinist and said hey can you modify the miniscope with these new features I don't know how to use AutoCAD but here's a picture on a napkin and then our machinist did it so we really try to work with the community so that people with all different levels of experiences have a great idea and if we can connect you to the right people to help make your ideas come true we try to help with that in addition to this what we spend a lot of time doing is also giving hands on workshops because we understand not everyone who reads our source code or tutorials would get it and so we try to interact one on one with scientists from all over the world so on our miniscope.org website two and a half years ago so we just hit 3,000 wiki users it's a heat map of registered users on the wiki and we know about these 400 labs that have bought the parts for miniscopes and have built it in their labs and what's been really interesting is we only know the number of people by the discussion board as well as our vendors will update on how many labs are now buying the products we don't sell any products but are buying the parts to make the miniscopes but what's really exciting is that sometimes on Twitter someone will tweet at you sale a miniscope team and they'll be like oh using miniscopes and blah blah blah and we never knew that they were using miniscopes and they never interact with us and just by going on the wiki and downloading the tutorials and the part files and everything they were able to completely do it without our help and so the price of this is also one one hundredth of the commercial price and so we think that this really reduces barriers for science and so for example we were recently doing a workshop in New Zealand and they were saying you guys are lucky in this state because it only cost $150,000 $150,000 here in New Zealand cost $250,000 for the same product and if you guys didn't commentate this workshop we would never have access to this kind of instrumentation so so far we've been teaching these workshops for two and a three more years and so we've had 12 hands-on workshops we kind of traveled the world and we decided well let's just take all barriers away and wouldn't be fun if we just raffled off free miniscope systems out of workshops so every year at SFN we ask our vendors to donate parts so that we can assemble 10 systems and for free the workshops are free you come and you learn and you hang out and then we also just raffle away 10 systems okay so when you come to these workshops, thank you what do you learn, you learn about how to assemble the entire system all of this information is available on our website so I won't go over that we also teach you how to test it just on slide then we also teach you which I think is the hardest part is how to perform the surgeries so that you can implant this animal animal doesn't die, cells fire and they behave and so all of these workshop resources are available so if you can't make it to a workshop or you go to the workshop, you're inundated you don't remember anything, you go home you log on to the workshop you log on to the miniscope.org website and under workshop resources it has all of the powerpoints that we teach you today so you can refresh your memory or go on to the discussion board so one of the benefits about doing this kind of open source science as Josh had mentioned is not just that you have something plug and play but that as scientists we want to constantly hack into our system I really like that term and modify and make it changes for our specific research questions and so for recently for example one of the changes is to make it wire free so we wanted to explore more naturalistic behaviors in which the animals were not tethered so we developed a new wire free system or the no strings attached miniscope and so now instead of powering it through the cable we can power it with the lithium ion battery that we tethered to the miniscope and then instead of transmitting data through the cable, we just data log it onto a micro SD card and this is L on bio archive and we're hoping to show the designs on miniscope.org we're happy to show the designs with anybody but right now it's just getting the manpower to get it assembled in an organized fashion so here's just an example of an animal wearing the no strings attached miniscope as is exploring social cups there's females underneath those cups and it would visualize neural activity or here we're having mice who want to learn spatial maps running down a 25 foot long meter track for example and I would just make one comment I don't have time to talk about all yes it's a wrap up the hardest part of where we're at now is what do we do with hundreds of thousands of cells how do we make that into meaningful information so a student of mine is trying to take a lot of published analysis out there but creating an interface that is helpful for both people without coding and then biologists who've never touched MATLAB and creating a Python package in between that is user friendly and that gives you lots of visualizations and intuitions about altering parameters does to your output of your data and this is currently on GitHub and this is summarizing what I just told you of our system both the hardware system and the resources and to thank everyone that contributed my collaborators and funding sources and again Daniel everybody who's really the genius the physicist behind all this but we all work very closely together and thank you so much for your attention this is a very silly question but maybe you would just be nice to verify that what's a company patents and designs or something anyone can go in and rebuild that themselves and there's no bridge on a patent they're not selling anything making any kind of profit so that's a fantastic question and we do patent it by Stanford and License and Scopic the company that commercialized it but as research institutions we have an implicit understanding with other research institutions that we can build it and for research purposes this is why we don't sell anything we make zero dollars all of this is volunteer and so as long as you do for research purposes however with that said there are companies like LabMaker and there are other companies that have commercialized either the components or complete systems so there's a company in China and LabMaker I think is a company in Germany so there are international rules that may or may not protect them from such infringement so that's wonderful that it really is open source and thank you also that allowed sorry did you allow the original patent allowed for commercialized sort of a little bit more user friendly way more user friendly so it's great for the community I'm curious how you stay funded right so you just mentioned this amazing story of somebody sending them a picture of something to be machined who's paying the machinist and how does this innovation and also support for the community keep going so this original project was funded by some funding from the Dean so our Dean said here's a little bit of seed money see if you can get something working and I think the open source spirit really caught on and I really think we bought a lot of good will in the community so then when I was a postdoc at UCLA we got a U01 to really do the not only develop the technology but also share it and then before I left it was another $10 million in NSF that funds all these workshops that we go on and so I think NIH is really trying to encourage open science and open sharing data reproducibility so although I can talk about it a lot of our raw data is put online so people can download it and take a look at the raw pieces so I think there needs to be more funding sources and more opportunities for this kind of open science development alright so our final speaker for this session is Anisha Keshevan who's a postdoctoral fellow at the Institute of Learning and Brain Sciences at the University of Washington so the name of my talk is about going from the wet lab to the web lab so I'm primarily interested in studying the brain so here you can see a 3D reconstruction from an MRI image of the human brain you can see this really unique folding and we're really interested in how the shape of the brain relates to normal human development during adolescence how it's related to mental health disorders to general neurological disorders and also how it changes as we age and the thing is we really need a lot of data because we're also unique we're also special and our geometry is all so different that we need to make sure we sample from a really large population so we really are now moving towards collecting a lot a lot of data and it should be easy to scale up right to scale up our analyses to really accommodate these large data sets because I mean we have computers and we have really like fancy algorithms but the reality is that we're human and so this Lucy this is how I feel it's my relationship with data because the machine is just like producing it and I can't keep up because I'm only human oops but the other thing is like humans we have our strings so in this example it's a deep learning algorithm that's like 57% sure that the first panda image is a panda you add a little bit of this like salt and pepper noise and now it's 100% sure it's a given and as humans we would never make this kind of silly mistake and this is funny but when you think about self-driving car technologies here's an example where these kind of vandalized stop signs are being read as 45 mile per hour you know speed limit signs and then this right turn is read as a stop so clearly we have our strings and I think that we really need to combine the strengths of algorithms with the strengths of humans so algorithms they're really fast but sometimes they can be really stupid humans can be slow but sometimes we can be smart so I really think that by designing the optimal interface we can perform better signs especially for large data sets and the thing is that design matters so here's a book called the design of everyday things and here you see a really dysfunctional teapot and the idea is that there should be designing machines that work for us rather than designing a machine and then us adapting to how it works and in science my hypothesis here is that open design really matters because if we work openly we can overcome our limitations and we can really optimize for our strengths so we're inherently slow so we should design for collaboration but we're smart in some ways so we should really design for the human visual system and so I propose that the internet is our functional teapot so right now I'm studying pediatric mental health and there's this really large data set being released from the healthy brain network where they're releasing 10,000 MRI scans of children aged 5 through 20 and this is a really important age for mental health because 80% of mental health diseases are diagnosed in this age range and the main scientific question we're asking is how do brain tissue volumes change during development and this is really important because if you think about these normative growth curves for babies we don't actually have normative growth curves for the brain yet and so it's really important to understand normal variability in brain structure and then understand how individual differences in that variability are associated with mental health and the catch is that children move a lot in an MRI scanner and this leads to a very bad quality image so for those of you who haven't been in an MRI scanner you need to sit still for 5 minutes to get a really good image. So because this is inherently visual we need to do visual inspection at every stage of the analysis and this can get really unwieldy in the standard wet lab model because asking the people around you to review 10,000 images is not a cool thing to do. So like a good millennial I turned to the internet and I wrote this app called Mind Control it stands for Brain Quality Control and it's a collaborative interface to crowdsource the quality checking of these MRI images and so you can go to mindcontrol-hbn or healthybrainnetwork.herokuapp.com and here we have a little leaderboard so we can see that other scientists have actually started visually inspecting these images and you can open for example here's a bad image and you can see it takes a while to load because this is a good amount of data and what you're going to be asked to do is to click pass or fail and then rate your confidence score. So finally this brain image and you can scroll through it and you can look at every single slice which is our normal procedure for this kind of thing can change the brightness and contrast and you can see there's kind of this like banding and there's this blurriness that you can't actually tell the difference between the two main types of tissues in the brain the gray matter and white matter and this is what we want to measure so you can see from here that we get a really bad measurement. And so I got some expert feedback from my former mentor Satya Ghosh and I wanted to play on my phone and at first I was like oh my god I spent a year writing this like thanks a lot but the reality was that I wrote you know a dysfunctional teapot so I tried something new I kind of iterated on the design and I wrote something called brainer officially it stands for brain data review so I went on Twitter and I wrote are you at work but feel like playing Tinder why not play brainer instead and help neuroscientists rate the quality of brain images so I've left to feel bad quality images built with VJS and Firebase hashtag citizen science and I maybe had like 50 Twitter followers at the time and I'm very proud of this tweet because I think now it has over 400 retweets and over 600 likes and this went viral I know I'm proud and the instructions are pretty simple if you see a bad quality image you swipe left as you would do on Tinder and if you see a good quality image you swipe right and you could even win prizes I really had to commit 100% to this whole idea and so depending on how many swipes you make you could win you know some mysterious monkey badge and this turned out to be really popular we have someone who I don't know who's really really like committed to winning that monkey badge and did 6,183 swipes and there are a lot of people who just helped out from the goodness of their hearts we got so many ratings each image was viewed 20 times but often there was no agreement almost kind of like Tinder people just started swiping right for everything and I thought what have I done I tried to solve a big data problem by collecting more data and now I have to somehow fix it so what you're seeing here is this kind of bimodal distribution of images that mostly pass and as this was updating in real time this peak over here was just slightly moving to the right and I thought this is bad what have I done so we needed some way to down weight the raiders and this is where algorithms really helped us out so we created this matrix of image IDs and raiders and we had this like expert rating that came from mind control and each raider was a feature and each image was a sample and keep in mind we had some missing data because not everyone rated all of the images but we used this algorithm called XGBoost which handled the missing data and it's kind of based on random forests and it actually gave each raider their own important score and it did a great job at kind of removing the people who did many many swipes and it said ok you weren't very important so this was promising to us and so the corrected rating distribution looked like how we would expect and from there we were able to feed these labels into a deep learning algorithm which basically gave us a really high accuracy score the ROC curve was .99 and so at this point in time we have over 100,000 ratings over 400 users and near perfect accuracy from our classifier so yes I declare success it's a functional teapot and so maybe some of you are wondering right now this is great but how do I build something like this of my own and thanks to some funding from Elife Innovation we've created swipes for science which is kind of this generalized template to create your own citizen science app you could use it for classification of images and so this is Brainder List which is Brainder for lesions a collaborator at USC has run various stroke lesion segmentation algorithms on her data but she doesn't know which algorithm is the best so on Brainder List you can swipe right if you see a good quality tracing of a stroke lesion. We also have Wailder this is a Tinder for Wails so oceanographers at the University of Washington really liked Brainder they said oh my gosh we have sound recordings from the ocean and occasionally we can hear whales but we don't have any sort of training data set to create an algorithm to automatically identify the whales so I said great let's do Wailder so I created a spectrogram of a 5 second sound clip and you can swipe right if you think you hear a whale or a dolphin or some sort of biological signal you swipe left if you don't hear anything for text annotation we have something called abstract so this is an app to annotate abstracts I was trying to do some background research on autism and I wanted to know basically what are the sample sizes of autism neuroimaging studies throughout the years are they increasing are they decreasing how big are they and I realize that this is really hard to get because this information isn't the sort of metadata that's stored for scientific publications so an abstract you see a scientific abstract and you see all the numbers are highlighted and you simply tap on the numbers and you submit that information to our database and then right now I'm working on brain spot which is where you spot the region of the brain that has gone outside the segmentation borders and so it's basically an annotation of XY coordinates on an image so if you have any data annotation needs please come see me but to go back to our scientific question how do brain tissue volumes change during development and so I posted a preprint it's on bio archive as a PDF but it's also in web form and so if you scroll down you can see this is all my text and you can see that we have an interactive figure for figure four another random thing really quick is that I added a little unlock sign next to all the citations that are open access and by clicking the unlock it takes you directly to the actual PDF like Wikipedia I hate it when journals you click on the author name and it takes you to the citation like that's useless I just want to read the paper anyway okay so how do gray matter volumes change during the ages of development from five to 20 years old so here we see a scatter plot on the X axis we have age and on the Y axis we have gray matter volume in terms of cubic millimeters and your hurry up each point is linked to an actual brain image so when you filter the data by its quality you can see why certain images were left out of the analysis so this kind of helps you with filtering your data in a fair way you can change how you threshold your data by data quality you can also change the metric that you're looking at you can choose to split by sex you can do a polynomial fit if you so want and you can see this is really great for data exploration but it's also a bad design in a way because I'm p-hacking in a way so I also added a thing for a number of comparisons so basically it's a really fine line between a functional teapot and a dysfunctional teapot on the one hand this interface was great for exploratory data but it was also terrible if you were actually using it for p-hacking so to summarize of course open science is honorable it's great that I got people involved it's great that the data is open and the paper is open but really I think open science is strategic especially in the age of data-driven discovery I don't know how as we start collecting more and more data sets I don't know how we're going to review it visually and we have to do good science so with that I would like to thank my web lab only four people on here are actually at the University of Washington with me everyone else I've interacted with through Twitter or through Brainder and so I'm happy to take your questions Wow just really amazing we need to talk afterwards but we also look at our brain development in my lab but so within that question it's very so story can you use this in other types of data beyond just brain structure resting say you know like basic EPIs and so forth yeah you can really as long as you create a JPEG of whatever you want to annotate you can add it to swipes for science and what sort of expertise do you need so you know the swipe left swipe right could be it could mean anything yeah so right now we've hired a user interface UX developer and so she's really working on prototyping the like optimal tutorial template so you do need to write a tutorial for your users but the ideas that would be presented in a way that is intuitive to people who aren't scientists and it's primarily geared to identify artifacts resulting from head motion for Brainder yes yeah excellent work thank you so that was really lovely and I love the gamified component but I'm curious if you've looked at how that might compare to say using M-Turk to do like codifying our annotation for you in the same sort of setup instead of like sending it out to the interested masses just pay people a little bit of money to kind of do the same thing we haven't used M-Turk yet it's definitely something we're considering yeah I kind of just I kind of just wrote this I didn't quite expect it to be so catchy to be honest I worry a little bit about M-Turk because the motivations are a little bit different and also I want to make sure that I can pay my workers really well I feel like there's a lot of instances where people aren't paid enough so once I have the funding to pay people at a fair wage I think I would definitely consider it yeah select a bar stool 10 or 15 minutes to ask them all some questions so moderator I'm going to take this opportunity to start and ask you guys a question so in Rebecca's opening remarks she talked about the future of science is this community working together and not individual scientists and I think a lot of you touched on that and so I was hoping that we could go down the line and have you elaborate a little bit more about what you see as either specific opportunities or specific challenges to this new model of working in science or what we need to do to adapt our workflows or our standards as a community and how we do that so yeah I basically talked about this about going to the web lab I think the biggest challenge right now is that web technology is this ever-evolving field and it's really hard I mean you have to be a full-time almost full-time web developer to really kind of build the interface to work with people on such a large scale I'm really lucky that my postdoc supports me learning this stuff but I think it's a lot harder for other people so that's why we're building Swipes for Science is a way to make it easier yeah so I mean I obviously pitched the advantage of getting the literature out as soon as possible through pre-printing but I'd say one big challenge is that I think folks are appreciating the opportunity to work the research out and having you know a citable inversion record of their work out there but they're not sort of embracing the open science value of a pre-print and if you you know so I will go to a pre-print and I will look for the data and look for for the for the raw material that would help me evaluate it and also compare it and use it and it's maybe buried in a PDF and not actually something that you could say upload into some data processing tool it may be hidden it may say that the raw sequencing reads in the world of genomics are yet to be deposited and that's sort of against the spirit of the approach and I'd say that you know ultimately we're going to have to make that mandatory sort of co-submission for for pre-print records so in terms of community development and neuroscience I think we are not it's really difficult to take advantage of all the community development that is happening like within every lab there are people with developing code and building tools that they don't necessarily have the time to share but if we had better platforms for people to make those tools available and especially people whose job was solely to take things that are built in labs and there are maybe not ready or are not polished enough or documented enough to be widely shared but integrating those across lots of different labs and creating tools based on those contributions that will be shared more widely because I think there's a huge amount of development that's happening inside every lab and very little bit actually gets shared. I think a practical challenge that we think a lot about is how to get your postdocs or restaurants the credit they need so that they can get tenure track jobs if that's what they want in the future. So for example when I was a postdoc at UCLA the three of us worked really closely and we were all in because we believed in the site of the labor project and somehow luckily with one paper the three of us all got tenure track positions but that's not common and it's really hard I think to commit to three different postdocs for example from different fields that work on a project and share one paper because somebody still has to be first co-first. So I think that has to do more with how we evaluate work in science and how we assign credit with journals and all of that and I think our current way of doing it is quite archaic and it's not supporting this idea of collaborative science and I think we are moving forward to changing it but I'm curious about how you see funding for the kind of positions that oversee collaboration or support collaboration that aren't necessarily the science we need to I mean so I think the big funding agency should think seriously about contributing more in this domain I also think it's possible for labs to pool some of their funds to create these resources or sort of centralize repositories where people can dump the code that they've written and then people will take it out and polish it and make it more accessible to others given the amount of money that's currently being spent on closed source tools we took a small fraction of that from each lab and pooled it then I think you can have a big impact but of course somebody has to oversee that and make sure it would require a pretty big change in the culture but practically I want to comment I think I'm fortunate to work with a program officer for one of our U grants who's really trying to to patch that whole insufficient investment in teams and people to build tools that enable data sharing but it happened after the fact so the awards went out and everybody started generating all sorts of data they said you need to share and we're like we're trying but we can't or at least we can't link the relevant data so I think I guess my broad advice is talk to your program officer a priori and encourage them that if you want to have the kind of success you want to have as a program you might want to consider building this into the portfolio you may wind up finding yourself if that's something that you want to do in a long way so I would say I'm really proud to be at Pitt but the School of Medicine is a big battleship that's trying to become more nimble here and so the processes of evaluation and the processes of hiring of developing the kinds of positions that maybe aren't a normal tenure stream and I sort of a new model that is really somebody who's going to be say a professional data parasite but a data parasite that's going to be sucking together all the good data that multiple labs are producing those kinds of institutional changes they're being talked about but it takes a while for them to trickle up to the point where they're properly recognized and paid I'd just like to comment on that and I think that's really important that we're now in a proper decision to go there was also because of the institutional support for Open Science and particularly our project and they knew that we wouldn't, by the way, necessarily get NIH or NSF funding or federal funding and so they supported us generously and through very creative resources to get us what we need to continue to do this Open Science kind of work so yeah, institutional support is absolutely critical especially at the early stage Really great presentations so much, a lot of food for thought I have kind of a crazy question each one of you in the space at a conference that I was yesterday about the Innovation Institute where, you know, they really work with you to try to get funding so you can start your own company to develop your own project and then diversity makes money, you make money and I just wonder I mean each one of you could have, you know, gotten a patent and start making a lot of money what's pulling you towards just being all giving what are the you know what I mean that comes to mind, I'm just really curious about it so I was just in a job interview the other day and I was showing another thing I'm really passionate about is data visualization especially for clinical data and I was at a hospital and I was showing the chief data officer and the first thing he said was you should start a company, blah blah blah and my response was that I really wanted to be in a hospital setting, I wanted to be in an academic setting because really I'm kind of the glue that brings people together and without people around me I would feel like not as effective so I don't know if that was necessarily right but yeah, this is said to be a lot especially when your web developer and the developer right now is like just yeah, so I don't know is the real answer but vaguely it's that I want to work with people I mean I'll just comment that I'm on both sides of that so I'm working with a group from Innovation Institute on something and I don't have a clean answer for you except to say that we're keen to share technology particularly for education and basic research as soon as possible we sort of figure out that it can be monetized we do so a bit a bit late so maybe it's it's just sort of a lack of strategy we do have something in the works and ultimately we're trying to figure out what's the best way that we can get to where they are which is sort of licensed the technology but ultimately distributed freely for those who need it free for me I think science was always the number one priority and put in the extra effort to make the tools that I built to do science more accessible to others kind of selfishly so they would then contribute back and make the software better for me down the line and it's definitely paid off I'm still using Open EFIS on a daily basis at the Allen Institute and because I mean it's flexible and because there's been such a good community development effort the software is much more robust and usable and it would be kept up and so I guess I never really had the motivation to start the company I mean we turned Open EFIS into a nonprofit so that we can have an open bank account and just be able to exchange funds and have some money to hire support people but turning it into a full-fledged company I think was never So I absolutely believe there's room for both commercialization and research taken I think we had always had an idea so we had a lot of people that said could you just turn scope into a company so we could actually like get you to help us rather than just depend on discussion boards or whatever and we used to even have suck it up Fridays because we just couldn't answer all the emails and so then it was like every Friday we have one hour where we just respond to all the emails and whatever didn't respond in that hour because we were postdocs and we actually had produce papers and science firms supporting us but I think our idea was always that we would continue to patent for the protection so that we can continue to disseminate open source but we could also have option to commercialize it at a reasonable price so that labs could then actually pay for the services because not everyone wants to figure things out themselves and I think there's absolutely room for both as new faculty just haven't had time to even get things going to my lab but I think that something certainly in our future that would go beyond mini scopes but just no technology in general I just want to quickly comment I think that that model that you guys have built is remarkable and it would be great if that model were more broadly shared and all that I can say is still foreign to the innovation institute at Pitt which has transformed for the better a lot recently but this sort of model where you're actually going to protect it to some extent and then you're going to give a lot of it away it's still great deal of attention particularly about the kind of credit that investigators would get for that model so it seems like the answers to the first couple of questions really sort of arcane toward micro publication the idea that you can publish a figure a short piece of software or something like that and we also talked about how it would be nice to have different funding models that would accommodate that but we can want lots of things and not have them so what I'm wondering is do you have ideas about how to shunt micro publications into the current sort of reward systems in terms of funding, tenure and activity so so yeah we're speechless because it's a terrific it's a terrific challenge so I can say that my former and current institution look at that differently and my former institution was moving to consider those in a sort of as an addendum section under VEDA on by the P&T committee which I sat on that is definitely not the case for the school medicine at Pitt yet but I think the science is moving I mean as always the science moves faster than the evaluation processes it's going to change when some of those so when some of us are actually on the other side and I think we're really excited actually about where our community here in Pittsburgh is moving to recognize that I would say that that discussion has not formally even happened so yeah I had a question that's related to money again and the question is more there's already a system in place where universities get IP for a discovery that gets patented when you apply for a grant to build an open system how does do you have to actually say in the application that this is going to be open and does your university have to sign out on so for example in some of the miniscope grants that we've gotten it's very specific exactly how we're going to be open how many workshops how many people are we going to teach this to what we're going to put out there when we're going to put it out there and this is actually why we bio-archive the wireless because to wait for the time publications come through NSF has been like no you guys did not meet what you said you were going to do you did not make it accessible to the community so at least from our experience we've had to be very very explicit and they come through checks and we have to send very thorough reports of where we've gone to teach these workshops and how much they're teaching components I just want to say that early on in the opening business process we applied for a few grants to get funding both in the United States and Europe and we specifically said we're building an open source system and one of the main criticisms was that this doesn't have to be open source this is something a company can do and so there wasn't that much interest in funding the open source product I think that sentiment is changing we'll probably have more work early on maybe five years ago or so feeling that open source wasn't that valuable so for my postdoc fellowship I'm a year into my postdocs I'm still relatively new to this but my postdoc fellowship is really great because they encourage people who work with open science and who develop open source tools so that's been really great previously at UCSF where I got my PhD I just didn't really tell anyone so thank you guys we're going to take a short break we'll reconvene at eleven there's still coffee and plenty of snacks out in the lobby you can bring food and drink back in the librarians give you permission so go and discuss the food hi how are you we're combining we're not we have a team but they want to get into the room we don't have we have a team we have a team we have a team we have a team we have a team we have a team we have a team people on the steps there's a director congratulations thank you thank you Good. Yeah. Is there one? Yes. Uh, part of the week. Yeah. We have a famous poem. Yeah. Do you know about it? I don't know your poem. Uh, there was a big war where Norway seen her house week. It's a hundred lines long. It starts off with three, ten thousand speeds.