 Scientific publishing is broken. The internet was built by scientists for scientists so that we could share our research to more people faster and easier. The internet has changed the way that we share a lot of information. I'll give you an example. So I arrived in, let's say, Brussels. I'm not familiar with the city. I want to find restaurants that I want to go to. In the past 100 years ago, perhaps, I have to rely on a guidebook written by a single person. It had pages and pages of restaurants which you have to manually flip through. You have reviews from maybe one or two people in the book. And if you have the address, you have to sort of navigate another map to go there. Nowadays, you just go onto Google. Within 30 seconds, you can find all the restaurants that are located around you. Click onto each individual, one of them. Look at the ratings, filter by whatever dietary requirements you have. Look at the reviews by all the people, what they have reviewed before. And with another click, it gives you real-time direction and timings as to how you can reach that restaurant most efficiently. If you look at how research is communicated between 100 years ago and nowadays, it's true. Yes, we now have links that we can click that you can go to related resources. But it's not like we have open review information or reviewer's information. The interactivity of the data or the graphs that are shown have not really progressed that much. There's clearly room here for us to improve the ways that we communicate research using web technology that we have. I quoted our editor-in-chief, Elias editor-in-chief Mike Eisen at the beginning of this presentation. I'm going to borrow from his presentation. Again, I encourage you strongly to watch the whole talk on this YouTube link that is on my slide. What is this mass rover doing here? So on average nowadays, it takes nine months for a paper to go from a submission to a acceptance. Nine months is more or less the amount of time that you send a rover from Earth to Mars. So imagine the situation where you print the paper out, you put it on this rover, you send it to Mars, you have this rover churn out the paper on the surface of Mars, you use those little camera eye thingies on the rover to take photos of those, you beam them back to Earth and that research that you can see on camera would be open to everyone because this is like government funded and all. It would be taking the same amount of time and the most shocking thing, it takes even less money than what publishing costs. Mike Eisen did some calculations. I'm not going to spoil it. Please go watch that video. He's fantastic. But so I hope we can all agree that publishing nowadays is slow, is expensive, is closed, and is emotionally draining. Who likes reading papers here most of the time? Okay, that's good. When I was a PhD student, I mean I can read maybe one in detail every day, maybe two maximum, but it's just really, really draining and I wish that I didn't have to print all the PDFs out and all of these things. I wish, we wish that there's a way to make publishing and research communication faster, more cost effective, open, and ultimately user friendly as well. And this is tied into the mission of Elife, which is the nonprofit organization that I work for. Our mission is to help scientists accelerate discovery by operating a platform for research communication that encourages and promotes responsible behaviors in science. So I'm the Innovation Community Manager at Elife and I operate the innovation part of this organization. How many of you have heard of Elife? Great, that's more than I expected, but I think most of you would have known us for being a journal in the life sciences. So we do operate a really good journal in fully open access, completely online in the life sciences domain, publishing papers from cell biology to neuroscience. But actually what we're doing there is we're using this as sort of a playground for experiments in publishing. So those experiments come from the Innovation Initiative and the Innovation Initiative's mission is to drive open innovation for open science. So yeah, as I said before, our overall mission is to change publishing from slow to fast, from expensive to more cost effective, from close to open. The method that we employ in innovation is through the use of technology. So what do we mean by open innovation? What I mean by that is not only open source, but something that we're calling open by design. This concept has been pretty much inspired by the Mozilla Open Leaders Training Program, which is fantastic. It's not just about opening the code, which is also important, but actually as the speaker before me was talking about it, to actually reach out to the community, it's very intentional, it's very strategic, it's very inclusive. So this is how you can really factor in the input from multiple stakeholders and making sure that all the users are included in the design process. Why is this important? How many of you have seen this video of this soap dispenser? Okay, for those of you who haven't seen it, it's a very simple yet striking example of why we need to have diverse user testing. This is the soap dispenser. In this video, a person with pale skin put his hand under it, and they managed to get soap out of it. Another person with darker skin put their hands under the soap dispenser. There was no soap coming out. So clearly there was a problem of testing and design process. If you think about this in the tools that we designed for research, for example, if it's not tested in underrepresented communities, then we have a problem, we potentially could be excluding them from the process of research, discovery, evaluation, or consumption. And I'm open to arguments on this, but these are existing biases that we do not necessarily want to propagate. The other problem is, again, mentioned in the morning in this session. So if you have tools built for research that are not open, let's say someone published a paper that uses a particular package that is closed. If you would like to reuse that particular pipeline, you can no longer do it because you don't have access to it. And so I think this is why we are at E-Life, we're committed to investing in open innovation. So this is our vision to create an ecosystem of research communication tools that are open, inclusive, user-centric, and to create this with the community. So how do we make sure that these tools can fulfill these criteria? For that, we have to put together the users of the tools, so the researchers and the people who know the technology, so software developers, web designers, et cetera, et cetera, in the same space to give them time to talk to each other, to empathize with each other's situations, and then to create those prototypes and solutions. So this is exactly the reason why we designed this event. It's called the E-Life Innovation Sprint. It's been running for two years now. So 2020 will be the third year. It took place in Cambridge for the last two runs. We put together 60 researchers, technologists, product managers, designers, policy advisors, et cetera, two days, and they work on a great variety of projects. These are some of the prototypes that came out of the Innovation Sprints for the last two years. You can check them all out. I think the 2018 projects were a bit more mature. So for example, Plotted is like a lightweight endorsement sort of like button for research. You can debate whether or not that's a good idea, but the idea for us is to put it out there so that you can see it and use it and tell us what you think. And this is a theme that you will see recurring in the work that we do. So these are all fantastic projects. One of the projects that I really want to highlight that sort of came out of the Sprints as well in 2018 is the following. So this is called the Reproducible Document Stack. I'm hoping that this video will work. It's basically putting back code and data into the narration of a manuscript. So this looks like a normal eLife paper when you browser it. So this is all life. The demo is there. And with some of the figures, you can click on this button on the top left, and that will expose the script that is underlying that figure. So here this is written in R. If you remember, the previous plot was a bar plot. So I personally really don't like bar plots. So I decided to change it. So this is now when I was changing it. And immediately you can see that that was life-re-executed in the browser, and that changed it to a dot plot. So you can try this out for yourself. It might be slightly broken now because we're trying to make progress on it, but it's working. You have to trust me for that. But yes, so we've been developing this project in collaboration with two open-source organizations, Stance and Substance since 2017. In the eLife Innovation Sprint in 2018, a group of people made very significant progress on enabling us to launch that prototype that you just saw. What I'm trying to illustrate with that is something that we learned through this process. Again, going back to what I talked about before about open-by design, we try very hard to actively communicate what we're doing and all the decisions that we're making. So to make sure that the community is informed and that they can provide feedback at various points of that development. The other thing is what I talked about before that we like to prototype. So the design process for us and also something that I really agree with is not one where you plan out everything from the beginning, but you do an iteration cycle of launching something, getting feedback, fixing it again, and then getting feedback and repeat that. And so when we put out this demo in February 2019, we got very positive feedback for it and it encouraged us to develop a roadmap towards making it scalable and publishes friendly, basically. And hopefully in a couple of months, we will be rolling out a way for authors to basically submit these articles to us and to make their own articles that has been already published reproducible in that manner. The other thing that we learned is something that again, I really think, I think Robin who spoke before me is emphasizing this, is that there are many stakeholders within the product that we're working on. We need to really listen to them and to make sure that we take their opinion into account, but at the same time, it's a balancing act between launching something and also the sort of trade-offs between satisfying different types of stakeholders. So publishers, for example, want to make sure that the product will be compatible with the publishing workflow, which is very complicated. Researchers, we want researchers to be able to use whatever tool they were using before. So in the workflow that we're going to launch in April, researchers will be able to start from a Jupyter Notebook or an R Markdown or a Google Doc if that's what they were using. We cannot just ignore users who don't know how to code. We really want to make sure that at some point they would also be able to make their articles computationally reproducible. Sorry, that's a lot of empty talk for something that doesn't really exist yet, but we're working towards it. If you want updates, there's the link to sign up for them. The other thing that we learned through developing this product and also through running and organizing the sprint is that actually probably the most powerful outcome of our work and goal is to empower people and communities. So to give people the confidence and the skills to be able to develop products to use them open by design. This is sort of a summary of the new work that we're doing this year. This includes, earlier this year, we launched a new mentorship and training program to train people to develop research communication tools openly. In this training program, we teach them skills from mind thinking to marketing to sustainability. If you have an idea, please come and talk to us. We offer support from funding to mentorship to promotion. And so we are happy to work with you to make sure that you also, as an innovator, get something out of this process rather than just having a good tool out there. So that was a short talk, but here are all the things that you can do to stay updated to the things that you can participate in. So we have a community call actually coming up next month. So if you're interested, if you've never worked on research communication tools and you don't know what is out there, this dev room was a great starting point, but you can hear about some of the projects in this community by joining our call. We do have E-Life Labs, where we regularly feature open projects and people that work in this field. If you would like to participate in the fantastic sprint that I described, there's the link to sign up for updates. Yeah, thank you very much for listening and I'm ready to take questions. Thanks. About the reproducible document stack, what backend did you use for that? So how was it generated? Because it kind of resembled to me as a Jupyter notebook, but I might be wrong. Yeah, so the backend is... The demo was a binder backend. If you know binder, it's a Jupyter-based container. This is as far as I can go. Yeah, so it's basically containers, so you can access... When you re-execute that code, you connect to that container that was generated from that image that you changed. Because we want it to be reliable and performant, we are working with Stancila, who is doing a lot of work to basically make that bit more performant and making sure... Because if you launch that demo, you will realize that it takes about five minutes to settle down and that's not the ideal situation that we want to end up at. But yeah, it's not as intuitive as we thought, like when we first started this project. But yeah, we're happy that we've got great partners working with us to make sure that it's going to be performant. I think you mentioned that once you have started publishing reproducible papers, you're going to go back to existing publications and sort of redo them. Do you have a strategy of how you're going to get people to do that? Because I imagine there's going to be some resistance. I will repeat the question because I forgot that. So the question was, how are we going to get authors to actually re-do their... We publish their papers in a reproducible manner. Great question. I think it's difficult. We rely on E-Life authors agreeing with our ethos, basically. There is not much incentive at the moment to make your paper reproducible. It doesn't give you an extra DOI. It doesn't give you any credit. But it's cool. I agree. And we actually have authors emailing us asking if we could have their papers up as reproducible articles. Where the idea here is, I always say, to make it very easy to do the right thing. So the user experience that we provide is a one-click experience, hopefully. So this is all work in progress. But the idea is that you will go on Stanislav Hub. You will paste the link of your original E-Life article. So you need to be accepted first. You'll paste that link to the article. You can basically, in a Google Doc type interface, drag your code chunk into the article. Save that. Send the link back to the production team at E-Life. And we'll do the rest of the work for you. And you'll get that fancy article. And so it should be really easy. And you get something that you can tweet about. Yeah. Because I guess at some point, even if we're buying the individual there is a scalability problem. Yes. So, repeating that question, how big are the datasets that we're working with? At the moment, we are targeting mostly tabular data, so small CSV files that are directly used to generate those graphs. It also has to do with the amount of compute that will be used. Let's say you have a very popular paper that's accessed by 100,000 people. We can't afford to have everyone rerunning all the analysis if it's, like, I don't know, genome alignment. So at the moment, the first prototype, again, is something small, CSV, that can be attached to the reproducible article format that we have. But we are looking for, you know, we are working, we are, like, my vision for this, for the next step, is to work with the community to figure out a roadmap of what features we need to implement next. So that will also mean collaborating with other people, especially open data folks who have been working on this for a long time to figure out how we can work together. Yeah. There and then, yeah. Myself, I'm actually a regular reviewer for the Royal Society of Chemistry and American Chemical Society of such institutions. We often get a lot of junk on our desks to begin with. And one of the things we take these months for is actually to force the researchers to give us the original data that they used to make their publications. We absolutely, 100% sure that they're telling us the truth. So it's, I mean, I'm not so surprised by the length of that. And I've seen cases where I'd be happy to drag the reviewers through a much lengthier process because they just were trying to hide some people. The authors, you mean? Yeah. Thanks, thanks a lot. You have to really tear it out of them. Like, we need the hard data in your supplementary materials because we need to know if this is true and sometimes they even retract. So thanks a lot for that comment. If I repeat it. The comment is about how there's a reason for the lengthy reviewing process before publication because we do actually, the reviewing process adds value to the publication. I agree 100%. I don't want Eli to go out of business. That's not the reason. I do think that publishers and reviewers do add value to the publication. My question perhaps to you and others is that why can't this process be open? So I'm sure you may be aware of preprints. You can put, I mean, nowadays anyone can publish anything. You can tweet a photo of your cat and that's publishing, right? The reason why, you know, people don't maybe recognize that is because it's not been reviewed. It's not been vetted. And there is no reason why the publication shouldn't be disassociated from the process of reviewing and why reviewing cannot be done in public. So imagine in this case you will have the paper that, you know, maybe has incomplete data. Someone else in the open will say, hey, I'd like to access your data. Could you please upload it? And this whole discourse could benefit from being exposed to a wider audience, I believe. I'm happy to hear otherwise, but yes. I believe, okay. Just a comment. I work in a different domain in atmospheric science at the European Geosciences Union. Publishes 20 plus journals over 20 years now in a public review model. These are high-impact factor journals that are very visible in the community and the whole peer-reviews public. You can browse through initial version, a reviewer's comment, the replies. All the work goes public for every paper. So this is the first part of the comment. The second part is that whenever I read through the reviews that are public there, I have this feeling that why the reviewer didn't take a bit longer and go through more detail to add even more volume. I'm not sure if especially encouraging people to bring the reproducibility in which adds more tasks for the reviewer because now you can play with the simulation or something. It takes even more time if you can play with it compared to just the text. So I'm also skeptical about shortening this time. I completely, sorry. So the comment was about the fact that actually, yeah, the longer, because of the richness of the research nowadays, we can actually afford a lengthier review process. I agree completely. I just think that it's just, like why is the research paper final? Because it shouldn't be because knowledge is always evolving because you will always have an evolving research community who will join and leave and knowledge gets updated. So, you know, a paper should not be the endpoint of a certain research question. It should be a starting point for a lively discussion. And that should all, you know, be a publicly participatory. And E-Life is doing a lot of work to make sure that that is the future that we pay for publishing. I'm happy to talk afterwards, but thank you very much for your comments. Thank you.