 Thank you very much for taking your time to join us on this webinar on the executable research article project. My name is Emmy. I'm the innovation community manager at Elife and I'm very happy to be joined today today by Alex, who is a designer and software engineer at Stancila. And Anja who is Elife's marketing assistant who will be helping us host the webinar today. So, before we start just again hoping to bring your attention to the open agenda that we'd be following during this webinar. So this serve the purpose of letting you know the order of the events and also for you to be able to ask questions. So, if you have questions during our presentations and demos we appreciate it if you could put the questions in the questions last section, that's towards the end of the open agenda. Or you can alternatively try to remember your question and we'll have a dedicated Q&A section towards the end of this webinar where you'd be able to use the raise hand button in zoom to be able to ask the question verbally. And we will be notified when you press that button and we'll be able to unmute you so you can ask your question. Alternatively, you can type your question into the zoom webinar chat box. So that we'd be able to see those questions and ask them. So I hope that is clear. And we'll move on to the main part of the webinar. Thank you all a brief introduction to E-Live Innovation. The E-Live Innovation Initiative is a separately funded effort aimed at providing funding training and communities for creative individuals, innovators and teams within the academic and technology industries. The primary outputs of the initiative are open solutions aimed at improving the discovery, sharing consumption and evaluation of research. The executable research article project is something that E-Live Innovation has been working on since 2017. It used to be known as the reproducible document stack project. Some of you may be more familiar with this name. And the project is developed in collaboration with Stensilla and Substance. The motivation for this work is that we see that while code and data are increasingly important in research, they're often left out of the main narrative of a research paper. This negatively impacts the reproducibility and reusability of the research that's published. So this drives our vision to create an executable research article, one that encapsulates usable code and data within the flow of a manuscript. We hope to deliver progressive enhancement, which means that if you are a casual reader trying to understand if you want to dive further into reading the paper, you'd be able to see a static research article and be able to browse through quite quickly. Whereas if you are a deep reader who is, you know, looking to interact further with the code and data that ability is also provided within the research article. The tools that we develop, we hope that they will be future proof. So they should be platform tool and language agnostic. They should be accessible and easy to use by everyone. And ultimately our aim is to encourage the reuse of published research. So since we've now realized that vision, I think it's easiest to show you what that sort of look like in our minds. I'm just going to switch over quickly to my other screens. So here you'll be able to see a executable article that we've recently published. This is by Lewis et al. This is published in 2018 as part of the reproducibility projects cancer biology series. Looks like almost any other article that you see normally on the elive website, except when you reach some of the articles, sorry, some of the figures. Here you see figure one B. There's blue icon on the top left. If you click that, you'll be able to see the code that is generating these two bar plots here. What I'm going to do is that I can actually modify this code in the browser. So here I'm going to make a couple of modifications. For those of you who are familiar with our, you can see that you can probably understand that I'm trying to change this code to display a dot plot instead. Now I've clicked run here, and you can see that change immediately manifesting in the figure. So this is actually something that we've demoed back in February last year. Since we published that demo, we've received a lot of positive feedback and folks coming to us and asking if they can publish their papers as reproducible executable documents. So in order to do that, we need to build a stack of tools, tools that will allow researchers to author their manuscripts with their code and data embedded tools that will allow us to host these articles and set up the reproducible execution environments that are needed to allow for those life of the executions and tools for us and any other publishers to accept and publish these executable research articles at scale. While we're moving towards these visions, we strive to develop error in accordance with these three core principles. We commit to working in the open. We're not trying to win any tools race, and the aim is to maximize our reuse of existing open technologies wherever applicable. All of the, all of the tools that we create in turn are created openly and the code base is on GitHub. We aim to communicate and update the community on our developmental progress and milestones, and to actively collect feedback and foster collaboration, which is exactly what we're doing here. The second thing is that we also understand that research and tech ecosystems evolve very quickly, and hence it is important that the tools we develop are interoperable and future proof, so that the infrastructure can be efficiently maintained and users don't have to constantly switch and learn how to use new tools. And finally, by building in the open and keeping our tools modular, we hope that other innovators in the communities can then build on top of our technological innovations. So I'm very happy to say that thanks to the hard work of the Stensida team and the elives technology product and production teams. We're now ready to work with elive authors. So you're to create and publish executable compliments to your published research. So we really want to make sure in the last year and a half we've really worked hard to make sure that the workflow for authors is as simple as they possibly can be. And that you can do this compose this error compliment with tools that you're already familiar with or working with. So, just a simple diagram before we head into a demo of the authors workflow to show you approximately what that looks like. So for very simple steps. I'll be showing you a platform post in Silla hub in a moment. That's the platform where you can take your elive article URL. Link it into a project and then convert that elive article into a arm rack down or Jupiter notebook. Depending on what to which tools that you're more familiar with. If you've done that conversion, you can download that file and add the coat chunks that shown before into the arm rack down or Jupiter notebook locally. So now your articles enriched, you can upload that back into Silla hub along with any necessary data files. And basically your set just have to create a snapshot and share that link with the elive production team. So I think a demo will speak clearer than what's so chime do this life demo now. So we'll see. Please, if in during the demo if you have any questions. Feel free to put them in the agenda. So let me just head over to Silla hub. So first thing that you need to do when you go to the Silla hub website. I'm just going back that you need to sign up for an account. So here I'm signed into my account, but it should be a pretty input intuitive to sign up for an account using your email address or or kid or Google account. There are plenty of ways to be able to do that. Once you've signed up for an account, you'd be able to see a project dashboard similar to this one. So the first thing that you need to do is to create a new project. So click this new project button, going to create a project within the eLife account today for this demo. I'll name it demo, put a date on it as well. Now click create project and projects have been created. So, as I said, you need your eLife article. So today with for this demo I'd be working with this particular article authored by James Watson and team. So you can have a look at this. Let's have a look at this article and what it looks like. So have an introduction section and the result section has two figures. My aim is to make this figure executable here. It's having access and looked at the GitHub repo. It's our markdown file that they've used. And so that's what I'm going to work with when I'm trying to make this figure executable. So taking a note of the article ID here in the URL. This is 43154. This is what I'm going to do in my new stenciler project that I would go to sources. So here on the in the user interface, you can see that it explains that sources are remote files that you can link to your project. So the eLife article in this case is the remote file to link that I'll click the new button here from journal. So remembering the article number 43154. The path is how the source will be mapped to in the project. So I can put something like article create this source. So that article has been very quickly pulled into the folder of this stenciler project already. And you can see this in the file directory. It's an XML file because that's how he likes all articles. So it's a it's a structured text format. So now I've got this article file in my folder. I can proceed to converting that into an our markdown file which I'll be doing the enrichment in. And that you point to us this three button with three dots convert to and then select our markdown. I'll click convert at this point. Just give it a couple of seconds, and it's finished. So heading back to the files menu, you'll be able to see that the our markdown file called article.rmd is generated seven seconds ago. That the conversion has been completed. I can go ahead again with this three dot button. I can click download here. And now I've saved this our markdown file locally. So let me just quickly switch over to my studio. So you could see the enrichment. So this is something that this is the file that I sort of done the previous process before. So, but this is exactly how the converted our markdown would look like. So you can see the nicely preserved metadata here regarding the author's name and their affiliation and contact details, etc. Be able to scroll past that into the introduction section, and then the results. Everything is very nicely preserved. The figure that I would like to modify is figure two, if you remember from the paper. So here you can see that that blocks here this indicates the location of figure two. I need to tell stenciler that this is not a static figure anymore, but a coat chunk. So changing that figure to chunk. And then this is the link to the original image file of the figure. So I'll go ahead and remove that. This is the figure caption title and the figure caption itself. I'll leave that. And then here is where I will insert the coat chunk. So, as I said, I've browsed through the GitHub repository of this of this paper to find the corresponding code that is needed to generate this article. But of course, if you're authoring this paper, you would have those files. You have access to those files yourselves. So, this is the code that I found. So I'm just going to actually copy that into the new file. Meanwhile, you can see that I've run this and it actually is the same as the graph that we've seen before. So the figure that we've seen before in the static version of the paper. So, once that is done, let's say you've completed your enrichment, I'll go ahead and save that. And then we will go back to the stenciler hub environment. So, let me just, now I will go ahead and upload the enriched armot down file that I've just done into this folder that we've created to some files, be able to do that locally. Open that and upload. So now the file has been uploaded. And you can see it appearing here. It's called paper dash enriched or AMD. So the next thing I would do this, this code chunk actually has no associated data file. So I didn't need to upload it. The coach run on his own. I'll create a snapshot. Okay, you might want to make it into a main file so that the project knows which ones to serve. Definitely. Sorry, forgot about that. Yeah, so to make that your main file. What you need to do is to click the dots here again and just click main file. So I'll create a new snapshot. Now called snapshot to because the last one snapshot one. So here's your preview of the executable complement to the original article. You can see that again it looks very similar to the static version, except when you reach the figure that we've now made executable. Click the eye. And the code is here. The code that we put in is here. So this takes a while, but I will give it a go to try and run this document and we'll see if it worked. So when you browse an executable article, you have to click the one document button here to be able to reserve that to start the executable environment that is, you know, powering this executable article. Just waiting for it to run the simulation so it does take a while. Just a while we were here. And we have some time and encourage you to put any questions that you may have into the question and answer section in the open agenda. And we'd be able to answer that to the end. So this is actually once you've completed this snapshot. This is the link that you'll be sharing with the production team so we only need this link to be able to then serve this surf and publish this article on our website. Hopefully that's so that's completed. Again, I hope you recognize the graph that you saw in the fact I can now go back to the original article. This is the graph that was static before. And now you have the executable version. The display is not entirely perfect here but the teams working on it to make it look a bit more similar to the one in the previous version. And again, there's still that ability to be able to edit edit this code and browser, for example, change some of these simulation parameters and the changes will be reflected in the graph real time. So I'm going to stop sharing at this point. And this is the end of my demo. And I'll now hand over to Alex to elaborate a bit more on the schematic, the schema and also the discoverability of your articles and how it relates to error. That was a great demo. And thanks for having me here. So I'll start sharing my screen and kind of talk a bit about some of the, you know, to pull back the curtain on some of the technology that's powering this executable articles, as well as, you know, the work that E-Life Innovation has a lot of this to work on. And so for us the big key thing about the era is that we see them as empowering the authors to publish discoverable content and for readers to reuse latest research for their own work, since they have access to all the underlying data under the code and all the aspects that make a manuscript reproducible. Sorry. To power that we worked on two projects. One is the schema and the other is encoder and the work hand in hand to allow us to first of all capture all the semantic information about the manuscript that has been written by the author, whether that's from the JaxxML or the R Markdown, and then be able to convert that into other formats. And you mentioned Amy about all the work being, you know, future proof and being able to be developed further. And so all of our, the underlying schema is based on things like existing standards such as schema.org and micro data and JSON LD. And what this allows us to do is when we see an article and we're trying to convert it from one format to another, we're able to identify all the semantic elements, such as you saw in the conversion that we maintain all the list of authors, their contact information and so on. And that that means that when you're going to work and convert from one format to another, we're able to translate those elements. But, you know, this process does have some draw some limitations in that we capture semantic information and not things like know what font size is, or what color something is. And so to kind of demonstrate a bit what that looks like going from R Markdown to HTML is that you can see that we have a title we have some text and code chunks. And we were able to embed all this information inside the HTML using the micro data format. Sorry, micro data attributes. So we can see that this is article element conforms to the schema or article, and then we have a headline, and we have a person here who's an author and so on. And the reason I think this is exciting for us is that it allows us to publish naturally machine readable and discoverable articles. Google is one big search engine, but also other search engines and other programs can parse this data and improve discoverability of the articles. Besides the published journal. So when people are searching for other articles to reference in their own work, or just getting more organic exposure. This is a really great and easy and standard compliant way of doing so. And the other thing is, as we said that the schema org is very expensive. And so the schema that we've developed is very focused for for research manuscripts and has things like the code chunks and references and citations and so on. And we're seeing some uptake of the schema by other companies, such as high review, which is I think a peer reviewed platform that you like has also partnered with. And lastly, just before I start to wrap up is that HTML is just one of the formats that we support converting to and you saw me convert to our markdown. But we can also convert to PDFs Google Docs Microsoft Word, and any other formats that the community has a need for. And the thing that we think differentiates from other existing solutions is that we have methods of embedding and maintaining through a pretty simple aspect so even when you convert to a PDF. We embed the code elements and the source code inside the document. So that when you then try to go back to your own markdown or the source format. That information is still preserved. And I think that's really exciting for distribution of reproducible articles. But then, as we're doing user interviews earlier on last year, we found that this workflow was really convenient for gathering feedback from supervisors and managers, you know, people who might want to comment on that copy, or the text of a manuscript, and they want to read it in the Microsoft Word document. It's easy to convert your source document, such as your notebook or our markdown into word, send it to them, they can make textual changes, and then send it back to you and you can convert back to our markdown and have that preserve the text changes while preserving the code as well. So you don't have to copy paste across documents and which I think everyone knows can be very error prone. So I think ultimately, for us, the open and reproducible research is critical for compressing the time between doing research and then seeing the impact that it has in the field. And it's, you know, it's especially, I think, critical for the papers that you demonstrated just now, I mean, where they include simulations that cannot be captured in the static versions. So either just static HTML or static print ones, where each iteration can lead to different results and different intuitions. So that's where I think the reproducible articles that can be executed can have the most impact. I think that's it for me as well. If anyone has any questions about any of the process about how we're handling some of the conversions or the executable aspects, please feel free to ask away. And thank you again, Amy. Thank you very much Alex. So I made a note here to talk about our feature direction. So we're, we've recently launched, of course, so we're very excited and we're keeping a close eye on sort of the, the, the, the, the traffic that we're getting to the articles and also looking at how people have been interacting with those articles and and how they've been playing around the code. In the, in the short foreseeable feature, I'd say we are focusing Eli is primarily focusing on really getting our office on board so we really, really need your help and support here to compose areas with with us and so that we can publish them. And we are, we have a lot to learn here and say, we, the main reason why we'd like to work with all of you is because we, the more articles we publish and put bring through this process the more we'd be able to, you know iron out the process and be able to see how the tool stack can cope and handle different types of data, different types of workflows and different types of tools that get integrated into into the articles. And so this is very important lesson for us in terms of being able to scale this up further and to improve the performance of the tool stack as well. In the longer run. Yeah, we were really interested in sort of making this towards a point where, you know, folks can come to us and be able to submit articles, no longer in the PDF or Word format, but that's of course quite a long journey ahead. So I'd say that's sort of a longer end point that we hope to reach. But in the meantime, we're also interested to look at how this error tools that can work better together with existing tools that researchers are using different research domains within the life sciences. Again, different sets of data and perhaps data repositories that you're using tools that you're using to analyze your data as well, and to really improve focus on improving the interoperability of the tools and to be able to make them work well with researchers in different fields of research that you live is publishing in. Alex. Maybe if you can highlight a bit sort of where you see some sort of going with this. I think for us. Yes, you know, we've come a long way but we still have what to do in terms of improving the workflows and improving the, you know, just how much variance we can handle in the manuscripts and the conversion aspects. And the process of getting an author being able to share their project as fast as possible in a way that can be consumed by readers. I think we're, we have a process for that. And the thing that I think that we really want to refine on is the just is how how the integration with the published with the journals works. And so, being able to have a good, good workflow from the, the start of the authoring process to how it gets published and consumed by the masses. We're trying to maintain their reproducibility and the, you know, all the underlying data and the code is key. And right now there's that manual step of where the author gets to hand it off to you, but I think there's improvements that can be made in that process to fantastic. Yeah, just, just to sort of end this section, just to bring your attention back to sort of the immediate calls action, let's say. Yeah, you have this link on the screen right now that will lead you to the original launch article that we've published. I'm an eLife author and you're interested in publishing an era with us. Please fill out this form that you can reach by the button here to click this and you get to a form where you'll be able to give indicate your interest and we'll follow up with that as soon as we possibly can and bring you through the process. Here are contact details. If you have any further questions, we do have a bi-monthly, sorry, every two month newsletter that will bring you the latest updates and developments on the tool stack. So if you're interested in keeping up with our latest development, please do sign up for it using this URL right here. And of course, our websites, Twitter accounts and email addresses were always interested and would love to hear your thoughts. So thank you very much. At this point we'll move on to the questions. So thank you very much for all of your questions in the agenda doc and we'll try and get through as many as we possibly can in the next sort of 20 minutes or so. All right, I'll take it from the top, I believe. I actually just, so yeah, just a reminder, you could also use the raise your hand button in the Zoom webinar control panel, and we'll be able to see that and then mute you as well if you do want to ask any questions verbally. And also do put them in the chat box in the Zoom so that we could see them and be able to address them as well. So yeah, I'll take it from the top. We have a questions on open source tools. So I mentioned that so stack is entirely open source and the code base on GitHub. So they do use, at least from Eli, we do use open source license. Alex. So I'll put the link to the GitHub repos, sorry, not repos, but the organization pages in the agenda so you can access it there and have a look. We do really encourage contributions from everyone, if you can. So there are many ways that you can do this, you can, you know, if it's something that, you know, you like this, there's a bug or anything. If you're familiar with GitHub issues, that would be super helpful. So if you have organizational collaborations, do email us and I think we can, that's sort of the best way to move forward for us. Alex, maybe on the social side, maybe do you have anything to add? Yeah, I think for us it's also getting our tools exposed to as many different inputs as possible. So different structured documents, different file formats, and just identifying any edge cases that might exist. And that is extremely useful for us. And so even if they aren't technical contributions in terms of, you know, code pull requests, just bug reports and things like that are equally helpful for us. Thank you. I do have a raise hand. Michael von Papen. Sorry, I'm going to get the pronunciation wrong. I'm going to unmute you and you can ask your questions directly. Michael, where you can speak now. Yeah. Hi. Yeah, so thank you very much for this great endeavor that you're going to make papers more interactive and reproducible. I really like that a lot. And I think that will benefit a lot of science actually. But it's a very large endeavor, I think. So I myself, I'm working for Fast Genomics where we are trying not only to provide interactive publications, but basically the complete data analysis pipeline. And I wonder if you're also aiming for long term to go in that direction, maybe to make the complete project reproducible. Yeah, thank you very much for the question. We, yeah, I wouldn't say we've actually made a decision on that. We definitely interested at the same time, you know, we want to, at least I speak for you for your idea that we, you know, we are a journal. And so we're focusing on sort of the consumption of that research. So the complete reproducibility would mean be able to take raw data and see all the workflows of pipelines and code that take that raw data all the way to the published figure. It'd be great to be able to do that, but we also are very keen to keep in mind sort of what users want when they read the paper and how they will potentially interact with the paper. So I do, you know, if there is, if we see our readers really wanting that reproducibility and execute and want to interact with those workflows within the browser on the website, we'd love to be able to explore this. I mean, we'd love to be able to explore this anyway. And therefore for us to, you know, be able to prioritize that. So we definitely, as you said, it's a large endeavor we need to prioritize our next steps and to be able to figure out what would most benefit our readers and our authors. So to be able to do that, we would be probably conducting, you know, some some research and we definitely want to hear from, from, you know, our communities to see how and what they'd like to see in the near future. I hope that makes sense. Yeah, so if I can just shortly follow up. So our users, they are working in this reproducible environment that we are providing. So for them, it would be interesting to go the other direction, not from the paper to the Jupiter notebook, but from the Jupiter notebook to the paper. Do you also provide something to do that? Alex, do you want to jump in here and see. Sure. I think the workflow that Emmy demonstrated is, I think the first step in that they're taking the already reviewed articles and converting them into executable ones. There's no reason why you couldn't go straight from the Jupiter notebook to an article. And you know, like, that's just the second half of the process that Emmy demonstrated. So to kind of, if I may backtrack a bit about the reproducibility of the whole project. I think that is definitely something that's been so is looking at in that right now we have the older the era articles that get published, they run inside containers that are that can be specific for the project. And we were trying to simplify the process for authors to generate the reproducible containers with all the dependencies and things like that. So, I think there is a trajectory where we are aiming to provide reproducibility for the complete project. Yeah, thanks very much. Thank you very much. Before I allow the next hand that is raised, I want to jump to one of the ones that we have on the agenda. So the scenario at the moment replace a research article that has been made executable with an error and article with a separate do I at the moment it is a compliment to the published article doesn't have a do I so do I is with the original article. So at the moment in error is not siteable but if you would, you know, like that to change please let us know. We see how what we can do about it in the future. Thanks. Do you have anything to add on that front. No, but providing, I think, providing the eyes for the snapshots, which just going to expand on our immutable version copies of the entire project. So when me click the snapshot button, we download and capture the files as they are at that point in time. So yeah, sorry, we are looking at providing the eyes for those snapshots but it's, it's again, something that's on roadmap, but we need more user input. Sounds great. Thanks, Alex. Now move to we see a raise hand from a shallon so I'll I'll now and meet you and allow you to talk. Hi. Yeah, this is great initiative. And I think the, the demand is, is very high. A specific question about image analysis pipelines and I think partially got answered by Alex that if if the if your analysis pipeline has custom tools for image analysis. It doesn't seem that the current workflow will support the support making that type of article and executive article will have to wait. Will we have to wait till the Docker, the feature of using the Docker containers in your in your article is available. Or yes, we shot the question is if, if the article has significant amount of custom code at this point, let's have our own GitHub code. Can we turn that type of article into any array, or should we need to wait till the Docker functionality is available. I can answer this and right now I think it's our own immediate role plan so in the very near future we're trying to introduce support for providing a custom Docker file, which you can state all the dependencies and things that you need. Manually, and I don't think that should take us very long. It's just something that we haven't exposed in the user interface or the product yet. But if you would like to spin up to stencil, we are looking for early adopter projects to work and collaborate with on testing these features. It's a good fit for us to collaborate on together on this, if you'd like. It sounds excellent. That's great. And I have one follow up question to the collaboration on the topic of collaboration. The repositories you have. Do you anticipate other institutions other than stenciler contributing significantly to this project. And I'm thinking of, so I work for Chan Zuckerberg Biohub and Chan Zuckerberg Initiative, for example, supports bio archive. And Allen Institute and other institutes are interested in, in, you know, advancing the pace of publication, reproducibility of research and exposing the data. What's your philosophy behind the collaboration. Are you looking for collaboration initially only from authors or are also tool builders or in terms of timing. When should the authors be in touch with you when should the tool builders be in touch with you. Thanks for the question. I can, I can answer the first part of this. We're, we're open for collaboration now and honestly this is a good time for technologists as well as authors to get in touch and let us know what you think. We're actively trying to map out sort of what to do next. So, whether you have, you know, working, you are working on a tool that could be potentially compatible and interoperable with with error and stenciler. Just let us know send us an email and we'd be happy to follow up on that. Thank you. Alex anything to it. No, I don't think so. I accept that we are also very open to contributions and engaging dialogue with future directions and, you know, partnering with people who are interested in expanding this work. Yeah, please reach out. Thank you. Thank you. Okay, I'm going to go ahead and answer a couple more questions off the agenda. So, there was released a question to the DIY question where how should authors initiate a dialogue for error after the article is published or during the initial submission or either mode. I'd say in the initial submission, you could indicate that you are interested in having an error component to your published article when it's eventually accepted and published that wouldn't influence, you know, any of the editorial processes. So the the peer review and production is independent from the error publishing process. But equally, if you already have a published article doesn't matter how old it is, as long as it's on the life. We'd love to hear from you if you are interested in publishing in there. And it makes question. It's on MATLAB. So, there are many scientists using MATLAB for data analysis and open source toolboxes are built on MATLAB. And similar question is whether the error will be compatible with any proprietary data analytics software. Do you want to have a go, Alex? I don't have a specific for MATLAB since I don't have too much experience with it or familiarity with it. I will say that if it can be, you know, installed on a Docker image and create an environment for it, then we should be able to support it. But I think that's as far as my knowledge on this matter goes. Yeah, I'd say at the moment, unfortunately, we don't support it for now. We prioritize integration and interpretability with open source language and tools. So, but if there is, yeah, we're not quite sure how to work around it yet, to be honest, if it's a proprietary stack. If there is a demand, we are happy to have a conversation, maybe look into it. Can't promise anything at this point, but they definitely do let us know if that's something that you'd like to see. Okay. More questions on the agenda. Let's say in a future where most articles are executable, what do you envision a research communications world to look like and what directions with error lead to. Thank you for the question. Yeah, I mean, this is, yeah, again, me personally, I guess. I yeah we at the life we'd like to see that era really promotes reuse of research published research that people because readers could gain further understanding into the research that's published the method that's been used, the data that's been used in the research, they'd be, you know, more easily, I'd say, and definitely sort of quicker. They, they'd be able to reuse other people's methods more easily as well. And that's exactly what we want to see, we wanted to see, you know, less reinventing the wheel per se but more sort of building on top of each other's work meaningfully and giving credit to others as well. So I think that's that's sort of where we'd like to see it. I'm aware of more sort of more ambitious endeavors in terms of collaborative code editing and that sort of thing and I think, you know, that's really like again being interested in innovation and research communication I'd like to see that happening in the future. Alex. Yeah, I think it's many of the same aspects, you know, it's like you can't really have a whole conversation if you're only seeing half of the picture right and, and just get getting exposure and shining light on the underlying the works that power the insights and the research is I think what what is exactly for me personally. Awesome. Thank you for asking that. Next question. There seems to be a huge skill ability challenge if many articles are published this way it will require quite significant amount of computing power. I'm guessing the solution necessary will be both will have both a technical and editorial site. Could you share thoughts you already have on this please, even though I guess it's a work in progress. Yeah, thank you. I think we definitely have thought about this. In fact, it's a very nice problem to have if we have a lot of people browsing the articles and using it and running it. I think that's that's the problem that we'd like to have at this point. It is definitely, especially on the editorial side, definitely a challenge to be able to process and produce that many articles. Yeah, we're exploring. I think, I think Alex you may be able to elaborate on this a bit better. Yeah, we're definitely interested in sort of collaborations looking at how we can just like so you can you can theoretically take the tool stack that we have ready and run your own instance of sense of the hub and then use your own computer sources right so it doesn't have to go through elive so I think that's something that we are interested in and maybe Yeah, so as you said, you know, like all our work is open source so there's nothing really stopping you from going and spinning up their own stencil hub and compute resources for us. Everything is done in a scalable manner so you know whatever resources that the cloud computing giants of you know, Amazon and Google and them they have to offer I think the financial constraints are the only limiting resource for us to be able to handle the load. But as you said, you know, that's a good problem for us to have and it's something that we can but on a purely technical level I think we should be able to address it or we're already in a place to handle it. Yeah. I think there's a sorry just to wrap up I think there's a that I think for me the more interesting question is, what is the tutorial and the review process look like when there's extra information such as, you know, the technical competency needed to review the source code and all these other aspects. Yeah, definitely. I cannot agree more. I think I think that would be sort of the next sort of interesting challenge within this this bigger roadmap and one that would definitely be very interested to solve. Thank you for for the question. Hope we've answered it. Just scrolling down the agenda. There's question, who can post error articles does your article need to be published before hand in another magazine or are you prime the primary source to publish. Are the authors the only one authorized to make the error or can anyone that knows how to make the code publish it. That's a really, really good question. So at the moment, I think us elive is the sort of the only one who is offering the service via sensor. So the error surface so you do have to be an elive author at the moment to be able to publish an error on the elive website. If that makes sense. Um, you can just use them to help to enrich any articles that you like and share the snapshot with anyone that you like. I think that's a short answer to to the question where, whether anyone is able to produce an error. So the answer is yes. It wouldn't end up on the elive website unless probably when we have consent consent from the authors, the original authors but nevertheless you're welcome to take articles and make them executable and share this snapshot with everyone. And if you do do that, do share it with us. We'd like to know. Alex. Yeah, I think just as a bit of a caveat. While we're in this early stages, we have limited the onboarding. So when you do sign up, you will be placed on the waiting list. And that we we progress. We periodically invite users to be able to create projects and make their own era articles. But for now, the hub is only, you know, you're only able to make full use of the hub if you're invited as a collaborator on an existing project where you can add documents and do other things. If you if you get invited to the full, full featured, I guess, here. But, you know, as I said, please don't let this put you off on signing up because we are adding people over time. And if you if you think you have a great project that would be a good fit for for these sorts of art, the source of, you know, papers, please sign up and send us a message using either the intercom button on the bottom right of the hub or an email and we can start, you know, we can upgrade the accounts fast. Awesome. Thank you. We have two minutes. We have one last question. Since we're sure readers are viewing these on the web, do we have plans to allow interactive figures, for example, plotly or JavaScript D3. I'll let Alex answer this. Yes, yeah. So we, we had early support for JavaScript, but it's been removed temporarily. And while we focus on our and Python. But I think, as you said, you know, like the, the interactivity for us is just a one way of developing intuition for the underlying concepts and providing the more interactive widgets and you know, user interfaces for for developing that intuition is something that is very important to me, at least. And we are working on, we have, you know, we've had conversations about developing custom interfaces for, you know, domain specific areas such as, you know, ability to explore proteins or different types of enzymes and things like that. So, yeah, it's it's something that is very, very interesting. It's very, very interesting to us, but nothing, no immediate plans just yet. Thank you. All right, we're coming to the end of the hour. Thank you very much once again for all of you to all of you who are joining us and for all your wonderful questions. If you have any further questions, please don't hesitate to reach out to us by any of these channels will more than happy to hear your thoughts and your ideas and your feedback on the work that we're doing. Thank you very much once again for joining us and thank you Alex and Anja and hope to see you all soon. Thanks for running this Emmy and thanks for everyone for joining. Thank you. Bye.