 It's not Melissa and nice of you to ask everyone. It's nice to see everybody streaming in it's about 1229 I'm going to leave it maybe till about 1231 give everybody about two more minutes to come in and then we'll get started with the short course in our short course and we are ready to record. Alright, so we are recording this session. If you do not wish to be recorded please go ahead and turn off your, your camera right now that's totally fine. I have put a link for the day over in the chat. So it's our three dash our medicine dot net li-fi dot app and I'm going to go ahead and share my screen so you can see what that website looks like. So this is basically the collection of all the links that you'll need throughout our time together today. If you go to start, you'll see links to the two slides up top. You'll see also some kind of bookmarks over here which are some links that you'll need throughout. One thing I want to call your attention to early is this little cloud icon we are going to be using our studio cloud for this workshop. So if you could go ahead and click on that it should take you to the our studio cloud space and I'm going to demo how that looks for you I'm signed in with my current admin account so you will see something slightly different. I'll demo that in a moment but that's on the website as well so go ahead and make sure that you have an our studio cloud account if you don't have one you can create one. I recommend using a GitHub authentication or another Google authentication so that you don't have to create a new account. And I'm going to go ahead and get started then. And again we are recording so alright so we're going to start with the basics. I want you to go here and log in to our studio cloud. So the short link is rstd.io forward slash r3 dash cloud. And now that I've moved on from that slide it is going to be at the footer of all of the slides. So you should always be able to see that our studio cloud link and it's also on our website as well so be sure to sign up for our studio cloud. Go ahead and get in that space and before we get really started I'll make sure that you guys all have accounts. So short introduction. Our team today. My name is Allison Hill and I'm product manager for data science communication at our studio. And I have taught at our medicine I think the past three years is that right Stefan. Yeah. So I really love coming here I'm actually a previous medical researcher my former job was assistant and associate professor at Oregon Health and Science University so I'm excited to speak to you all today. And to share something kind of new it might be a little bit of a surprise to everyone but we're really excited we have a team of our studio people here to support us. But our instructor team is myself, Stefan Kedalki and Paul. Bill and the waiver. I'll let each of them maybe introduce themselves really quickly and then we'll be passing it back and forth between the three of us Stefan do you want to do a brief intro for you. Yeah, sure. Thank you so much, Alison. I also can't tell you guys how excited I am that we were able to get Alison back to to teach again at our medicine. I was going to say for the third time in a row and she told that thunder for me so I'm an assistant professor of lab medicine at the Children's Hospital of Philadelphia, and and I'm a I'm a very heavy our user. And I'm the chair of this conference. So, so I'm very happy that I was able to get that I was able to help out with this workshop and thank you so much for for joining today. Paul is my intern this summer, Paul. Yeah, like Alison said I'm our intern at ours at our studio this summer working on the our markdown team. Before this, oh, still am I'm a PhD student at Iowa State University, studying the impact of human activity on the environment so I use a lot of our base a lot of reproducible reproducible research is something very important in our work and very excited to share some stuff with you today. So I'm really excited that stuff in and Paul could both join me today and we also have I understand a pretty strong our studio team behind us as well to help out. I'm going to encourage the our studio people here to to speak up and feel free to talk I know we have Tom mock joining us and I believe Charles Teague from the development team so I want to welcome everybody and we'll be sharing some some newer tools with you all today so we're pretty excited about it. I think the tools that we're going to share with you today are things that I don't actually believe are going to help solve science or business reproducibility crisis. I believe having been a former scientist I believe there's a lot of deep seated problems that can stem into the reproducibility crisis. But what I do think the tool chain that we're going to teach you today can help with is something that at least I experienced as a researcher and I think a lot of my colleagues did as well which was the confidence crisis. So, I used to obsessively check my data and be worried that the data that I was presenting or the data that I was publishing or the data that I was trying to get to use for getting funding. It wasn't the exact accurate data or there have been some changes or you know I had various pipelines of scripts and wrangling code that I was using, and always trying to make sure that I was using the most up to date data, and I had used the most up to date script and that the figure that I was showing was the one that was tied to the data that I thought it was. So, I used to spend a lot of time doing this until I discovered how to use our markdown effectively and so we're going to be showing you some tools today that I think will actually help with that crisis it also has the benefit of making your work more reproducible. I also think it helps with what I think of as the cheese crisis. So, this is actually probably like a very mild version of what my schedule as an academic researcher used to look like. The blocks were pretty much covering my, my, my day, and I had very few little white blocks within which I could actually code and I could go into a project and I could, you know, revisit where I was at and get up to speed quickly and be able to, you know, be able to leave myself enough breadcrumbs so that the next time I knew I would have to do that, I would be able to jump right back in and be effective and do smart things with my data instead of going around in a hamster wheel. And using our markdown also helped me with that because I was often on several grants at a time this might sound familiar to you several different research projects and within a given week you might be flipping and flopping between various research projects and they might be very similar but have just slight differences so it's very easy to start feeling like you don't know exactly where you're at. And I think the tools that we're going to show you today will also help with what I think of as the cheese crisis. So I'm going to take changing your mental model a little bit. So, you know, the image on the left is the typical blank word document that you get when you open up word for the first time. And for some reason, you know, while blank documents often create sort of fear in us when we're programming. This kind of document doesn't scare you as much as it looks very friendly there's the source is the same as the output essentially so as you're writing you're actually seeing what you're getting so it's kind of the ultimate like what you see is what you see in the editor, whereas using source code to produce outputs can feel a little bit different you have a source code that is stored and a file and you then do another step to be able to create the output. You might often have these these twins, twin files living next to each other, one that's the source file and one that's the output file so you might be rendering to HTML for example if you're lucky and you're able to use HTML. You might be rendering to even word or PowerPoint formats. So it's often a little bit of a mind switch to change your mental model from thinking about one document that is the sole source of everything and the thing that you end up sharing with people. And then splitting those up mentally and moving on to being able to think about I've got this source file and I have this output file. And I want that link between the two of them to always stay there because that link is my reproducibility so that allows me to be able to say I know that the source that I'm using is creating the output that I'm seeing. So this is a preview of what we're going to be going through today in our schedule. So session one what we're in right now is going to go for about 50 minutes total until 120. And we're going to be covering just some basics. We'll take 10 minute breaks every 50 minutes so from 120 to 130 my time Pacific time whatever time zone you're on from 20 after the hour to 30 after the hour, we'll be taking a break, and then we'll start again we'll have another chunk, 10 minute break, another 50 minute chunk and 10 minute break and then we're going to wrap up with a 30 minute Q&A and a little bit of a wrap up also. The format today is going to be some live demos. We also have some slide decks that will present, excuse me present some information. And also have interest first in there, some solo activities called your turns and those are going to be activities where you do jump into our studio cloud and work with a document, try rendering things work with code chunks. So you're going to be using our studio cloud during those your turns. And in those your turns I usually set a timer that's not to stress you out that's to help pace you so that you see how much time we're going to be spending on a certain activity. So the whole idea is that, if you want to you can opt out, you can do as much as you want or as little as you want you can sit back and relax and just watch this whole workshop or short course. It's up to you and sensitivity to people's different situations during the pandemic we decided to not have group activities during this workshop. So you're going to be working on your own during those solo activities. We're going to be free we know that you are often taking time away from work and other family obligations children at home other family members at home to attend this kind of short course so we want you to feel comfortable keeping your video off, and you know, needing to multitask, as we all are doing right now. So on or off it's up to you I'm going to ask my co presenters Paul and Steven to keep their videos on so that I can see some people, but everyone else you're free to turn it off if you'd like. And so it's really kind of a choose your own adventure style do what you want we hope you get something out of it. And actually now that I say that we're all going to get something out of that. I can guarantee you that every single person in this workshop is going to learn something that you didn't know before and that includes me. And that includes probably stuff and I'm Paul as well. I'm going to do a live demo right now of just logging into our studio cloud so that everybody can see what that looks like. So I'm going to take this link here the R stood.io R3 cloud. And I'm going to open that up in a different browser window that has a different account open for me. R stood.io R3 cloud. And so you should get a screen that looks like this that says join the space. I think we're on a different screen than you sorry. Oh, thank you. Okay, I'm going to stop to share thank you very much. And please do interrupt when you see that. I'm just going to share my hold on screen and then hope that I've turned off all the notifications. Is that sharing screen. No, let's try it one more time that sharing screen. Yes. Okay. All right, so this is what it should look like when you follow that first link the R stood.io forward slash R3 cloud. You should get this question join space. I want you to click join space. Welcome to R3. So this is our workspace in our studio cloud for this workshop. And if you go to projects up in the upper tab, you'll see the two projects that we'll be working from today. So we have 01 basics and 02 authoring up here for now. And those are assignments that when you go into them, you'll enter the project takes a little bit to open up. It's deploying it's doing a lot of really fancy stuff under the hood. And then when it opens, you'll be able to see essentially the R studio desktop IDE experience in a browser window. Okay, so my project is now open. And you can see that I have console over here. I have a file pane over here where it shows you the files that I've added for you already so there's 01 dash markdown 01 dash explore. And when we get to those will tell you which ones to open and which ones you need. But for now, I want to make sure that you all can enter the space, and you can access the projects, and that you can open up 01 dash basics. So if you're struggling, I can't answer any questions right now, because I've got a lot of things going on on my screen I'm trying to limit distractions. But Paul, Tom, and Stefan can all help if you throw it in the chat if you're having issues, please throw it in there and they can try to help you problem solve a little bit. I'm going to stop sharing on this screen because for some reason it won't let me share my full screen so I'm going to go back and go back to my slides. Okay. So that was getting started with our studio cloud. So, what I would like for you to do is to hop into that 01 basics project. Open up 01 dash explore dot QMD. And I want you to take a few minutes to try to identify these parts in the source code. So there's some metadata. There's some text, and there's some code. And if this was easy you can try to find all the code that produces a plot. So I'm going to set the timer for two minutes and let's take a bit to explore that 01 explore dot QMD file in the one basics project. Okay, two minutes is up. Can everybody see my cloud screen right now or can you only see my slide slide. Just a slide. Okay, I cannot figure out how to share the whole screen on this one only gives me the option to share a whiteboard. I'm going to switch slides again then so I'm sorry I can't share my whole screen for some reason. Okay, so this is the file that I asked you to explore and I already see the questions percolating and I know exactly what you're thinking. So I would love to know, I can see everybody in the participants view. And if you guys have used zoom before and you know how to do reactions if you have that ability you can give thumbs ups. I'm wondering if people can give me a thumbs up for if you thought I was going to be talking about our markdown today. I see a lot of thumb steps. Okay. So that would be a totally natural assumption and we actually had this labeled I think our markdown for reproducible research at first. We've actually changed our plans we are teaching a new tool to the new to the our studio. We're going to be using a tool called Corto. And so what we're going to be doing is teaching you guys for the first time. This is the first cohort of people who are going to be learning this new tool. And it's essentially going to be a successor to the our markdown ecosystem. And hopefully at the end of this workshop, you'll see why we're so excited about it and why we're excited to demo it for you today. The QMD file is different from an RMD file in that a QMD file is a Corto markdown file. So dot QMD is the extension. But if even if you were expecting an our markdown workshop today I'm hoping that what you saw when you looked at this file was something very familiar you saw some metadata at the top in the YAML right we're familiar with the three dashes if you're not familiar with YAML. So you're having keys and values as metadata for your document. Then you see some text here. And so this is a note that Paul wrote to myself and Stefan as we were prepping for this workshop. And then the parts in gray are code chunks. Those might look familiar to you this is the way that we interact with our inside a QMD file. You can see some other kind of fancy things like there's a hashtag in front of some words that indicates a header we're going to talk about all of this today. But if you're used to our markdown this should look very familiar to you. So I'm going to have to stop sharing my screen and get back to my slide. So we just looked at the source anatomy for 01 explore QMD together so we saw that there's metadata there's text there's code and we're going to go through all of those elements in this first section. So I know that many of you were thinking what am I looking at a QMD file is going to be the file format that we're going to use today to interact with Corto. We are using Corto in this workshop. Corto is a tool that you've probably never heard of. And that is for good reason we're all actually beginners here I'm a new Corto user as well as Tom as well as Paul as well as Stefan. Corto is a very new tool and we're all getting used to it. So you're not alone. You're probably the first ones to hear about it. So here is the documentation site for Corto it's at Corto.org and we'll be sharing these links throughout the workshop. The Corto.org website is actually pretty thorough right now it has a lot of great information in it but what I want to cue your attention to is this link up at the top called about. So this gives you a little bit more of a description about Corto. It's a scientific and technical publishing system built on Pandak. And the overarching goal of Corto is to make the process of creating and collaborating on scientific and technical documents dramatically better. And that's exactly why we thought this would be a great tool as a fit for this workshop. So it was a change of plans and we're really excited about it but this is also the first time that Corto has been taught. So that's why I have a number of our studio people here to help out. So this is Corto I just mentioned that it's a technical and publishing system built off Pandak and if you're familiar with our markdown this might be familiar. A lot of the our markdown output formats that you have access to when you use the our markdown package or an extension package are based off of Pandak which is as kind of illustrated here sort of a Swiss Army knife of a document converter so it allows you to take Markdown as input and export to a number of different file formats you can export to PDF Word documents and point presentations and HTML for example. So what is Corto exactly though? It is the command line interface so it's installed at your terminal actually it's not an R page. And that when you leave here today if you'd like to install it locally there are some instructions on the quarter dot org website for installing it. In our studio cloud instance we have actually installed it on the back end for you so you'll be using Corto in our studio cloud with the RStudio IDE. So the RStudio IDE has support for Corto documents with the daily version build. So if you've never used a daily version build there's a way to download daily versions of the RStudio IDE that are fairly stable. And so you can use that to interact with Corto locally as well so come back to the slide if you end up wanting to go further with Corto and you want to get out of that RStudio cloud place that I've put you in. But also Corto because it's not an R package that means that for all the different output formats if you're used to the R markdown ecosystem you might be used to being a little bit overwhelmed by the number of packages and figuring out which package goes along with which output formats and how you can create X if you want to be able to just do that. What is the package that helps me do that and then now I have to learn a whole new maybe API for doing it and oh now tab sets don't work, or now cross referencing only works with this output format. You might have experienced this pain if you're an R markdown user and that's some of the pain that Corto is designed to take away from. Before, when we have our markdown that's a great tool for people who use R, but if you're a person who's on a team that uses Python. This was often kind of leaving the pythons out in the cold. They were all alone they didn't have literate programming it didn't have the easy ability to be able to render documents that had their figures and tables and all their code in it that could actually be version controlled and shared very quickly and easily. So it was for our users and it was built into the R studio ID Eve. But now with Corto we can have these two play together so we can actually have Python and our code and so the question of what's a QMD file, a QMD file is a Corto mark, markdown file that allows you to have both are and Python chunks in it. If you wanted to use a dot RMD file, that actual file extension with Corto, you could but you'd only be able to use our chunks. Also, the ability to use ipython notebooks as well. So how does it work. It works very similar to how our markdown works and that you have good ideas code and data, and that's what you're coming into this process with. And you want to put them in something and so you can put them in a number of different formats you can put them into an ipython notebook, you can put them into a QMD file, or you can put them into an RMD file. So you pour all of that work into it and then either Jupiter and Nidar takes the baton from there and deals with your code and inserts it into a plain markdown document. So that means that by the time it goes through that cycle to the dot MD file, it's sort of deactivated there's no executable code in there anymore the code has been executed and any output that's been created plots or statistical analyses that have been run anything like that. That's all been done. And you're seeing the output in the actual MD file but there's nothing executable after that. After that MD file is created then Pandoc takes the baton and says okay what do we want to make this markdown file into. And then you can render to HTML PDF or Word doc for example or any of the other three plus. And you don't need to install anything additional. There's no national our packages that you need you're just able to use all of those and you're able to use all of the core functionality that we're going to show you with Corto, and all the output format format. So let's take a first look at Corto. I want us to all render together. So I am going to unfortunately have to stop my share screen and reshare my screen and go back to my studio cloud instance. Can you see my cloud Paul I can see you can you thumbs up me can you see my cloud. Okay. All right, so if you go over here you might be used to seeing a knit button but because we're using a dot QMD file and because we have a partner YAML that we're going to explain in a little bit called format you might be used to output if you're an R markdown user because we have that. The ID actually automatically detects that we're in a Corto file and it gives you now a render button so I want everybody to click on the render button. And you should see a job startup like I'm seeing. You can see it's processing the file this is actually the knit our portion. Then this is all the knit our portion so you can see that it's actually kind of going through the chunks and it's executing the R code. And then you can see right below that after it exports o one explorer dot knit dot md. Then you start seeing what Corto actually produces, and this is output from Corto so you can see that it's calling pandak, and we're rendering to HTML from Markdown. The output file is going to be named o one explorer dot HTML. And you can see that there's some additional add on sitting here that comes sort of by default. I'm reading in some metadata that I have in my ammo here. And then you can see it created the output and what popped up in my viewer is actually that output. So you can move your pains around a little bit if you want to see this more clearly. In the viewer pane you should be able to see a rendered HTML file that is this document so you can see that it has a title it has pulled name and a date. It has the text. And you can see that the code blocks that we saw over here. And gray are also in gray over here in the rendered section, but you can see that there's now this addition of output so for example loading tidy verse outputs a bunch of junk. We'll talk about how to clean that up. There's also actual output that you want so for example this code chunk here that says glimpse mock data. You can see that code chunk right here and then it actually has some output that you can see which is helpful for when you want to share the actual results with somebody else. In this section there's some more our output and I believe there's a plot at the end. There's a very, very simple plot at the very end. Two plots. There we go. So that is a simple corto document and how to render it to HTML I'm going to stop my share. And back to the slide. Alright, so that's the basic anatomy we talked about metadata briefly text code and output they're all in there in the output and the output file that you can see. And there's basically a one to one correspondence between those things in the source file and in the output file the only addition is the output all of a sudden. So what we're going to organize this workshop is I have a series of QMD documents that you're going to run through and they're each sort of thinking of like the the letters that I used to write to my collaborators and research assistants when I was working on research projects. So we're kind of using dear data as an inspiration to have these be sort of the more in between products from a research study that you'll need to use typically, instead of thinking about the final final research paper as a thing that we're working on. So if you think of the sort of the address the as the metadata, then you've added text to it. You also have plots, you have tables we're going to talk about all of these elements in our QMD files. So what's inside. We're going to start with the text, and I want Paul to take the screen from here and he's going to do a live demo of the core to a visual editor. All right, thank you Alison. I'm going to take the whole screen here. No option I accidentally clicked on whiteboard once, but I don't see it. All right, so we'll start off with the slides. Can you guys see my screen. I see slides. Okay, cool awesome. Yeah so today, because Cordo is based off of Markdown, we're going to have to, we're going to take some time to make sure that everyone's familiar with using Markdown to write documents together. So to take that we're going to talk about the arm our studio visual editor. So in later editions of our studio you might have noticed this little draft button in the upper right hand corner. This transforms the document from the raw text format to rendered Markdown output. So I'm going to switch my screen over to the back to the cloud. Okay, and then I'm going to close and go back to files. Oh one dash Markdown dot QMD. And I already clicked it, but so right now if you're in raw text mode, you should see something like this and oh one Markdown dot QMD. I'm sorry. Is anybody else seeing just a blank black screen right now. Yeah, it's black. How about now. There. Yes. I have two monitors and it keeps jumping around but okay anyway so we're back here. So we have the one Markdown dot QMD file open and right now it's just plain text but if we click this draft button on the right hand side. You're going to be able to transform it into the visual editor mode. And what the visual editor mode is doing it's rendering the text as from Markdown to the actual formatted output. So today we're going to go through. So in the next couple of minutes we're going to go through this document together and make sure that we're all comfortable. One, using the visual editor and two writing stuff with Markdown. So this first line here I put a link here that you can follow, just to have like a little cheat sheet handy on the side so you can refer to it later if you want to do some, if you forget how to do stuff and Markdown. So I'm going to go ahead and close this window out. So if you want to make a header and Markdown you could pre you could put a hash marks in front of the text. So these are different sizes of the different headers, but let's go ahead and make our own header here. So like for instance we could say one hash market and this is a header. So that Markdown knows to take that and put it into an h1 element. Another way we can make a header by taking advantage of the visual editor is by going on the new line and we can press this slash command. And this this. So this is what the visual editor is doing here it gives you this option to insert different elements and options here so we could, if we didn't remember how to do that. So you can put a hash mark to make the header, you can press slash and go to the top, the heading one top level heading, and then it'll get it ready for us so we could say, This is another header. So that's how you use ours and an important reason to use this is because it helps you navigate the document. So if you click on this line or this icon over here with the little headers, you'll see this little sidebar menu show up on the right hand side, where there's actually links to sections of the document, based on where you put these header, these header lines. So for instance I put this is a header up top, we can jump down to tables on the bottom, we can jump back up by going back here. So it's very useful for navigating through document, but also you know separating your thoughts to make it a little bit more readable. Let's see what our next slide says. We went through this. So next we're going to talk about editing text or formatting text within the markdown file. So on the left hand side is the raw text that you'd be putting in so and then on the right hand side is the rendered output. Can everyone see this awesome. Okay awesome. So on the left hand side if we put two asterisks around the text it'll show up as bolded text on the right hand side. If you do under our underline beforehand we can see it gets italicized on the right hand side. And then let's go through a few slides and then we'll jump to the our markdown documents for not switching the share screen back and forth. We'll talk about formatting text we'll talk about making bullet lists. So on the left hand side if you want to bullet list of items you just skip a new line. Just like here and then you put this dash in front of your text items. So then by and then just putting a new line afterwards so then on the left hand side we see whooping cough polio diphtheria hepatitis or a host of other infections. And because they have the dash in front of them they get rendered as a bullet list on the right hand side. Similarly, if you begin the list with numbers, then it'll, it'll enumerate the items when it gets rendered. So we see one here, one here and one here, and it's a list just like before but now it gets rendered as 123. So something to point out is that you might be curious like why it's just ones over and over. And that's because markdown knows how to sequentially order them it knows how to number them correctly. So we're going to wrestle with saying like 123 and maybe sometimes you want to like change this to move it later in the list, but then you have to switch all the different numbers around. Markdown knows that if you just put all ones down there it'll correctly number the items when you render the document. Okay, so let's go ahead and jump back into the document. So we can see this in action. So like right here we have, I'm going to jump out of the visual editor so you can see the raw text. So we see vaccines are one of the greatest or one of the great trends of modern medicine. And then when we render it in markdown we get the bold and italicized just like before. And then also we have the bulleted lists like we said here, and you can add items to this list by just going to the end here and just saying enter and then another item, and yet another one. And so again if you don't remember how to, if you forget like how to make a bullet list with plain markdown you can use the visual editor to take advantage or use the visual editor to start a list for you by pressing slash. I didn't look for list. So I'm just typing ally and then bullet list pops up. And then it, well it's there's no separation here but so puts it in with the other list but it starts to list for you. So this is another list. And again we have, so then moving on to numbered lists now. So, again we can just go back to the, we go to the end of the last line, press enter and it'll automatically number it for us. Here is another item. And so, keep in mind that this is the rendered output if we go back to the raw text. Oh I guess, oh visual editor is being smart about it for us. Actually editing the markdown to be the correct numbers. But let's go ahead and let's make another list. I'm going to go back into the visual editor. And just like before I get by slash press li to go to the numbered list. And then visual editor will start another list for me so I could say. And three, and that works very nicely for us. All right so next we're going to talk about images. So let's jump back to the slides for a bit. We're going to talk about images and links. So the, the markdown syntax for an image is this. These empty brackets so exclamation point these empty brackets, and then a link to the image in parentheses. And then it gets rendered into markdown markdown knows to take whatever images here and render it as an image. And we can also do links in kind of a similar way. Instead of starting it with an exclamation point we just leave it alone here and just have open brackets. The label of the image, and then the, the link to where we want the hyperlink to where we want the link to go. When this gets rendered, we get this nice little hyperlink here that we used to and then we can go ahead and just click on that. It'll take us, it'll take us to that website here. And let's see what's on here. Okay, so let's go ahead and see that action on in the cloud. Okay, yeah, so in the visual editor it's already rendered the image. Let's go ahead and just verify that when we jump out the visual editor that we see the exact same thing here so we see the exclamation point we see the caption, or the alt text. And then we see the link to the image and it renders. And again, if you don't remember how to. If you don't remember the syntax you can do slash I am to pull up image. I think you actually browse. So visual editor will open up this window to have you browse to the location of the image. So it's in, I guess it's a hyperlink. So I'm going to copy the hyperlink from here. And just paste it in there. And then mark the visual editor will take care of making the actual markdown syntax for us here. So if you're real quick, yeah, I want to make sure that you don't skim over the sizing near of the images because that's another really awesome feature about images and the markdown visual editor is being able to resize them. Right, so instead of having to like wrestle with HTML or CSS to resize images you can actually edit it directly with the visual editor. If you click on the image itself you see this little border pop up around it. I'll point your attention to this triple dot thing here so when you get more advanced with like using CSS and stuff like that you can mess with all these things directly but for now we're going to talk about this changing the width and height like Allison pointed out. So we can lock the ratio here and just mess and just change the width and the height and quarter will take care of writing the markdown styling to change the size of the image. So, we can see actually how market, how the visual editor is doing it by just going back by turning off the visual editor and going back into the plain text, and seeing that is just adding the styling element of with 300 on the end. All right, and then next on the slides we have a check in so I'm going to switch again, jumping back and forth. A lot worse than all tapping so check in. How do you add headers in our markdown and markdown rather. So, I guess. Allison, do you know how to add headers and markdown. Oh, I will take a guess I think it's the third one. Wow, incredible awesome. Yeah, so the hash is the header and the number of hash marks you put in front of the header makes the header a different size. And again like if you I provide I provided a cheat sheet at the top of the document so you can look it up yourself. Or if you didn't remember that, take advantage of the visual editor slash command to pull up these different things. And then like we saw earlier for to make a list, we can either just make a bullet list by putting the dash mark in front of it or you can put an asterisk also. And you can make a numbered list by doing the one right in front of it. Oh, okay, so let me jump through tables real quick and then we'll, I'm going to jump through tables and citations and then we're going to have a little activity for you guys. So let's jump over back here. So, I think one of my favorite things about the visual editor is how easy it makes tables and markdown so to demonstrate how nice it is let's jump back to the raw text, and this is the markdown syntax for a table. You basically get to draw these lines and make sure that make sure that they all line up. And there's different syntax for centering the text and everything. And for simple tables is not too bad but when the table starts growing or we need to change the content of the cells it can get pretty onerous onerous, but visual editor makes it super nice because when we jump to the visual editor is actually a command for inserting a table so just like the other commands for visual editor we press slash and to start typing in something so I start typing table and table pops up, and it asks me how I want to make rows and columns so I'll just say three and three. Breakfast, lunch, dinner, and then, yeah, then we can say pancakes, sandwich, steak, and let's jump back into the raw text and we can see that visual editor is actually writing the markdown syntax for us so we don't have to worry about aligning it properly, we don't have to worry about making sure that it's centered nicely on the page or whatever. It just works. And beyond that, we actually edit the table pretty nicely in here too so for instance I could just, if I have too many rows I could just right click inside this table and delete the row. I'll delete this row too. I'll delete this column. And it just takes, and then the visual editor will just take care of that for you. Some other cool things with the table is that you can also change the alignment of the columns. Like right click on this column, click center, click center, or I'll change this to the right. And it'll take care of that for you. So these are the, this is the syntax for centering a column or right aligning a column over here. And then finally something I think is super cool about the visual editor is that you can actually add, directly add citations to the documents using DOIs. So here's a couple of DOIs here, and we're just going to go ahead and just, I'm going to show a few different ways that we can add citations. So, I'll say, actually let me go down a little bit. So citation one, I'm going to say, so the first way you can add citations is by doing, well first I'm going to copy this. I'm going to say at, oh I need to be individual editor. Yeah, so this is only working in the visual editor. So I'll say at. And then you notice that this little thing pops up where it says search or DOI, and I'm just going to paste in the DOI here. And you see that it pops, this really cool pop up pops up with the citation information for that DOI number. You also notice that's creating a references.bib file. So if we click OK, it's going to create this references.bib in our directory. And it's also going to go to the top metadata and tell the file what the directory, what the references are. So that's one way to do citations. So let's, so here's another way to do citations. Just like before we can do slash citation. And now there's a bunch of different options here. Actually, I need to grab this. Sorry about that. So let's go back to citation. And there's a bunch of different options here. So if you have your own bibliography file, if you have Zotero in later versions of the RStudio dailies, you can actually pull from your local Zotero files. You can search in a couple different databases. But again, we can just paste in our DOI here and it's going to look it up here. And it's going to show us how the, if it's going to look, it's going to ask us where we want to add the reference and we can just go to insert that. And then finally, if you, a third way we can insert these is by using the insert menu up top, we can say insert citation. And then we get the same menu up there. So let's go ahead and just do that. I'm going to add this citation. Oh, I think this is actually my paper that, okay, never mind. This is embarrassing. I swear it's real. But anyway, so now that we have the citations added in the references. Let's go ahead and actually render the document. And let me show you what visual editor will also do for you once you have these citations. So we have, we added the citations at the very bottom. And I'm actually prints out a bibliography for you as well. And this is because he told in the top metadata, what the bibliography is. So because it has that, when we render it with Cordo, it's going to output that bibliography on the bottom. And so just to make it a little bit nicer, we can just say, oops, you just say header one bibliography. Let's go ahead and save that and let's render it again. Well, it's not, it's not rendering for whatever reason, but yeah, so that's all we have for now that's now let's go to an activity for you guys. Let me go ahead and actually 119. So I want to stay true to my schedule so we are going to take a 10 minute break right now. And we will meet back at 130 after the hour and if Robert Klein is here I want to make sure we troubleshoot because he's not having the forward slash working in his QMD so I'm going to set a timer for myself and share my screen real quick let's see if I can. I am going to share my screen, and then Robert if you're around. We'll go until one third 130 my time. Okay. Yeah. So what is going on let me go to my, I'm going to stop sharing this green and go back to I don't know why I can't share overall screens, but if I go into my our studio cloud workspace. So the 01 explore QMD, which you could be in but it doesn't really matter it could be an arm defile or it could be a QMD file so all the features that Paul just described in the visual editor should work with R&D or QMD but we happen to be within a QMD file so I'm going to click on this little draft icon here that says switch to visual markdown editor. This is the first time I've done this in this project so it's a switch to visual mode you're activating visual markdown or activating our markdown visual editing mode this mode enables you to compose markdown using a familiar word processor style interface. There's a link to the documentation and then it also allows me to say don't show this message again so I'm going to go ahead and click and say use visual mode. So now you can see kind of more of like a render version with the outline, the YAML is kind of boxed off. And here's the text so I'm going to do a slash and when I do that forward slash on a line all by itself I see the menu so Robert are you not seeing that if you follow those steps. That was it. I was not the mode. Gotcha. The right mode. This only works in visual editor mode yeah so as Paul demonstrated you get it's kind of like a hot key like you can just do that and find anything you can scroll down and see anything so if you're just like I don't even remember you can actually insert emojis this way. So I can actually choose from the emoji picker like this is the bomb. And then you can also use if you don't want to use that forward slash you can use any of these buttons up top which are probably more familiar from a word processing standpoint to be able to bold italics underline Bulleted list numbered list links, all of those things. So yeah everything's possible when you click on the draft icon, but you can also do this in a Dart RMD file right now like if you download the, not even a daily build of our studio but if you download the official build of our studio ID this is all built in right now and works with our markdown files. Okay. Good. Great. Well, I'm glad we were able to troubleshoot that. I'm going to finish up my break and then we'll be back. I've got my timer going down here. I'm going to stop my share and then reshare so that everybody can see my timer. I have a little bit less than two minutes left but I see a really good question about I have papers written in our markdown that RMD files. Are there any known issues with building rendering old RMD with Cortel? Yes, they're not necessarily known issues but there are things that you're going to have to change about your source to be able to have them work in Cortel. So the first one that has gotten me several times is that you need to change your YAML. If you have knit to a certain output format with our markdown you have that specified with the output key in the YAML. With Cortel you'll need to use the format key and you'll need to make sure that one of the formats supported by Pandoc is the one that you want. So for example, you know if you have knit to something like distil right now, we haven't fully replicated the distil experience in Cortel yet but it's on the roadmap coming up this fall. If you were using the articles package for example for using like a law tech article template that has not been done yet in Cortel as well so it depends on what you're trying to mimic. If it's a plain our markdown document using the our markdown package to render and using one of the base output formats, those should be largely replicable in Cortel. You just need to be able to find which Pandoc output format that maps on to. And then there might be some other YAML keys that you'll need to kind of go through and massage different output options. Book down, book down and block down. I know it always auto corrects to blow down. So, book down you can do a Cortel book right now we won't have time in this workshop to do it but let me show you in the documentation where that is well first we're back from our break. Hi everybody. I wanted to show first of all the documentation for the visual or markdown editor because this is actually available. I'm going to throw it in the chat. This is available right now in our studio ID you don't have to use a daily or anything like that you can use this with our markdown. Our markdown document, other than shrink and slides I believe those are the only ones where this does not work currently. So you can use a lot of the features that Paul just demoed with an our markdown document. The difference is that we're using a daily build right now of the ID a special build for our workshop. So that we can use a lot of these features with Cortel built into the ID float this. And in Cortel, we're not going to be able to get to it here but the people who are asking about book down and blog down you can go into the projects tab of the Cortel documentation and you'll see creating a book book structure, but cross reps and book output. There's also websites but there isn't going to be something exactly like a hugo site in Cortel, Cortel is going to be more creating like a distilled website for an our markdown website if you've made a website like those before. And maybe similar to this documentation site that I'm showing you right now that was built with Cortel the Cortel docs are built at Cortel. If you want to make something like that, then you can make a website using Cortel and get pretty far. Same with a book, but if you're interested in replicating the exact blog down experience I think that's going to be a little bit different but you can take individual posts or content from a blog down project and render them with Cortel, but still use maybe some package under the hood to use to access hugo basically to serve your site, do all of those things. So that might be a little bit more of a hybrid approach. But one of the great things about Cortel is that it's made for single input files, also immediately expandable to multiple input files so it has this notion of projects, and you can create multiple. multiple rmd files or qmd files as inputs and have them be rendered into a book, a website, and have that work the same way as it did when you were making the single document. Okay, so we're a little bit behind schedule. So I'm going to head back to where we left off so with Paul we talked about the visual markdown editor, and all of the things that we can do to our text. And the next thing that comes after text is code. So when you're in a qmd file just like with an R markdown file, you can insert our code. I'm going to try to let me see if I can get my participants back. I'm wondering how many people here feel very comfortable with knit our chunks I'm wondering if y'all would rather me skip that if that's kind of like very basic if people could give me a thumbs up how if they feel very comfortable with knit our chunks I'm just kind of trying to. I see some some thumbs up which is great but not a lot so I'm going to keep going forward with knit or code chunks and those of you who feel really comfortable with it you can maybe sit back and relax a little bit. And know that we'll we'll get where we need to go. This is the first time teaching teaching corto so there's going to be some some things that I've missed timed here. Let's do a little bit of a review so one of the things that I think is trickiest about working with any computational document is figuring out what you're going to see printed when you actually do the rendering so you've got this source code and you've got this output and there's nothing worse than being surprised by what's in or what's not in the output. So using code chunks we insert our our code into our documents. And if you think about what is the fate of this chunk over here so I'm taking a data set called mock data, and I'm piping that to this function called distinct. And what you end up seeing when you render is actually the, the code itself plus the output and that's because knit our produces all output faithfully. So that's its default is that it's going to show you both the code and the output. So it might you predict here if you saw this code chunk. You might have a guess that you would see something print out, and this has often taken me and other new users by surprise as well. But here you would actually just see the code because I'm not actually printing anything. So when you would render this unless you actually printed the object in underscore sites, you wouldn't see anything in your output file. If you did do that, then you would be able to see that for example in this mock data set, there's only one site currently. So what about this one this is some gg plot code that's taking the mock data and making a bar plot out of that data. So if you had this code in a code chunk, you would see both the code and you would see the plot that it creates in your output when you render. Let's see. It's really hard to switch must stop sharing. And I'm going to go back to sharing my cloud. Let's see, I can see Sylvia Sylvia can you thumbs up that you can see my cloud. Thank you. Alright, so here I'm in my own explore QMD file and you add code chunks the same way as you would in our markdown document if you're used to that. So for example, I'm going to go down to under summary and I'm still in visual editor mode here it doesn't matter which one you're in this works in either mode, but you can click the insert a new code chunk. And then here, like, let's do the mock data pipe it to distinct arm. And then you have the option when you're working in any QMD document or RMD document to say, Okay, I'm in the studio ID I want to run all chunks above it, and you can see that it's actually running those. I think there was a question about an error I don't know that I haven't seen an error in this file I hope not, but I didn't see a response so I'm seeing output that goes along with each chunk. And then I clicked on that one that said run all previous so that means run all chunks above it, but not run the actual one that I did so to run the actual one that I just added as I can run this current chunk. And then you can see the output here. And now one thing you can do as well as you can click on this little gear icon at the top and this is in the ID in our studio. And if you don't like this say output in line you can say chunk output and console, and I'm going to remove it all. And then you can see that if I had back up to where I was. And if I run all the chunks above it prints all of those outputs to the console, instead of adding them here so here I'm going to run that same chunk again and you can see that that that prints there. Either way that's more for your interactive use and you can still render when you want to create something shareable. See did it actually render. There we go. And then bath asked a really good question about why is it. Why is that job. It's actually watching file for changes why is it staying open. So the idea is that you're able to actually set this up so that you have render that you're rendering on save. And so anytime you'd actually anytime you'd actually, sorry. Day. Anytime you'd want to be able to make changes to it. It would detect that like click save. There actually is a button on my local machine that has render on save. And it's not showing up in our version here. But that's why I think it's watching for file changes. So it's not totally fully featured here in the studio cloud ID but there's a button that allows you to do that. I'm going to stop sharing there go back to our file. Okay. So that was our code. So we showed how to add chunks how to run them interactively to be able to run all the chunks above it be able to run the current chunk and as well being able to render and just seeing it all and then also changing where that output goes when you're working interactively so whether it goes in the console or if it stays kind of in line with where the code is two different options. So the third thing that we've seen when we render is output. And this is really great this is what saves us all that time copying and pasting when we're trying to create something that we want to share with somebody else and say you know, one of those observations where you're like huh this is weird can you take a look or one of these things that I find in my data that's really exciting and cool and I want to share with somebody, but you have to be able to share the output to be able to do that. So we're going to massage how your output looks when somebody else looks at it is to use chunk options, and these are enabled through the NIDAR package so even though we're using a QMD file and we're using Corto now we're actually still using the NIDAR package under the hood to be able to render our code. So we're still doing a little bit of knitting. Here's a chunk for example showing glimpse mock data. And you can see that over here it's printing out faithfully the both the code itself and the output. Now if I wanted to change that and if I wanted to say not show the code at all, you can use the NIDAR chunk option called echo. And you can set these chunk options differently in Corto and this is actually going to be enabled eventually in all of our supported our packages as well that use NIDAR under the hood. Right now it's only enabled in Corto it's the special syntax that allows you to have chunk options using a hash symbol so basically like a common symbol and then pipe. So the pipe is right above your backslash it's like a shift, shift and backslash on my keyboard. And so you enter hashtag pipe and then you can list your options there so here echo colon and then false. So if I use that it's the same thing as putting it in the chunk header, and you can do either way or you can even mix them you can have a hybrid approach. So here you can see that I'm not echoing the code. You can also do the same for evaluating the code so you can set eval true eval false so here you would only see the code itself you wouldn't see the output from it. You can also use a chunk option called include and this is really helpful if you just want to have code that runs in the background but you don't necessarily want to share it so you might have some gnarly code that creates a ggplot that you want to share with people but you don't necessarily need them to see how the sausage was made you just want them to evaluate the plot. So you can set that chunk option to be include false combining chunk options you can stack them essentially so echo true results hide. You can also string them together using a comma to separate now notice that here there's a colon to separate them and if they're in the header there's an equal sign so that's kind of a gotcha that's going to be a little bit hard to get used to I made that mistake multiple times and preparing for this workshop so road road block ahead. So there are two options that are really really helpful when you're first doing a QMD file or an arm defile is message and warning. So this is showing you the the typical output from loading the tidy verse, which is noisy right. It tells you all the packages that it loads and it tells you all the conflicts and you often don't want to stand up in your rendered output. So you can do message false warning false on a chunk that includes that and then all you'll see is that the package library tidy versus loaded. So again to combine chunk options you can either stack them if you want to use the hash and pipe method. So place after a hash and pipe stack multiples and use a colon between them. Another option is to place between the curly braces and separate them by commas using a list and in that case you're always going to use an equal sign instead of a colon. Either way you can stack multiple chunk options so that you can create a very custom customized kind of report for a certain kind of audience based on what you show them and what you hide from them. So there's a ton of chunk options there's 53. So we only touched on a few of them but I encourage you to look at the NITR documentation because there's a chunk for a chunk option for almost any use case. So if you've ever thought oh I want to be able to hide this but not show this or show this but not show this you you're probably in that our option chunk option world. Now through text that was a visual markdown editor we've talked about our code, and then we've talked about the chunk options for controlling how the code looks when it's output. And the last part that we're going to talk about right now is metadata. So this is the part of the very top of the document so we're sort of working back up to the top of your document that part that's in between the three dashes it's fenced off. It's called YAML block it's called metadata and YAML stands for YAML and markup language I think it actually changed recently, but if you're thinking to yourself that I don't know what YAML is and the, the name YAML doesn't help me and the spelling it out doesn't help me either then you're not alone. It's basically a way of specifying keys and values together. And the way I think of it is the way I used to organize my Word documents for example was to basically store my metadata in my files themselves. You know, I'd maybe store the date that I, that I wrote an article or wrote, you know, a note to someone. I put that on the document itself or I've maybe put it in the file name, or I'd put it, you know somewhere that I could track it because I'd need to keep track of these things. And so that's kind of like separated out in your QMD files. So it's actually stored in the same document so you can see the author name, the title, the date for example. And just basic metadata that you would put in a document. And the nice thing is that it does some things that allow them to show up when you actually render the document as well. So the title will show up as the title. And so it's helpful for being able to organize your files and keep that stuff at the top. Another thing that's nice about Corto is that you can actually store execution options up in your YAML as well. There we go. So execution options refer to those knit or chunk options that we just talked about, but now we're talking about setting them for the whole document. So if you've used our markdown, you may have gotten used to using a setup chunk. And here we're actually going to put them into the YAML now, which is a nice and refreshing way to do global options for your whole document. So the way you start with this is using the execute key in your YAML to kind of cement this down, and then you can stack them one on top of each other. So here if you used this in your YAML for your document, you'd hide the code for every single chunk. You would hide the messages and you'd hide the warnings and you'd set the output width for all figures to be 100%. So you can still use individual chunk options, and you probably should to override these settings, but this is nice for being able to set up your default chunk options. And then they're up in your YAML. They're really easy to track and I hope easier for people to collaborate on documents and be able to see what they're changing. The other thing that you can put into the YAML is your output format. So we've already shown you that we've been using format HTML. You can also use a different output format. I'm going to ask you to go back to your cloud setup, and I'm going to actually do it with you instead of using the timer because I want to be cautious of time here. So I'm going to demo how to you how to knit to a render to a Word document, and I'd love for you guys to follow along with me if you'd like in your studio cloud project. So what I'm going to do is I'm going to add doc acts as an output format. So right now you have a document that has format HTML, and I'm going to add doc acts onto it so that has an additional format to render to. So let me escape here and then reshare my screen here. Okay. So I'm back in 01 dash explorer QMD. And you can see that the format is here format HTML. So what I'm going to do is I'm going to hit enter on format and I'm going to indent it for the HTML and I'm going to add default here. And that's what allows me to be able to have just plain old HTML as an output. And then I'm also going to add doc acts here as a default. I should be able to see my renders. So what you get this notice in our studio cloud that it's finished rendering a QMD document to doc acts. And since it's a doc acts file it's going to allow me to download the file. So I'm going to click allow, and I'm going to open it locally. And I'm going to actually have to stop sharing my screen and then start sharing it again to show you the Microsoft. There we go. So this is the Word document output from from the exact same thing that we were just looking at his HTML. So you can see that it printed everything out faithfully. And it looks pretty darn good actually for a Word document. So you can get out of that copy paste mode of taking figures that you're making an R inserting them into Word documents to be able to share with other people. And you could set this to be, you know, echo false for the execution options and not show any code for example and just be able to share this with people so they could see the outputs. Okay, I'm going to share my screen again. Okay. So that's how you add additional built in formats, and you can see all the formats on the Cortot documentation site. So when you click on this formats up here, you see HTML PDF Word, there's some work done formats and then there's some miscellaneous formats there's a lot of Pandoc supported formats so presentations and then there's even more formats. So that's the way to find out is the format that I want to make here. The final thing that I think is really cool about Cortot that I want to make sure that you know about is that you can add output options really easily to the output formats and there's a ton that that are built into Cortot that are things that you've probably wished for if you've used our markdown across the different packages and that some have sometimes have been enabled sometimes haven't been so I want to really appreciate what's built in there. How you save output options in your YAML is to stack them and nest them underneath the output format. So whereas before we were starting with format HTML. If you wanted to add an option for HTML for example to add a table of contents, you would nest to see under HTML. So how we used a single format. Hopefully you're thinking to yourself okay I'm going to be using the format key. It's a little bit different. Allison has done be numerous times in a Cortot document it's not going to work. And you're going to get mystifying errors. Paul knows. So it's hard to break that habit if you're in our markdown user it's format HTML. And how do you set options for a format actually the same way that you do with our markdown so you nest to the option underneath the format itself and you need to indent it so the format itself is indented to spaces and the indented about four. And I just showed you how we actually stacked another output format on top of each other so the way was to indent them both to spaces HTML and tag on default at the end and Doc X for example and so be is wrong here. And then how do you add up put options to one of the formats you would just stack them again and just use the the for the for indentation for space indentation to nest it underneath. So a would also be right I think a was right for all of them. Okay so this is another one where I would normally leave you on your own for five minutes but I'm going to walk you through it because I want to be conscious of time and I want to be sure to take a break at 20 minutes after the hour. So what we're going to do is I'm going to demo how to edit your YAML and add some very cool features from Corto into an HTML output format. So the ones that I'm going to focus on are adding a table of contents, this is a really popular one that you may have used with our markdown. And I'm going to open up that documentation page so you can see where I'm seeing where I'm going to be following. So, if you were on the Corto main website you'd go to formats HTML basics, and then you'd see table of contents right below that. So I'm going to be following that instruction. I'm also going to show you how to add commenting because if you've ever used hypothesis before you can add commenting very easily to your HTML documents which is nice without needing to know JavaScript. So I'm going to add a theme. If you're used to our markdown you may be used to using themes. I really like using themes for changing the look and feel of your document right away with different built in fonts and colors and they just look nice and polished. So I'm going to quickly do that. And I'm also going to add code tools. And then I will try to make it so that I can also add code folding at the very end of this. So I'm going to stop sharing the screen and go to my cloud. Okay. So first up, I'm going to actually remove doc acts for now. And I'm going to try to make this bigger. There we go. Okay, so the first thing I said I wanted to do was add a table of contents. So I hit enter after that HTML I go into spaces, and I'm going to do to see true. If I click save, and I click render. Let's view what that does right away. There we go. It's not showing up in the small window so if you do knit this in the viewer in our studio. If I went this way, I'd be able to see it. It's not showing up in the viewer pane for some reason so but if I click on this little view, this little kind of show a new window, you can see that it pops out and it does show me the table of contents over here. This is a really nice little table of contents I can click around and you can see that it highlights where I am actively among those four sections labeled table of contents. I'm in TOC true. So next up what did I want to do, Paul. What was the next thing I was going to add a theme was that my second one. Comments. Yes, commenting, commenting. And then I believe it's is a hypothesis true. I'm gonna have to go back and see. What is I guessed correctly. Okay, commenting hypothesis true so I'm going to render that. And I'm not actually sure if this is going to update. I'm going to, for some reason, I'm going to close that out and then re-render. Okay, so you can still see my table of contents, but I've added hypothesis JS this means that if I, it's supposed to be that if I highlight. I'm going to click on it. It's supposed to show me and it did when I tested it, the ability to comment. Okay, I've probably done that wrong, although I see a comment in chat, which might be somebody telling me how to do this. Okay, I'm going to leave. Is it commenting? Did I do it right? Try comments instead of comments. Thank you. Thank you. Okay, I knew I had done something. Okay, I'm going to go back. Okay, so comments hypothesis true allows you to have annotation and highlighting for people who go to your website. Now this is just obviously a local preview. But if you were to publish this on GitHub pages or Netlify, for example, you'd be able to have people interact with your, your document pretty easily and that's a really nice thing to have built in. That's using hypothesis JS under the hood. You can also add utterances comments as well. I just think I wanted to add a theme. And I think I'm going to try Lux. I believe Lux is a theme that is bundled with bootstrap. And those the options that are there for you for theme are listed in the documentation as well. So for some reason it's not updating my life preview here. But let's see. There we go. So it's a quick way to change the look and feel of your site. It's possible. And it's linked to from the portrait documentation is to go to the boot swatch site and you can see all the different themes the way you pick them, or the way you select them is to, to use no caps. So it's like Yeti, Yeti with a lowercase why I use the locks with a lowercase l. And you use the theme key. So real quick, I'm going to show you flatly. This is the one that I sent you Paul that was so ugly. Oh, it's like courts or something is awful. It's a good one for like making sure that my visuals are working. What was it. I'm digging through. It was courts it was courts. Yeah, okay. Okay, courts. It's like a piece of free. Yeah, I tell, I tell you that I can give you polished reports but I could also give you this. Oh, get ready for it. Ombre. Charles says sketchy is a pretty good one too. That she's a pretty good one too let's try sketchy yet. I don't see what's wrong with it. I don't think people at chop would love reports written like that. So it's not updating in the viewer. But there's sketchy the good ones that's a really obvious one that I've changed. So I actually really like Lux and I like flatly those are my two favorites but you are welcome to choose your favorite because they're all possible I think minty is another one that's got some cool. It's just a little bit too light for me. I have aging eyes. Kind of makes me feel like I want to dial up the contrast on my computer. So that was team table of contents and then I added hypothesis and now the last thing that I want to show you is code folding because cold folding is one of our more requested features. It's worked on output formats. And it's one that's very hard to implement similarly across the different output formats so that's one thing that Cortot really has made a lot easier so I'm going to add just code folding true. And render it. I'm going to try to lower this so you can see. Oh, let's see. Did I do it wrong. I'll try code fold, fold, see. That's why you're here, Paul. Beautiful. So now it's nested in these nice little code blocks, so you can quickly drop down and see the different code used to create it, but it's kind of hidden in like a little details tag that makes it kind of nice. The other thing I wanted to show was something from code tools. So since I can't look at my help at the same time I'm doing this Paul code tools. Is it just true. Yeah, code tools true. Okay, great. So this one is really nice and it mimics one of my favorite behaviors of our markdown, which is you get this little button at the top. So all code hide all code or view source. So this would allow you to kind of auto expand all those folded code blocks, or auto hide them all. And then also view source pops up the source code in a nice little window for you so you can see all of it at once so if somebody is just like I want to see the raw form D for it. It's in there it also actually has a clipboard icon here which is very nice. So you could just copy the whole source for example. I'm some of the other things that are built in. I want to show you real quick as well. And so these are things that are built in the quarter that I didn't even have to set up in my output options. So for example one is if you go to any of these headers. You'll notice that I have, let's see. I'm going to switch back to the Lux so I can see better. This is like too low contrast. There we go. Now you can see the word code up there was invisible on the last one. So show all code hide all code view source. And then if I click around you'll see that I have the different headers are basically anchor links. You can also see that every chunk has immediate copy paste. So you can see that little clipboard icon you can click on it it will show you that it's copied pasted, you can turn that off but it's easily on and any Corto HTML output format. And then you'll also notice that I have numbered figures. So here I have figure one participant enrollment by site. And I have down here figure to participant enrollment by site and study arm. So this is another really nice feature of Corto that you might have enjoyed in book down for example if you've used any of our markdown output formats and it's in there. Another thing that you'll see when I hover over this figure is that there's even an anchor link to that figure for example so you could link somebody to that specific figure that you're seeing. So you can, let's see, I don't think I have an example here. I don't have an example there so I'm not going to mention that. So code fold true code, code tools true. Those are really nice features and those are all covered under the HTML basic section of the Corto documents. So I'm going to head back to my Firefox so I can show you here HTML basics. And what I showed you was table of contents we did commenting. I didn't have any references in there but there are automatic reference pop ups here so for example if I hover over and it are there you'll see a little nice pop up that isn't our package for creating dynamic documents. There's external link formatting so there's a lot of stuff built in there's tab sets. So if you've loved your tab sets and you felt limited because we haven't been able to enable them across all the output formats. You can have tab sets in every output format now you can have them in your presentations built with Corto you can have them built with your HTML documents but with Corto. All of that good stuff. Okay. So here's where I ended up and I didn't show code summary that was the last one. So that's how you change the actual word that shows up I won't show it now but you can try it on your own. And then when we take our break you can play around with that. And it's also still documented on our HTML basics. So, I will wrap up this section with what is hard about YAML in general. So y'all saw me separate a little bit with YAML already who chose that it's hard to know the right key. It's easy to forget the right key I tried code folding instead of code fold. I tried commenting instead of comments. Indentations matter if you've already been using our markdown this should be a familiar pain to you we're actually working on some better debugging tools and the IDE itself so it'll actually kind of show you when you've got when we think you have your indentations wrong and when your keys might be wrong. But that's really one of the challenging things about YAML. And when you do get an error it's rarely an informative error message it's usually some kind of like gobbledygook about parsing and that can be really intimidating. So YAML isn't exactly user friendly we're trying to make it friendlier. It's also difficult to know which output options are relevant for different formats. So the quarter docs are pretty good about this they separate out by format which options are there and of course for HTML there's a lot more options built in for each format but in other words I've spent a lot of time making sure that there's some pretty cool options available for word and PDF as well. And it's also easy to forget the possible values for a given key so whether some things true or false or whether it's a string, for example, if it needs to be in quotes or not. All of those things can be kind of tricky about YAML and those are sort of just things that I want you to realize are totally normal for all our markdown or Cortot users is grappling with YAML problems. I'm going to skip this new data dump because it's 205 and I've only got till 220 so I'm going to move on to slide deck number two here. But you can go back and try this later. I had a new data set that you could use to to import and use the exact same 01 explore file to be able to re render and see data for Boston Seattle and Denver instead of just Boston. So I'm going to wrap up with these takeaways and then move on to slide deck number two. So, use YAML to set up meaningful metadata for your document this includes you know name title date things like that but that's also your way of specifying output formats. I'll put options for that for those formats and then with Cortot it's also your way of setting up your, your whole document execution options. So, if you've been an our markdown user you don't need to do a setup chunk anymore you can use execute as the YAML key and then nest your global options and the execution options of your YAML. Also style your document you can use the YAML to add options to style themes CSS all of those things that's the ways to change how your document looks and renders you can add code folding. You can add commenting there's a number of things that are built into Cortot so I encourage you to look through the documents and see all the bells and whistles. Next use markdown headers with a single or up to six hashtags. This is a really nice way to organize your document and as Paul showed you with the visual markdown editor especially it allows you see that outline mode really quickly and allows you to jump around a larger document pretty nicely. And I really encourage you to organize your your code. I'm going to show you a little bit more about NIDAR in the next section. So, so styling your text using markdown bold italics bullets and lists can also make your, your QMD documents more readable for other people. And we also talked about styling your output so using that our chunk options like include message warning eval. All of those. And my final bit of advice is to render early and render often we talked about the interactive mode when you're working and running code chunks in our studio. All the chunks above you can run the current chunk. Those are all really great as you're iterating and trying to work out your code but I encourage you also to do the full document render early and often so that you can make sure you don't end up in a place where your document doesn't render. So I'm going to stop my share real quick. I'm going to look through the questions real quick. I'm going to talk about conversion at the end I see a good code completion for the YAML header stuff and that is on the to do list I am telling you it is is being worked on right now. Auto complete for YAML. I know. Okay, I'm going to share. Okay, and I'm going to get started on authoring I'm going to see how far I guess. So make sure you're still logged into our studio cloud, we would be working off o2 authoring as the project now instead of 01 basics. And I'm going to go ahead and show you my screen as well. I'd like for you to open up o2 draft that QMD and read the source file take a few minutes I might show me this down to four minutes instead of five just to shave off one minute. So look at the source and answer the following questions what's the output format. Are there any output options are any nitter execution options set anything in the code look foreign to you and finally render this file. Look at anything in the output. See if it surprises you or not. So let's take a few minutes. I'm going to do it at the same time as well so you can just watch the screen or you can play around in our studio cloud it's up to you. Give me a thumbs up if they've been able to get the project to load yet. Oh, there's mine. Okay, so some people are in it. I'm just getting throttled here. Okay, great. Thank you. Okay, we have about a minute and a half left. I'm going to look at my participant view. Can people give me a thumbs up if they've been able to look at the document o2 draft q dot QMD 17 people 18. Okay. We're 20 minutes out or 20 seconds out. I'm going to go ahead and move on to the next thing. So, I'm hoping that to switch screens again. Okay, so here's what I was looking at and so you know we've got metadata we've got a single header here note so if I look at my outline view it's not super helpful might want to add some work down headers to make it a little bit more organized. But you can see that I have a bunch of code chunks here and the sum and substance of those code chunks is I'm loading in some packages I'm loading the data. And then I'm also loading in this external figures dot our script that I wrote and that was just so that I didn't overwhelm you with a bunch of figure making code. And that source file does is it gives you access to these, these four plots, these four plot objects. So when you rendered you should have seen them stacked right on top of each other so we have an age histogram survival percent plot, a survival by days plot based on follow up status and study arm, and an adverse events. And then I also have a static image down below so hopefully you guys noticed that as well that's happened that's happening to use NIDAR include graphics for that image. And then there's a final, there's a final plot as well. I'm talking about counts by site participant counts by site. Okay. Similar to code chunks in your output I think I have often fallen into the pit of not exactly being able to predict where my figures are going to go or when they're going to show up. So I always want to point out that you have to be able to print your plots to be able to show them when you render them. So if you saw this kind of plot in your code will it print yeah absolutely this is going to print and it's also going to show the code chunk above it as well. What about this this is actually saving an object so this code would create an object called age underscore histogram. And then this big chunk of code underneath is actually how you make that plot so if you were to render this into the output know you would not see this, you would only see the code used to make it. You would see it however if you printed age histogram as the object. So it's always important to remember what things you're saving as objects and which things you're trying to print and show other people. There's a lot of chunk options for plots when we talked about the knit or chunk options we talked about include eval message warning, but there's some specific ones for plots that are super helpful as well so there's some related to figure size and figure size. So there's big dot width, big dot height, big dot ASP, and big device so if you wanted to change it for some reason for one of your publications for example the journal required all figures to be, you know PNG is for example that can often happen so you can set that big device, and you can do that both for an individual figure, an individual chunk or you can actually do it in the execution options as well and make it for the full document. And the place to find all the documentation for that is in the knit or chunk, knit or options for plots. So there's that link at the bottom. So my favorite ones to share our out with this is a really quick and easy way to be able to resize a figure. So again using the syntax of adding options below the chunk. So with the hash and pipe symbol without with equals 70% and quotes resizes it just a little bit shrink it down you can also do out with down to 10% and it shrinks it down even smaller so these are ways to change the way that the plots look when you render. You can also add fig cap, fig dash cap as a chunk option. And this is the way to have a figure caption added to your figure so you can see here that it says figure one age distributions. And it's not going to show the code used to make the figure because I've set echo equals false. So all you're going to see is my beautiful age histogram plot and a numbered figure with this caption each distributions. And that's because I used a couple things that are very clever so it's not just fig dash cap. You can use a chunk label. And so chunk labels are ways for you to apply a label to an entire chunk of code. And so you can use hash pipe and then label. So here I'm labeling this chunk peak. You can also add it in the header right after the R so here is labeling a chunk option as peak. One thing that's a problem is if you have duplicate chunk labels you will get an error that looks like this error in the parse block duplicate label peak. So you want to make sure that you're using unique chunk labels, but if you do this, and you use chunk labels, then you can get some really nice benefits. And one thing that I like to think about is whether your chunks are house plants or crops. So essentially are they worth naming or not. And I think all of your chunks are probably worth naming you put a lot of work into them I think their house plants not crops. I think we need to name our chunks that makes it a lot easier for you as an author. A good chunk label that you can use lists over here so you can use my plot my dash plot my plot camel case, whatever you want to do. What I don't want to do is use underscores or spaces in your chunk labels are really frankly any other punctuation other than a dash. So my rule for thinking about chunk labels is to think kebabs not snakes. So you want words on a skewer, not anything with a lowercase underscore in it. So once you've got a chunk label, and you've got a figure that you want to be able to cross reference. You have to do a couple things to make it look nice and show up as a cross reference with Corto. So the first thing is that you need to label the chunk option with a label like that starts with fig dash so it has to be fig dash age or you know fig that fig dash site. And then you also want to use fig cap. And then you would be able to reference it in text and create a cross reference to figure that way. So this text right here at big dash age would create a valid cross reference here. So this is really nice this is something that previously was only enabled in book down and it's descendant packages but this is now available in single documents in Corto and projects in Corto so you can make books with cross references single documents with cross references. So to reiterate you have to have a labeled chunk that produces a plot. The plot label has to start with fig dash, and you have to use a figure caption using fig dash cap, not fig dot cap. But then you can have cross references for any format that you want to use. So I'm going to quickly show you how to do that. And I'm going to look at the time to 21. Actually, I'm Stefan, do you think I could quickly demo that when we come back from the break before you take on the next section to 21. Let's take nine minutes, and then we'll come back and do that and then segue over to workflows about four more minutes and then we'll get started. Okay, we have about a minute left I'm going to go ahead and start sharing my screen. Are you ready? Okay, so Stefan. I'm going to try to be do this as speedy as possible. So right at 230. No worries we don't need the 200 words per minute. Okay. Do you mind if I also show layouts real quick. All right. 230. So I'm going to do a quick demo to show you how to add figure captions and also maybe show you how things can go wrong with figure captions because I have definitely hit that as well. So I'm going to be in this O2 dash draft QMD file. This is visual editor mode is visual mode. And I'm going to scroll down to this figure that I have where it's, it says for the AEP CT plot I felt these recommendations, and then I have this include graphics call and so this is creating this figure to show you. I already does have a caption so it says BMJ 2016 paper and that's because I'm using that fake cap I'm going to switch that to fig dash cap, because that was my mistake. I'm having a hard time remembering my kebabs. So what I'm going to do now is I'm going to add a label and I'm going to add it to the chunk header but you could add it with that chunk option called label as well. So I'm going to call it big, it's called F3. I'm going to render that. You shouldn't see anything change or show up here. Just want to confirm. You do see it change because it adds figure one so it does start numbering it so once I add that fig dash as the label to the chunk that creates that graphic. It puts it into this numbered environment so instead of just having the caption that you're seeing here it's not updating in the IDE, but now it knows that it's a figure one. Okay, so that's kind of step one I have to have it labeled, and it has to have a label that starts with fig dash, and then in the text, if I want to reference that. I'm going to make sure I get it right the first time it's figure at shown and your at fig and then in the visual editor mode when you start typing, you do at fig and then you can see that what's popping up is it knows that this is in the figure environment so it's already giving me that option so if I had a few different figures in my document, I'd be able to pick them automatically I didn't have to do any kind of, you know slash anything I just started typing I did at fig. And I knew that mine, the name of my figure is fig dash F3. I'm going to save that render it, close these down because these are not updating. And there now you can see in the text it actually updated so I follow these recommendations shown in figure and then it's repetitive figure one. You can actually change the language for what that looks like but if you had that in a different portion of your document when people would click on it they'd be able to go immediately to the figure and be able to link back and forth. It's a really nice way to do figure cross references, but also figure numbering as well so even if you left this bit out and didn't need to refer to it in the figure say you're knitting to word or PDF. The nice thing is that you get automatically numbered figures for you so you don't have to do the in text references to get the benefit of numbered figures. I can't tell you how many times, I would write published research papers, and I'd submit them and I'd realize that I'd actually switched figure one and figure two, you know maybe my demographics table and my dropout table got switched, and then I'd have to go through the text of my article to change all the references in line to one of the figures so the nice thing about having those in the article even if you're not actually using the link the linking part because you're knitting to a output format that you know it's not really important. The nice thing is having that automated figure numbering and updating for you. I'm going to stop sharing my screen real quick, and share again the slides. Okay, so what we did was we opened up O2 draft type O, QMD. I thought I had done a find and replace. Sorry. Okay, so this is my image and I labeled that chunk with fig dash BMJ. I added a cross reference in the text like see figure at big dot dash BMJ. You can try this again with the counts by site code chunk that I added below. You know if you wanted to test it out what happens to see if your cross reference without the caption or if you use a chunk label that does not start with big dash I'll leave that as a choose your own adventure. Okay, so let me out real quick because this actually is something that I wish I had put earlier in the short course because this was one of the things that when I first saw the demo of it. It totally would have changed my attitude towards using literate programming for writing actual articles, and that is that you're able to actually combine figures into paneled figures, sub figures, very quickly and easily and you can do it across formats and so this is a kind of killer feature so in the document that you have right now I have sort of this lineup these were those four figures that I've created for you so when we rendered we were able to see all four of them stacked one on top of each other. So I'm going to show you how to do two things with those real quick one is tab sets these are only possible for HTML output, but it's a popular request in all of our different packages so I want to show you how to do that in Corto. So I'm going to take this syntax here, this is called like a pandoc div syntax these three, three colons, and then some kind of attribute in curly braces. So I'm going to quickly stop sharing the screen. Go to cloud. Okay. So what I'm going to do is I'm going to enter kind of down below here and I'm going to add a header says like my tabs that's creatively. And then I'm going to start oops. I can't do that. Okay. So it's dash panel tab sets. I'm going to do that in the visual editor mode it automatically sends you to this gray block and you can see that panel tab sets ends up up there. So I'm going to put two of my plots when do age underscore histogram and my adverse events percent plot. So I'm going to put age underscore histogram. And I'm going to give it a header age histogram so this isn't a code chunk even though it kind of looks like that there's no border to it. And, and this actually needs to be an actual chunk age histogram. Because otherwise it doesn't know that this is an our objects. Okay. And then I'm going to make this, I'm just going to add echo balls here. And then I'm going to say adverse events plot. I'm going to add a code chunk with ads and make it it's a PCT plots. I'm going to make it echo false. So when I render this what I'm expecting to see what I hope we see is that there's going to be a tab set object created and each of these was added with two hashtags so markdown level to header I'm going to switch to source mode real quick and show you. So it looks like I've got this panel tab sets with three colons and it's encasing these two code chunks, and each code chunk is going to be printing just a simple plot I'll run all the chunks ahead of it so you can see what this one is going to create. There it goes. I'm going to create the age histogram, this one is going to create the per cent plot. But when we render what we'd like to see as that they're going to be encased into tab sets and each of the tabs are going to be labeled by this level to markdown header. So let's render it, and then send it off to another window when it's done. So it should be at the bottom, and I added the anchor. So my tab sets is here and of course it's not working. This might be a spacing issue. Paul, what did I do wrong? You go back to the source. Yeah. Just the raw. Oops. Let's see. Oh, I need panel tabs that seems to be. Thank you. Yeah, no worries. And then the, okay, there we go. And then the space after the colon. Yes. Thank you. Okay. So do as I had in my slides, not as I actually did. I think that's it. I think it's just. The loading is dangerous. It's really hard. Cause I can't see my slides. I really need like a screen setup. So it's like a pop quiz every time I come into this. Oh, Crash. I'll listen. Okay. All are seeing all my typos. I'm going to nail it this time, you guys. Yes. It only took a few times. Alright. Oh, sorry. So age, Instagram, adverse events, plot. So that's really nice for if you wanna have HTML output. I love doing this for slides, especially when you're making slide presentations that makes things really compact. So you can have these tabs sets, but you can also do one of my favorite features, which is using layouts. And this is something that's really well documented, but I'm gonna show it to you anyway, because I think it's gonna blow your mind. So I'm going to add two hashtags to make a header here, and I'm just gonna stick in source pane so I can see what I'm doing a little bit more clearly. And then two hashtag, let's say layouts. So I'm gonna use my hash symbol and then the pipe symbol. And I believe it's layout in call two. Kabob. Yeah, kabob. I don't know why I like the dots so much. Okay, layout in call two. Okay, so now I'm going to pick two of my plots. I'm gonna pick these two, both the survival ones. So I'm gonna put those in there. And by adding this option, layout in call two, and let's do echo false too, because that would make sense. Lowercase false. Lowercase false. Right. I think I should respect both petition to accept uppercase, formally here, let's see. Yes, irrespective. So what it does is it puts it immediately into two columns. So this is like the magical way for you to get these subpanels and it actually works with figure captions as well. You can have sub figures with individual captions and then you can have a global caption across the whole paneled figure set. And this is really flexible. You saw me enter layout in call two. You can do in row, you can change this up. I'm going to add the AE percent plot here and change in call to three. So you can see that it goes up to three columns. This to me makes life so much easier for being able to format actual figures for publication. Go to layout, so you can see it did three in a row there. Now, the life-changing thing about this also is that it's not just for figures. You can do this for tables. You can do this with any art object that you'd like. So if you go back through these files, O2 draft revised actually does have some documents. It does have some objects in there like demo tab and follow-up tab and adverse events tab, which would be tables. You can try these things out and use that in call. Layout in call, I'm going to stop sharing my screen. I'm going to show you where that's documented so that you can rejoice at how easy it is for you to change the layouts. Simple, sound simple is very difficult. So if you go under using, it's under figures and layouts. And this leads you through a nice walkthrough. What I just showed you was what's under figure panels. So I showed you this layout in call, but I used code to do it. If you scroll down to computations, you'll see some examples with both Jupiter and Nitter plots. You can see it with tables as well. So you can see that I've made two columns of tables there. And you can also see how to make custom layouts by using numbers to specify the width of things. And those numbers are relative. It doesn't matter what numbering scheme you want. Like if you want things to add up to 100 or 100 or 200, it doesn't matter. And you can actually add white space in between. So this is a sub-paneled figure with two plots up top, a little white space in between, and then a plot on the bottom that's large. And then sort of buried underneath is a really great section also about block layout because we've illustrated this with figures and tables, but you can do this with any sort of block layout. So you can, similar to how we did the tab sets, you can use layout in call two with just plain markdown text as well. So you could have two lists, for example, that are side by side with each other. I can show you, I have to keep sharing my screen and re-sharing it. So as you can see, this is one of those features. Being able to lay things out was actually one of the things that used to take me a lot more time and work than it should have. And being able to do this programmatically actually would have been quite a relief. And it would have gotten me away, probably from my WYSIWYG editor if I had been able to very precisely control exactly how those looked because I was usually trying to get these things exactly lined up visually, but it's really actually hard when you're not scripting things to be able to make them look like that. So here you can see that list one and list two are in two columns. So you can do this with any content. You don't have to configure anything special for HTML. And I believe this also works for any of the other output formats as well. So I'm going to end there and let Stefan take over until 3.20 for a workflow. So Starry, you're first stealing your first 15 minutes, Stefan. Oh, that is quite all right. All right, so let me see if I can take over from here. Can you guys hear me? Yeah, cool. Okay, all right. My Zoom window went away. Am I sharing my screen? You are in fact, looks good. Okay, great. So hi, I'm Stefan. And as I mentioned earlier, I'm an assistant professor in lab medicine at CHOP in Philadelphia. And in my section of the workshop, we'll switch gears just a little bit and focus on workflows for collaborative clinical research that center on quarto or markdown or really a computational document which could be the quarto or markdown. And to give you a quick overview of this block, I was going to start out by discussing a quick case study and reproducible research for my own work. Then I want to discuss sort of a model for how a collaborative workflow might work when it's centered on a quarto or markdown document. Then in Meet the Stakeholders, we will look at a couple of the types of people that you surely will be involved with in your project and what are their interests and what are their concerns? And so this may not necessarily reflect exactly what you are in, but I'm gonna go over why I think this is, why I think this can be helpful. And then from this at the end, I'm gonna try to give some practical tips for setting up your project for success. So here's some background for the case study. So acute lymphoblastic leukemia or ALL is the most common cause of cancer in children. And although 80% of kids with ALL are cured with chemotherapy, 20% are refractory or they have a relapse. And relapse or refractory or RRLL, ALL is the leading cause of cancer, death and children with a five-year survival of only 10% after the second relapse. So CAR T cells, CAR T cells are a new therapy, a cellular therapy, and they can induce remission and more than 80%, probably closer to 90% of patients with relapse or refractory ALL, leading to long-term remission in 50 to 60% of patients. And this has been a really incredible game changer in cancer therapy. It came about nine years ago. So this is how CAR T cells work in here. I forgot to animate my slides. So I'm gonna just point at things. So here's how they work. You start by removing blood from the patients to get the T cells, which are part of the blood. Then back in the lab, and this is the kind of lab that I run here at CHOP, you genetically modify the T cells with a gene called the CAR G. And this is what allows it to attack the cancer cells. And then what you do, you grow up millions and billions of these CAR T cells, and then you infuse them back into the patient where you're taking them from. And so once in the patient, the CAR protein allows the CAR T cells to bind to antigens in the cancer cells and kill them. And so you can imagine that as this is happening and having all these immune cells simultaneously attacking the cancer, that can get your immune system into overdrive. And that's exactly what happens. So patients get really high fevers as they get breathing problems or blood pressure problems. And this can be life-threatening. And this is called cytokine release syndrome, or CRS. And in fact, CRS is the most important toxicity of CAR T cells. So this led us to the question of whether we can do anything to mitigate CRS after CAR T cell infusion, to reduce this kind of toxicity. And so we conducted a prospective clinical trial because it was not a randomized trial, but it was a prospective trial with 70 patients, 15 of whom had high tumor burden, which is the most important predictors for severe CRS. You don't really get bad CRS if your tumor burden at the time of infusion isn't high. So the intervention was to give a drug called tosylismam. And this is a drug that blocks the IL-6 inflammatory molecule, which is one of the most important inflammatory molecules in CRS. And it blocks it from binding its receptor. And we would give this at the time of the patient spiking a fever. And we only give this to a high tumor burden patients, because they're the ones at risk for CRS, for bad CRS. And so the predefined study endpoint was that we wanted to see life-threatening, or grade four CRS, in fewer than five, fewer than or equal than five out of 15 treated patients with high tumor burden. And then as the secondary endpoints included any negative effects of the intervention, of the tosy intervention on the efficacy of CAR T cells, you don't want it to decrease the efficacy of the CAR T cells, right, in an anti-inflammatory drug. So we also looked at permission rate, cell expansion, persistence. And so we did the entire analysis in our markdown. And so our markdown is really similar to quarter, which I think everybody here is very aware of. So this table shows that we met a prospectively identified endpoint of less than five out of 15 patients with grade four CRS in the treatment group. And this was made with the GT and GT summary packages. So there's beautiful tables you can make with GT summer. And these graphs here show that CAR T cells expanded normally inside of the patients in the intervention group, which is red, and that persistence of CAR T cells was not affected. So we're not creating any problems with the intervention. And so we use GG plot, of course, and survival of the serve minor package, which for some reason I don't understand doesn't have a hex tag. We also created an online only reproducibility supplement to the main script in which we published all of the code for the analysis. Now I want to demo this real quick, but before I do so, I wanted to mention this, for this we used the distilled package and built some additional custom functionality to create nice looking data dictionaries and de-identified data that I hope to find the time in the next couple of months to roll into a package and make available. But let's say I'm a reviewer and I see some predictive models and I see a predictive model in the paper. So let me actually try to switch to that right now. And I want to better understand that predictive model. So where's my zoom window over here? It's okay. Oh, here we go. So I'm sharing my screen and now I have to do a new share. Okay. All right, so this is part of the paper and there's some predictive models in here. So there's a logistic regression. Let's say I'm the reviewer and I read this paper and it's like, oh, this is pretty cool. But how did they do this logistic regression? So I can go to this, I can go to the reproducibility supplement and which I'm gonna show in a second in this other share. Can you guys see this, yeah? Okay. So this reproducibility supplement is an R Markdown document that was taken through Distill. And you can see that I have an introduction, analysis, data dictionaries and description of a computational environment. But now I'm specifically interested in, oh, was it figure, it was figure S2. So I'm just gonna look for S2 here. This takes me to table S2. Everyone sees figure S2. So I open up this triangle. There's a bunch of code here. So I'm gonna scroll down. Oh, okay. Yeah, that's the figure I'm talking about here. So, and then we're looking for something that says model C, okay, the logistic regression. So I can just look for model. And I see, oh, there's a model C predict function. And so it looks like it's a function that takes the levels of three cytokines as arguments and okay, here's the formula that they use to do this logistic regression. So I think what this, I really believe that such reproducibility supplement is a really useful way to publish analysis code alongside a medical journal article. So get back to my slides here. Okay. All right, so this is all well and good. But how can you make this happen, this kind of thing happened for your own project? So you can see that the supplement was actually pretty long. There was lots and lots of code and analysis went in there. And so how do you make this happen for your own project? And so for the remainder of my time in this block, or at least until 20 after, I wanna focus not so much on technical assets, but more on human and organizational factors that I found to be important and useful for doing this kind of reproducible clinical research, specifically in the context of an academic hospital at the U in the United States. So I hope this is gonna be helpful. So to do this, let's first consider the life cycle of the data. So what I'm proposing here is a slight modification of Hadley and Garrett's model in R for data science that many of you I'm sure have seen before. So I would begin by defining the objectives of the project and identify possible data sources that you'll need to meet those goals. Then you import the data into your favorite analytics platform, which of course is R and then you tidy the data by which I mean, we're shaping it for analysis and also performing some data cleaning and quality control. Usually the next step will be to transform the data in some way so that you can perform visualization and modeling, which are the two main engines of knowledge generation. This sort of typical data analysis, transforming visualizing and modeling happens not only once, but over and over. And this is what's called the understand cycle. And it highlights the iterative nature of data analysis. And the final critical piece is communication of the insights in such a way that they eventually get turned into action. And it could be through a publication or maybe a reproducibility supplement like this or a report to a stakeholder or some other way that the analysis can be used for decision making. And in reality, as you're working on this project the workflow flows in both directions back and forth and so you need to go back to a previous stage but when you actually want to reproduce the analysis this is kind of the, there's a unidirectional flow of how you want the data to flow through the system. So during development, it definitely goes back and forth. So now data analysis often happens as a collaborative effort. They usually have some sort of a subject matter expert. In medicine that's often a medical doctor but could also be a nurse or a pharmacist or a psychologist or bench researcher. And you may have a technical expert. And this could be an analyst, a biostatistician or a data scientist or really there's many other possibilities here. And sometimes one person plays both of these roles and that does make things simpler but there are real limits to what you can accomplish on your own. So I'm going to assume that we're going to have two collaborators, one technical and one a subject matter expert because I think this is a common scenario in the academic setting. And maybe if you're in this workshop you might identify more strongly with one with either one or the two in your work. So the subject matter expert might be in charge of the objectives and in a traditional workflow they might come up with a project and do the data pull and then they will send this to the technical expert and it's usually an Excel sheet, right? But what this does is and then your technical expert will work on this and they will tightly transform a visualized model. But what this does is it effectively creates a silo that the subject matter expert has no insight into. And this can cause errors that will be hard to spot because we're using Excel which doesn't record user interactions isn't reproducible. So the alternative that I would like to propose is centered on the idea that you have a central either one computational document or a set of computational documents that are central and shared and they would be written in quartile or more. And the idea is that you work on building the analysis together but do you build the analysis? So it's always in a state where you can rerun the entire thing to regenerate all of the analytic objects. I'll go into sort of a suggested set of files to do this later on. But so basically this is sort of the idea of this reproducible workflow that would allow closer collaboration, more transparency, more confidence in the analysis. And I think it could also save time by catching data problems earlier and more rapidly iterating through the understand cycle. And it wouldn't make it possible to publish the code at the end in a way that is useful to others such as in the reproducibility software. So now that we have looked at this goal, let's take a look at the stakeholders. So to figure out a workflow that will work and that produces the result that's useful for everyone involved, I think it's a great idea to figure out who you're working with. And this is really similar to the idea that as a first step of teaching a workshop you wanna think about who you're actually wanting to help. And I think going through a similar exercise can be useful here as well. So I'll introduce you to Ava who's our subject meta-expert and project lead and Ben, the biostatistician. And Ava and Ben are the two that will be doing most of the actual work of the project and will be kind of the owners and editors of the quarto or the R Markdown document. But then there's also Peter, the principal investigator or senior clinical researcher. There's often somebody like this. And then we'll also have Riva, who is an external researcher in the same field and peer review. And what I'm proposing is that we should look at all of these people to think about the things that we should consider as we're billing on the houses. So all right, so let's see if I can do this in 20 minutes. So let's meet Ava. So Ava went to med school on the US East Coast and just finished a pediatrics residency at a major research hospital. So she's currently a hematology oncology fellow and she's about to start her first two research years. And she really enjoys reading and learning about tech. So Ava is working with Peter and she's embarking on a project to analyze and write up one of the clinical trials that has been going on in her department and maybe it's a CAR T cell clinical trial. Who knows, right? So she would like to do this by writing a reproducible report with R. She doesn't know about quarter, so she's thinking R mark down, but you know, like, so she does wanna use R that. So Ava feels comfortable with Excel for data collection analysis, but she would like to learn R more, especially because she thinks data is great with it and also reproducibility. And she, last year she was at R Medicine. She took the 101 course and she started reading R for data science, although now it's sitting in the shelf gathering dust. And she's been using a little bit of R markdown in GDplot for a few small personal projects, but hasn't really found a way into using this stuff for her research. So here are some resources and barriers that I would consider for her. So Ava is enthusiastic. She's eager to learn new things. And she now has two years coming up of relatively protected time to do research as part of her fellowship. So she can devote multiple hours per week to learning and doing this kind of work. And she has the support of Peter, the PI, and Ben, the biostatistician for this project. On the other hand, some barriers are that, you know, coming out of many years of, nah, that was the next thing, but this is the first time she's worked in a clinical trial, so all of this is new to her and so it's hard for her to know where to start. Also coming out of many years of pre-med and medical training, which didn't include any programming or statistics, which I think is ridiculous. So learning R is very new and very different for her. So she's certainly good at learning and memorizing things, but some of the mental models and that make it easy to learn programming which just weren't exercised in her education. So, okay, so let's meet Ben. Ben grew up in Turkey. So he came to the US to get a PhD in biostats about three years ago and then has been collaborating and since then has worked in the old college department, collaborating with several of the physician investigators there. So he's not entirely new to this, but also hasn't been there for such a long time. So he plans to collaborate with Ava on the clinical trial analysis. And just like her, you know, they had a quick chat and both agreed that it would be really cool to do this in a reproducible fashion, which would be different for both of them and new and to have everything in a reproducible report. So Ben is pretty solid with SAS and R, both of which he learned during his PhD. But when he was R, that usually wide strips and I recently started using R Markdown as well. But usually, you know, he operates by the traditional workflow where people send him Excel sheets and then he runs us in analysis files which he has in his computer and many, many project folders. And then he only sends word documents or Excel sheets back to his collaborates and they're usually not interested in this code. So Ben is excited to work with Ava on a new and improved workflow for collaboration. So that's a positive thing. And also he's getting paid to do this kind of work. And so he gets to devote a number of hours per week to this. So this is great. But on the other hand, some of the barriers are that he doesn't have experience with actually sharing and collaborating, writing code with other people. He always writes it for himself and he's the only person who has to look at it later on. And he's also a little bit frustrated because even though a lot of people talk about reproducibility at places like R Medicine, it's tough to find a set of instructions of how you get going and there always seems to be something new and better. So that's frustrating and difficult. Okay, so these are the two main players. Let's go to the supporting characters. So this is Peter, the principal investigator, head of oncology and the PI of the trial that Ava is tasked to analyze and write up. He thinks it's great that Ava and Ben is trying something new and cool. He thinks it might lead to reduced rework. He doesn't know any R or any other program and that interest has absolutely no desire or interest in learning any of that stuff. And so here are some of his concerns. And I was actually gonna ask you guys to suggest some things that Peter might be concerned about. So maybe we'll take a minute on that. So why don't you just write in the chat one thing that you think Peter might be concerned about as he is, it will take too long. Okay, this is great Miguel. Yeah, it'll take too long, right? So that's, so writing code for all the analysis will take too long, anybody else? All right, I have two more. So I think one concern that sometimes, especially from sort of the senior collaborators come that if you share your clinical data freely, you might leak sensitive information. And then there's also this idea of you, if you share your code in a public place and your name is attached to it, but you don't understand anything about programming, it can be difficult to be put in the position where you really have to defend something you don't understand. So these are maybe some of Peter's concerns that we may wanna think about as we're building things. I'm not saying that we can completely address them, but this is something to think about. All right, let's talk about Riva real quick. She is a physician scientist at a peer institution. She's also an oncologist and she's very interested in the clinical trial and she is interested in this predictive model that's gonna be described in the paper, which she wants to validate in depth for her own research. And that may be an opportunity for future collaboration between the two senators. So that's very cool to make that accessible. And also, looking at being a peer reviewer, all that is being equal, she would rate a paper, the same exact paper more highly, if it was very clear that the analytic code and data are published on the same with the paper, then if that stuff is hidden away. So, these are our personas, they're of course made up and mixed and they may not at all correlate to the specific teams that you work in. But I think there are some useful thoughts in here. I definitely found it useful to think through this. But now I wanted to, for the last few minutes that I have, and I think I'll actually make it on time. From this discussion of personas to come, to put together, well, to suggest a few tips for how you can operationalize your project and get around some of these issues and actually use them in your favor. So I think the major hurdle that we encountered in here is that both Ben and Ava are doing this kind of for the first time and they don't have a lot of guidance. And so they may not feel super confident in doing the stuff and having something presenting things to the PI. And so I think it's worth thinking about how do you break down the huge task of cleaning, many data tables from our clinical trial and creating a dozen of more figures and tables and doing all of that collaboratively and reproducibly. And so in the last part of my block, I wanna offer some tips that I think have been useful. They have been aha moments for myself. I don't wanna share those and I hope this is gonna be useful. So one of the things that I see people do often that start a new project with me now that I'm becoming a more senior myself, people, students are starting to work with me or fellows are starting to work with me is that they feel like they need to do a lot of work upfront. So they have something to show for and then they get overwhelmed. I'm having trouble hearing you. Oh yeah, that's okay Siri. So then they get overwhelmed and then they give up and this isn't very painful. It really doesn't have to be that way. So I think a really useful thing to do early on is to come up with and commit to a process that allows you to bite off one little piece at a time and then also make sure that process works for everybody. And I'm not presenting anything that's like crazy here. So this may sound super trivial, but it's so important. You get to set up a regular meeting about your project, right? Weekly is great. Every other week is okay. But I think the longer intervals, whenever on the early Tozi project we were in the longer intervals, things kind of ground to a halt. So I'm looking really looking for weekly or bi-weekly meetings for my projects that I'm using this workflow for and make sure everybody who is a stake can join and that includes Peter the PI because their perspective, even if you don't have anything great to show for since your last meeting, no new table, their perspective can be really valuable and save you from going down a rabbit hole. So if you have somebody senior who is part of the project, don't take advantage of them. So next thing is write meeting minutes and review them at the beginning of the meeting. This is important for continuity to also to hold everyone accountable. Did everybody accomplish what they said they would? If not, why not? So they may be taking on too much work, but it could also be that there's some kind of a blocker that needs to be resolved. You should, during the meeting, you should also review any new analytic work, such as new tables or figures that have been worked on since the last meeting. And here it's great to have our studio with your quarter document open on a shared screen. So you can review the code, but not only review the code, but also quickly answer simple, what if kind of scenarios, what if I only look at the top 10 of these or and so on and so forth. So here the question should be, does it show what you expect? Does it fit with previous data? Should there be any follow up analysis? And this is where the understand cycle is happening. This is where you're engaging the understand cycle, right? This is at this meeting and each one of your meetings can be the iteration. So next you should discuss, you should definitely discuss any work that you as a group intend to accomplish until the next meeting. And that could be data capture, it could be data cleaning, creating a new figure or table or revising a figure or table or learning about distilled or something. You know, something that's a deliverable. And so for new tables, I think really the practice of creating a skeleton table is very useful. A skeleton table is a table that only has your categories but doesn't have any data on it. So this is super useful. And for figures, I think it's an awesome idea to sketch out what do you think the figure will look like with this data kind of drawn in on a piece of paper because both of those things allow you to work backwards from an idea of the final product, which is something that you really want to have in your paper to the data and the methods that you'll need to employ to get there. So this is a very effective way to, I think, to move forward. Finally, so you want to have a minute. So somebody should be writing meeting minutes and they should write down each person's goal and follow-ups. I think a good practice for building sort of a sense of joint responsibility for getting that stuff done is to write minutes collaboratively. So for example, Ava could draft them and then Ben edged them to send them out to it. So here's, for example, so I think writing goals like the example here, which was an example from our early TOSI trial can be super helpful to keep things on track. So you can see it's not a terribly long list of things. We have the persons who are supposed to do something highlighted and so really realistically what happens is a couple of hours before the next meeting everybody kind of does their stuff, but you're actually making progress every week. So that's cool. All right, let's talk about files next. How do you organize? Where was that file? I know that file was some years ago. So I think it's really useful to have strong conventions about what goes where and also separating the concerns that are in the data analytic model, which is cleaning data, analyzing data and reporting data, right? So breaking these concerns into separate, into separate or mark donors, this is really should really say QMDS of today, documents that do this kind of thing. So here's a suggested organizational files and just as inspired by the projects package, which I find useful, although it's much simplified from this because I found that really breaking things into these files and letting them kind of do everything myself. I found that more useful for myself. So some of the ideas here are that the numbers indicate the direction of your data flow. They don't necessarily indicate the direction of how you kind of build the analysis. You might have a ton of stuff in analysis already when you realize, ah, I need to do more data work. So you just tack that on. So really O1 data work and O2 analysis become kind of your lab notebook. They're not necessarily presentation grade documents, but they have all the code and they're supposed to be reproducible so that you can delete your O2 data tidy and O3 figures file, which get populated by these documents and everything should get regenerated. So essentially the job of getting your data cleaned up, verified in a tidy data set for analysis by handled O1 data by O1 data work. So this will load data from a database or any files that are in O1 data raw and creates files in O2 data tidy. And those could be, you know, a CSV files or essentially the ideas to have tidy data frames in there, maybe as an RDS or maybe as a CSV file, maybe both, I found it useful to actually have both for different reasons. And then the job of O2 analysis will be to actually take that data from O2 data tidy and create figures in tables and drop them into O3 figures. And you can save them as PNGs as various different formats, but that's where you'll have them available. And if you work with version control, you can actually use, take advantage of nice visual diffs to see how your analyses have changed after you've made a change, which is pretty cool. So another idea is that O3 report in my workflow, I've actually just made this a clean version of O2 analysis for creating the reproducible report, which, so this is just, this in my case was a distilled, these O1 and O2 were regular R HTML notebooks and O3 was a distilled, which was set up to deploy directly to a GitHub page. So these are some suggestions. This is work for me. I think there might be some useful ideas to take from this. I have no idea what the core to a project for distilled is gonna look like. I hope it's gonna look a little bit like this. Okay, so let's talk about style. So make your code easy to read, okay? So for a reproducible report, readability is way more important than performance. And because people that are interested in reading your code are probably for the most part not professional programmers and that may include you six months from the day. So really like read and stick to the tidy verse style guide. Is this is very useful and I kind of make everybody who works with me on an R project, it's not very long and it makes code review if that's something that you wanna do that you can do much easier. Also use pipes. They're great to emphasize sequence of actions and they make intuitive sense to Peter who doesn't know R. The pipes are awesome. Also the deep layer of verbs, those things make sense. So one of the really cool things about R I think is that they can using a dramatic tidy R is really helps the non-programmers feel more confident that they understand what's going on. Which of course they don't really, but it makes it easier to kind of explain to your PI how exactly you're doing something. Which can, if you can't do that can really grind things down. Anyway, this is one of my favorite XKCD comics. XKCD comics. This is Git attracts collaborative work and projects through a beautiful distributed graph theory tree model. Cool. How do we use it? No idea. Just memorize these shell commands type into sync up. And if you get errors, save your work elsewhere, delete the project, download a fresh copy. And this is how I use Git. This is how I use this Git. So note on version control. So if you've never used a central version control system like GitHub before, I think you should look into it. So it's similar to Dropbox, GitHub allows you to keep a central copy of all of your work and be sort of a disaster backup that you really should always have. But the other thing that really can do for you in this kind of a project when you're going through iterations is that you can do code with you. And you can actually do with the visual, with visual diffs, you can actually review changes in your analytic objects, which can sometimes be really interesting when you see that something that more than once something that I didn't think should have changed and that allowed me to find a subtle bug in my analytic code because a Kaplan-Meier looked just a teeny tiny a little bit different and I realized I dropped three patients. So this kind of stuff can happen and you can really, so it can be really useful. But so the RStudio IDE supports Git and GitHub and also GitHub Enterprise, which is your on-premise version of GitHub and Jenny Brine's book Happy Git with R is awesome and it's a great introduction. I do, however, think that there are legit reasons for not using a central version control system when you're working on clinical trials. And this is the case, I think if you don't have an on-premise instance of GitHub Enterprise or GitLab, it'll really allow you to store sensitive information. Also, I do think that there is a learning curve to this additional overhead. So you may not wanna learn R and Quarto and GitHub all at the same time. So I get it. If you do not opt to go with GitHub for your project, make sure that still doesn't get you out of not backing up your data. You definitely wanna use something like Dropbox or OneDrive or whatever your organization supports for disaster recovery. And then the other thing that I really recommend is that to stick to a convention for including the version of the file and because you're gonna have to do manual versioning of your data. And so here I really recommend going with the ISO 8601 date. So ISO 8601 is a lexicographic date, which means that essentially it means that when you sort things by name, you actually sort things by the correct date. So and then maybe also wanna tag things by like whoever last did this. And I think this works. You can make this, you don't need Git and GitHub, but if you have the time, I suggest you look into it. And if you have the option, you should look into it. So doing quick recap of this, of the stuff we talked about. So the, and I know we're three minutes over, but I'm almost done. So the reproducibility supplement, that's a mechanism to usefully publish and now it's called alongside a journal article, including a medical journal article, which doesn't usually publish code. A reproducible collaborative workflow that centers on a shared computational document like Quarto or Markdown can really increase the confidence in the validity of the analysis and have a lot of other benefits as well. So then we looked at the stakeholders whose needs, barriers and concerns you may wanna consider. And they include subject matter expert, technical expert, a principal investigator or senior clinical researcher and a peer researcher reviewer. And then to operationalize your research, I recommend you adopt an iterative workflow with regular meetings, organize your files write easily write a readable code and using virtual control. All right. Thank you. That's all that I have for this session. It's 324. I think we should take one minute real quick. Paul has a small announcement for the link, right Paul? We're gonna. Right. Yes. And then we'll take a five minute break. Right. So the next section is gonna be me showing off a few things of what you can do with Quarto. And then during that time we're gonna be collecting questions through this link here. So we're gonna take a break, go ahead and put your questions in there. We'll come back, I'll show off some stuff and then we'll get to your questions afterwards. So the questions can be anonymous if you'd prefer or you can ask in the Zoom chat. We'll stay and hang out and listen to all your questions and try to answer them. The nice thing about Slido is that you can see if other people have asked your same question and you can up or down vote it. So that will allow us to better triage some of the most important questions or the most common questions that we can find. So take this time to take a quick break. We'll come back right at 3.30. But if you do have a question, a burning question you really want answered, maybe try hooking up with Slido. The link is in the chat. It's also on our short course website where the question mark icon is. So we'll see you guys in five minutes, 3.30. So Stephan, are you hanging out? Yeah. Or are you taking a break? Okay. Hi. Then I dork out with you for a second. Okay, so I had so many notes while you were talking. I thought one of the things that I showed in, I guess it was in the O2 documents was being able to source an R file and kind of what you were saying about the workflows for the different O1 data, I think you had the data work, O2 analysis and then O3 is like the supplement. I think being able to source a separate R script is really helpful for those later ones because you often find that those earlier, RMD or QMD documents that you're working with have maybe a lot of code and it's really hard to read. And once you get to a certain point, you've sort of gotten what you needed out of that document out. So I really liked that you talked about the separate RMD documents. So I wanted to point out that people can actually use that trick of doing source and have an R script that's external. You can also use child documents and that are the same way you do with our markdown in a Porto document. And that's another way to kind of reuse work that you've already done, but have new R markdown documents that are sort of like one level of abstraction hire where there may be just like they're pulling code from different places and executing it and getting the output, but they're not necessarily the single source for all the code. And that's a really nice place to know that you can get to when you're having those meetings and you're moving on from the meetings where you're talking about wrangling to like the final paper, you're not like, oh gosh, here's all that GG plot code that I don't need. Or here's like a really, really long like sequence for analysis just to get like one number out. So that's kind of nice for the workflow. So I wanted to call people's attention to source in the O2 documents. And then also looking up in our child documents as ways to have that so that you're not focused on making a clean, clean, clean document. You're able to like reuse the stuff from other places. The other thing I wanted to mention was exported figures. I'll show that maybe right at 3.30 when people come back because that's another thing that I found that meetings was really helpful was like, you know, there's the report, but then in a meeting sometimes you're wanting to like pull up like, oh, what was that GG plot where you had like the violin plots versus the scatter plot versus the box plots? Like let's pull those up really quickly. So I always found it really useful. And one nice thing about Quarto is that it automatically exports all those figures. So I'll show that really quickly when we come back. And then the other thing I was thinking is that, you know, it's kind of nice, like those intermediate documents, those are really nice because you don't have to worry about output format as much. You can just like use HTML and make it really useful for yourself and share it, even just like sharing your screen or publishing it. And that's where like tab sets and stuff like that is really nice because I used to have like, it was like, I think of it like the eye doctor test. So like we'd have a bunch of different figures and it'd be like, which one do you like better? Figure one or figure one A, figure one B, or figure one C, you know? And like with tab sets, you can make it so that you don't have like a dump of a lot of stuff and these really long documents. You can have it like side by side and let people be able to explore that. So it's kind of nice to think about like, I actually feel like the most high-powered stuff that I did with our markdown as a scientist was like the stuff before the final paper. Like all those little pieces that go into it. Yeah, I agree with your point about like creating all these additional documents because as I mentioned, you can use them for various things like pulling them up together or maybe putting them on a PowerPoint slide to show them next side to side. Actually, one thing that I was planning to mention is that when we were doing this early toasty thing, I was working with a very senior PI and he was not gonna learn R. And so we actually just really looked at PowerPoint slides with the figures and that worked well because one thing that we remember is that you really wanna reduce the cognitive load for the, especially for the non-technical people as you're reviewing stuff. And so this is where it can be a real downside to kind of scroll wildly through a markdown document. And so for that reason, I think it's really a good idea to kind of spit out the figures when you come to figure review, maybe just look at a PowerPoint slide that has nothing on it except for the figure. Yeah, I totally agree. So we're back at 3.30, so I'll share real quick what we were just chatting about. So I'll share my screen. I think you have to turn off your screen share. There we go. Let's see. So welcome back everyone. I was gonna real quick share some thoughts that I had after Stefan's great redux of like how to collaborate with people who don't use R which I think is most people's lives if you're doing medical research. So I was gonna share that, one trick that might come in handy when Stefan was describing those workflows of having like multiple RMD documents for different stages of where you're at and where you're at with your collaborators is to eventually move to a place where, once you've maybe extracted all the value you have out of a certain RMD document, like you've figured out which figures you wanna show. It's really nice to have external files, like you can have an external R file that creates all the figures for you and external RMD files even like QMD files as well that create the different things. So those are the things that maybe hold most of the code at a certain point. And as you get kind of further along in your research project, you can do things like either source and R script like I did here. And that allows you to just kind of like plug and play and plug in like the graphics that you've made in where you need them to be. And you can do this with an R script, like with source, or you can also do it with child documents using NIDR. So you can actually pull in different RMD documents. I'm not sure exactly how it works with Cortot yet, but that's something that I'll test and get back to you guys on. But I recently did this for a paper on the Palmer Penguins at Alson Horse and I did where we basically had a ton of figures and we created a separate file that created all the figures. And then the actual paper itself was more of like, as Tom talks about like the control panel, like it was my big combination document. And that's for a certain stage in the analysis, but it's good to know how to do it. So you have an example here of how to actually do that, which is a really good workflow example. And then the other thing that I wanted to mention that might be relevant again for what Stefan was talking about was one of the things that changed my working life as a scientist was being able to pull out the figures from an RMD document and being able to share them with other people really quickly. So here, because I've already rendered this O2 draft, it created this subfolder called O2-draft underscore files. And inside there, you'll see figure-html. And what you'll find is there is all the PNGs for every single figure that we made. So you can actually open them up and explore them. And if you name your chunks that create those, something intelligible, this will be a lot more intelligible for you. So the reason it says unnamed chunk 4, 1, 4, 2, 4, 3, is because those were all unnamed chunks. I didn't name those chunks that created those figures. But if you use knit or chunk labels, what automatically gets exported when you click render is something more like this, counts by site-1. And that's because I had that one labeled here. So that one had a figure label attached to it. And that was really nice for me to be able to attend a meeting and have people be able to really quickly see a bunch of figures all at once, and they didn't necessarily want to scroll through a document. So I just wanted to share those two things real quick before we hop into the Q&A. So let's give a few minutes for people to ask their questions. And I'm gonna ask Paul maybe to pull up what I asked him to prepare, which was the fireworks display at the end. So to kind of show you guys all the things that you can do with Korto that we jam packed this full, way too full actually of information about Korto. But Paul is gonna real quick do an overview of some more stuff that you can do. And then that'll give you guys time to ask some questions in the Slido and we'll tune back into there. Oh, can we save a copy of the project in our RStudio Cloud Workspace? Paul, do you know how to do that? Can we see? No. Let me share my screen real quick then. Yeah, I'll look into it. I'll just do it really, really quick. Yeah, go ahead. Okay. I think you have to stop sharing your screen and all. I sure do. There you go. Okay, I'm gonna do it real quick. Okay, so back in our studio cloud, the way you do this is to go into each of those projects and it's actually really, really nice. You click on more and you're gonna want to, here, let's go up one more and let's go up to up here. If you click on the up arrow, you get to this point and you get to just click on project altogether. And then you do more export and then I'm gonna call this one like O2 authoring. It should download a zip file with all the folders in it. So the trick was you see this view, click on the double dots to go up one and then check that box, more export. And then it should, yeah, it automatically downloads with all the files in there. So that's how you get the files out. And then I think Paul does have some helpers for getting you installed with Corto locally. So I'm gonna start sharing my screen now, okay. All right, so before I get started with like the wrap up of the presentation, I wanna plug Tom's presentation of our markdown. Tom, if you're around, did you wanna say a few words about this? Yeah, so a lot of what Allison was talking about with like sourcing external files and using our markdown or in the future, Corto and QMD to do things. This presentation is really just about like motivating you with a couple of examples of how to do said things. So sourcing files, using child documents, parametrization of our markdown, that kind of thing. There'll be a lot of code examples in GitHub. So if you feel like learning a bit more tomorrow, feel free to stop by. Awesome, and like Tom was saying, there's gonna be a lot of things that'll carry over to Corto. So this is definitely a great place to pick up some cool tips. All right, so moving on to the wrap up. I made this little slideshow presentation. There's really not much in here, link in the chat. So we're gonna talk about installing Corto on your own machine, Corto support for Python and other editing programs in an example project site. And I'm gonna set a timer for 10 minutes and hold myself to it. So if we go to the Corto example site here, this is, so this site is also built with Corto. And on the introduction here, there's some instructions on how to install it locally. The gist of it is that you need to download the Corto CLI from the Corto GitHub. And I put a link in the description here. If you follow this link, it takes you to the current, the latest version of Corto and you just find the version that's, that matches your OS. Go ahead and download that, open up the installer, put on your system. After that, if you wanna work with RStudio like we did today, like Allison mentioned, we need to get the daily, the RStudio daily build. So if you just follow this link here, it takes you to the RStudio daily place. Again, find the one that fits your OS, install it. And once you have those two set up and good to go, you're able to do the stuff that we did on the cloud on your local machine. But part of what makes Corto so cool is that it has support for like other programming languages and other editing modes. So I'm gonna have to switch out the share screen. Oh, I didn't think about how to do this here. So yeah, so like I said, Python and Corto has support for Python and Jupyter. So let's see if I could get it going. You might not see it right away, but hopefully it pops up afterwards. So in the background, I'm starting a Jupyter notebook. Can you guys see that? Yep, it's showing up. All right, cool. And then I'm gonna open up another window for Corto's serve. So on this window here, I'm gonna open up a Jupyter notebook. Okay, okay, so open up the Jupyter notebook here. And so if you're familiar with Python, you should be familiar with what this looks like. And what I wanna point out in particular is that what makes Corto special with interacting with Jupyter notebooks is that all those chunk options that we learned for Nitter in our markdown actually work in Jupyter notebooks as well. So this is a Ipy and B document. And you won't see the actual like thing, the actual captions and labels show up in the Python document, but when it enters in Corto, you'll actually see all that stuff pop up here. So this window here is a live preview of the project that I made in Corto. So changes I'm making that document are gonna show up here. So for instance, if we look at this figure here, I labeled it fig polar, and I said the caption is a straight line projected in the polar coordinates. I also put down a citation or a cross-reference here. And so the Jupyter notebook, I'm just for, I don't have to know anything about, you know, using our studio, I don't know. I thought I don't have to know anything about using R, but if I know Jupyter notebooks and I know Corto, then I can get outputs just like this and just like you're expecting with a nitter and R markdown. So you see that it has the labeled figure, it has the cross-referencing link to the figure itself, and it respects the code chunk orders I put here. So for another example of this, here's like a huge brick of text, a huge brick of functions and test documentation that I made. And most people don't care about this, but again, because Corto knows about the code chunk options and works with Jupyter notebooks, I can just add this stuff here at the top of the cell and it's gonna render just like this here with this nice code folding button with the code summary description here. So I could click and open and close it. And so like something else that's really cool about this when we talk about collaborative workflows with people in our lab, you know, it's great if everyone works in R, but a lot of times people don't and sometimes there's a holdout, like there's people that want to just work in Python and they don't care about learning R. And so previously in the past you'd have like people, you know, make a Jupyter notebook and try to host it online and it looks awful and it's hard to style it. At any of them like the R Markdown documents and they look better, but the styles are completely different. What's so great about using Corto with these kind of setups is that Corto renders all the documents and it applies the same styling to all the documents. And so it all looks like one cohesive whole. So that brings me to the projects of Corto. So blah, blah, blah, blah, yeah, Corto projects. So briefly, Corto project is a way to combine all your documents together into one sort of whole. So typically you'll see like on the Corto org website, this is a Corto project known as a site. So, you know, you have all your different documents here and you tell Corto how to unite them all with the Corto.yaml file. Another version of this is Corto books, which is what this site is. So if we go back to the example site that I put somewhere up here. Yeah, so this Corto example site, this is again is a Corto book project. So what's special about Corto projects first off is that again, it tells Corto that all these documents belong together. It tells Corto how to render them together in one project. It tells them how to fix navigation between the two. So for instance, like this page is one separate Corto document and at the bottom there's links to get to the next page. There's links that it gets to the next page again. There's a search bar here. So now I can search within the whole project for different things. So I could say for instance, best blast hit. So I wrote some stuff on, you know doing statistics on a best blast hit level. So it takes me right to that section and I could go and check it out from there. But what's really cool about a book project is that we can combine all these different documents together into like one output document. So, you know, talking about that collaborative workflow again, let's say that, you know one person's responsible for taking part of this analysis, this person's trying to make figures, whatever. You know, you can have each people work within their own document and upload it to the site and then someone responsible for rendering it can go ahead and render the whole site, push it up to GitHub or wherever you're hosting your site. And there'll be this button here that says you can download it as a document. So let's go ahead and click that and see what that looks like. And maybe you can't see the word document. I'll just switch to that real quick. Do, do, do, do, do. Yeah, so I went ahead and just I'll put everything to a single word document. And so these, so this word documents made of four or five different Cordo Markdown documents and it's put it all into one thing. And so people interested in your work can, you know, look at the individual Cordo Markdown pages or they could just download it entirely as a paper. So personally what I'm doing is I'm working on a new paper for my PhD research. And so I have, I'm gonna have a page for literature review, a page for methods, a page for, you know discussion and results and all that stuff. And so this is a really cool way to work on it individually at a time but when it's ready to, ready for, you know publishing, I go and just download as a PDF and send it off. And I think that's it. Okay, are you gonna triage our Q&A, Paul? Let's do it. All right, let's check out Slido, see if we've gotten any questions so far. Maybe before we go there, there's a quick survey for the course that I would like to, I'm gonna post this in the chat. And it shouldn't take you more than two minutes to finish. So I really appreciate everybody's feedback on the workshop. And so look at that at your own later. Yes, please. And then let's go to Slido. All right, the top most avoided question is from anonymous. How do I get my PI to buy into this? He doesn't think it's worth the time. Oh, that's fascinating. Anybody else wanna take this or shall I? Well, my first instinct is to say that, you know the time you put it up front is worth it because when you're scrambling to get everything together for publishing, you wanna be able to just point and click something and point, you wanna have been doing it already. Yeah, I think for me, I think it's a graduate student, you can kind of do it without necessarily asking for permission ahead of time personally. I would have just done it. I don't know that there's too much overhead. It depends on if you've used our markdown or not but either way, I honestly don't know that there's a time sink element to it. I think it's all gain. I don't think there's ever a downside to doing more literate programming in your lab. I know for my own lab, when we started getting everything under version control and having everybody be able to see what each other was doing, it really reduced mistakes. It really reduced miscommunication and it reduced our need for a lot of meeting times. I felt like there was a lot of meetings where we were in like hamster wheels about like, oh, well, were we gonna exclude these participants or include them or is this data that you're showing me Masoud? Is this like the most up to date data? Like was this after we decided this exclusion criteria was implemented? Like there was a number of things where we have to go like racing around chasing our tails, trying to figure it out. Like the data that was done and the data decision was made. All of that stuff just ends up being taken care of when you have it all documented in like a reproducible workflow. So you're able to not just document like what you did but why you did it and how. So usually my lab documents were like at the very top, my notes about like decisions we made about this analysis. And then like a date, a time, who was there? And then like what we decided and what we talked about with the code below it. But trying to make like a decision document really helped me. And I think there's actually some really good if you're a graduate student, there's really good blog post by Lucy Dagestino McGowan about like dissertating and how she did it with reproducible workflows. And I think that like is just not even possible to do if you don't use these kinds of tools. I remember being on a bus with a student named Eero Wang in Australia and she just showed me her whole dissertation on her phone because it was a book down PDF and she was able to just show me and she was like, that's incredible. Like I think if you have a PI that's opposed to it, they haven't seen the light and that's okay but that's kind of something that I think you can still adopt and use and benefit from. Yeah, I would add to that, that I totally agree with not asking for permission. I think I would, I would... Evolved! Exactly, I would also say that proof just to show that it is productive, you know? And so here, this is really what my section was all about. So if you come in and you have a plan, and you know that you have some, even just a little bit of something to show for every week, you're not gonna care how you get it done as long as you get it done, right? And then you're actually gonna get interested in this. But yeah, I mean, I think there are also ways to do this kind of thing in a way that you do lose a lot of time if you don't have a strong organizational presence. I think the difficulty is that when you do things like everybody else is doing, you're probably gonna do, just by chance, you're gonna do things productively, but if you wanna do things differently, you kind of be more intentional about it. And so that's what I was hoping to discuss in my section, and so how you can be intentional about it. It's a great question. I remember being a graduate student way back in the day and literally going through this conversation with my PI about why should I even use R? Is it even valid for statistics? And so I remember running my SISTAT code and my R code for weeks and just showing that the results of our ANOVA's and our plots were all the same. And I started showing up to weekly meetings with our Markdown Report. That took me two seconds. Like I just clicked knit on the new data. We'd pull new weekly data in with our animals, our mice behavioral data, and I could finish my work literally walking to the office instead of taking like three hours in a day to do my Excel and then move it over into this for plotting and then move it over to SISTAT for statistics. Unifying is good. Awesome, lots of good advice. Let's move on to the next question though, because I think we could talk about this forever. Any workflow tips for gathering and incorporating comments from collaborators into manuscript revisions when they don't use R? Does hypothesis help here? I think it can. I don't think it's the full story, but I do think that we're... I know us at our studio are really thinking hard about this problem because we feel like we want people to be able to have that kind of like experience that you've all grown used to using Google Docs, for example. And we actually hear from a lot of users that they're like exporting their R Markdown stuff to things that they can then import into Google Docs and then they're doing some kind of rigmarole there, trying to get the comments back. Because a lot of times what you see is like the people who aren't comfortable with R and want to make comments, they just want to comment on the text anyways. They don't really... They're not going to like change the code. They don't want to touch it. They're like, don't show it to me. I just want the track changes. So that is definitely something that is on the radar internally here and close to what I think Cortot would aim to be. But in the meantime, I think hypothesis is a good enough solution for some of those moments, but they're not necessarily going to work for the final... Your final draft of anything, right? Is your final draft of anything? It's going to be a Word document or a PDF that you have to submit. But for all the stuff leading up to that, I think if you're sharing and getting feedback, hypothesis is a nice answer to that. All right. Next up voted question is, should we wait until Cortot is supported in a stable version of our studio? Or is it worth the pain of daily bills and potential problems? Looks cool and I am keen. If you're keen, I think you're safe. I think we've all been using it in prep for this workshop reinstalling. I reinstalled the command line interface, you know, maybe like once a day, once every three days or so. We stopped development for this workshop. So this is technically a stable release because we froze it at the beginning of August. And in fact, one of the developers is here, Charles. He's actually supposed to be on vacation, but I think he was keen to hear people's feedback. But yeah, so I think you can go ahead and download it. It's unlikely that things are going to break on you in terms of the overall structure of your projects because at this point, things have stabilized enough, I think, on the API, but I don't know if Charles is still here. I'm going to flip through the names real quick. Easy. He's here. I think it's pretty stable enough that I would encourage you, if you're keen, to go ahead and give it a shot and watch the news and watch the repo so you can see when things have been changed. So, hey, Allison, this is Charles. Yeah, hey, Charles, hey, Charles. I totally agree with that. And I think the one other thing that hopefully you can vouch for is we do, we try to fix things quickly. And so if we do, if you do let us know about an issue, we try to be right on top of it and turn something around as quickly as we can to get people back up and running. So we'll continue doing that. Hey, Aaron, I should note that Charles has basically spent the last year of his life developing this tool that only internal people have been using. So I think it's really exciting to hear people's feedback about it. They are extremely responsive to GitHub issues. It's all open source work. So if you go to corto.org, I think there's a link to the GitHub repository there and under about, you can see a little bit more about how to contribute, but we really welcome issues at this point, feedback for features that you're interested in or things that you're running into on different operating systems. That's extremely useful to us, but it is a tool that is still under very active development as opposed to maybe some of the more under maintenance packages in the R Markdown ecosystem. Those are not necessarily switching very frequently. This is one where I believe Charles and JJ are gonna be working on it pretty actively for the next few months. But that means really adding things, not necessarily, hopefully not breaking things. If they do, it's a regression and they wanna hear about it. But I think that we're to a point where I don't feel as scared to tell you to go ahead and use it. All right, I put some links in chat for the Korto GitHub, so you can submit issues directly there. And then we also have the Korto documentation site that you can read about there too. All right, next question is, any thoughts or use cases for NITR-SPIN as a way to render an R script to a report? And then there's a link. Oh, yeah. Yeah, I think that's really a comfort level for where you wanna put your thoughts. I think, so SPIN, for those of you who aren't aware of it, it's when you take an R script and you add some kind of, you can add like some minimal comments to it and be able to spin it into an MD document, basically. So it's sort of like a render, but it only goes to MD. And I think it's really nice for being able to quickly share a snippet of code, a little bit of code for wringling or for making a plot and then the output at the same time. So I usually do it when I've got somebody on the receiving end who's familiar with Git or GitHub and can like see that rendered spun code. Let me see, I've got an example actually in one of my repos of doing SPIN. So I think it's useful for that, but I think I probably don't like the way it forces me to write my words as comments because it's not as easy for me to write. I like being able to write full sentences without having to comment every line. So it's a little bit restrictive in that way for me personally, but for certain use cases it definitely works. So I'm gonna throw this repository into the chat. So this was one where I came up with a bunch of like smaller examples of code to share. So here's an example of an R script that I then used knit or spin to make into this markdown. So you can see that you're a little limited from what you can add to the R script. It's kind of more like really basic comments and every comment has to be started with that. It's a ROXigen tag, so it's a hashtag and then a single quote right after that. So I find it a little bit less readable. It's kind of more like a code first markdown second approach, whereas I think a QMD file or an RMD file is more of like ideas first code to support the ideas, if that makes any sense. So I think there's definitely use cases for it, but I probably wouldn't wanna write much more than a few snippets in it. I hope to link to knit or spin in there too for people who don't know. Awesome. And then these two questions kind of related so we'll just take them out once and workflows like those Stephen described are others finding that RMD documents replace most of the R scripts and can R code from a QMD be exported to an R file or to an RMD file? So QMD to RMD is just changing the Q to an R and then some minor YAML changes. YAML, yeah. Yeah, but so QMD to RMD is pretty straightforward and trivial. Exporting RMD to R. Yes, that you can use Perl. So you can use knit or Perl to Perl out all the R code from all the R code chunks from any RMD and I believe it should work for a QMD too. So that's a way to get out what you put in. I can pull up a, it's in a private repository but I have an example that I can share. You can keep going, Paul, the other question now while I do that. Yeah, so that's what, so that's why I thought that it was related to, there's another question about RMD documents replacing most R scripts because some of the pushback I hear sometimes is like, well, I just need an R script to run on an HVC. Like why do you need an RMD? Well, like RV is kind of like the sketchbook and then you can convert it to like an R script. Pretty straightforward. Or it's like the, you know, if you're wanting to develop it in the, in an RMD file, there's no trapping. Like you can always get all the figures out, you can get all the code out. So I'm gonna real quick share my screen. Yeah. So this was from a paper that's being reviewed right now. So let's see, oops, that's not the right one. So this is a paper on penguins, whoops, the text file. So this is the actual R markdown with the actual paper written in it. But, and this is kind of what I was talking about with the workflows thing. And I think Tom might talk about this as well. It's reading in all the plots from a separate R file. Let's see. Penguin plots. So yeah, here's all the plots are in here and they're just so long that we couldn't include them in the document. So we read them in and then kind of plopped them where we wanted to. And then because this is a format that supports that when you knit this document and you can run knit or purl as well, what it does is it creates this penguins.R file. And so all the code chunks that I marked, I set a global chunk option. This is a normal R markdown document. So I set a global chunk option of purl equals true. So you can see that all the chunks that are marked purl equals true got in here and it even pulls from that external file. So even from that penguins plots.R that I read in, it pulls from those. And then I think there's a few where I set purl equals false because you didn't really need to know what my global chunk options were in that penguins.R file. But every time I render this, it creates, it purls out all my code for me in a nice kind of commented form. So somebody could just run all this and be able to create every single thing. And this was actually a requirement of the journal. So it was a really nice time saver for us to mark, to make it an R markdown, make it useful for us, but purling essentially says, ignore all the texts, like ignore the narrative and just pull out all the code from the R chunks. And then you can control which code chunks go in and without and which ones don't with this purl chunk option. So I set it to true by default for everything and then turned it off for some of them that weren't necessary. So I think purling is really, really great for those kinds of use cases where you've got somebody who's like, I don't need to see the R and D file. It's like, if you choose to work in R and D, you don't have to feel trapped like your code chunks are all living in this one document. All right, so we're right at six o'clock. There's a couple of questions left but I do want to be respectful of everyone's time. What do you think Allison? I'm okay to hang out, people can leave but I'm okay to answer a few more questions if people have more questions. Okay, well, for everyone leaving, thank you so much for joining us. You really had fun today. Thank you all. It was really great to meet you. We have a request for an intermediate training course in Corto. So Charles, I think that's a good tour. All right, well, are there any other questions, Paul? Yeah, yeah, yeah. I can answer some on this. From Ray, but at least I'm hoping saying it right. I'm giving a lecture on this stuff next week. I would love to get thoughts on what text functions to teach and teaching stuff in stringer, X fun, scales and glue. So I've got a lot of mileage out of like the most basic thing I got mileage out of was transforming variables from snake case to title case using like string replace with string to title. So that's a pretty straightforward one. I also, it's kind of along the same thing but like separate to separate a column into two different data columns. I got a lot of mileage out of that. What do you guys think? Yeah, I would say one thing that depending on how messy data is, maybe before string to title or the string R ones, you could just do janitor clean names function. That to me is like the killer function. Like that saved me. It does so much more than just the string part. It takes care of symbols. If there's spaces in the variable name, I think it automatically does underscores. I mean, it just like cleans it up, makes it tidy. So I would always do like the here package to import with file pods. And then the next step would always be janitor clean names before I do anything else. I think those are really high value functions to share with people. All right. And then we have, I guess more of a comment. So much raw data today is in JSON format. Most texts and string processing challenges are wrangling these often irregularly nested data into tidy data frames. I see Tom nodding. Yeah. It's a challenge. Yeah. Tom and I had a bad foray into JSON. Nested JSON together. I'll let him answer that. So what I will say is that, yeah, JSON historically has been painful to work with. Well formatted JSON is actually handled really well by tidy R 1.0. So unness wider, unness longer, unpack and unchop are like tailor made for the problem of read in JSON, get to flat data frame or table. There's some examples on the tidier website about that. And I should have two blog posts about cleaning up JSON data with tidier. It's amazingly powerful as long as the JSON is formatted properly. If you get a malformed JSON, then it's just like any other messy data and it doesn't actually unpack correctly. So you do have to do some more complicated things. But good JSON is just like any other good data format. This reminded me of a case where I was scraping match statistics from a League of Legends server. And it wasn't in a nice format because all the different things were different sizes. I'm just looking at my code. It's like unness, unness, unness, pivot, unness, pivot, pivot, unness. So. It's horrible. No, I was just gonna say that we released a new tidier cheat sheet this week. So I think what Tom was describing about what's in tidier 1.0 should be nicely described. On there, tidier as a package hasn't had its own cheat sheet before. It was inside the data import cheat sheet. So that might also be helpful. And so in general, all of our cheat sheets got a huge makeover this week. So if you're teaching, you might wanna turn your students on to looking at those. If you were avoiding teaching with them because you thought they were too out of date, then now's the time. They're up to date as of today. And then our last one is a yes and no, I guess. Is there a tool for comparing two our markdown documents in our studio? Like a diff viewer or something. That is a really nice feature of Word where you can say like, here's the master and then here's the revise and then it does like the line by line. I don't know of anything but there might be some gene. I'm sure there's a package. There's always a package that does something like that. Diff R from Nash. Okay, maybe. And... I think Git is the ultimate. Like for me, like if you can use Git for version control of your our markdown or Corto files, like that is really nice. And if you can, even if you're able to work mainly in a private repository because you can't share the data or something like that, you know, being able to adopt a workflow that allows you to use version control, that's a real sanity saver. Being able to see what you're changing, why you're changing. There's so many times where I look and might get diff viewer and I'm like, oh, why did that change? I forgot that I had like deleted this section and moved it here and then I forgot to paste it over there, you know? There's all kinds of things that you catch by yourself. And that's everything. That's it, we cleaned out the questions. We did it. All right, fantastic. Thank you guys so, so, so much for coming. And I feel like, you know. This is so fun. I'm a little bit starstruck every time I see you, Allison. And now- Oh, stop. You did it three years. This is awesome. Thank you so much. Three years in a row. I know. So I hope that you guys will have some time to actually look at the main conference. I think our keynotes are gonna be really, really interesting. I'm excited about Karen Deep, right? Dr. Singh? Yeah. So both are both talking about kind of algorithmic biases and which is such a hot topic and- Yes. And we discussed this a little bit at some point, Allison. I really, you know, I really hope that something like ethical or, you know, socially aware algorithms will become a thing that our studio will look into maybe as part of tidy models. I think it would be really cool to do some of that stuff. So I hope that you guys find the time to join the- If you don't have access to the conference, I can help you with that. Just let me know. If you haven't signed up for the conference, I can get you- You signed up? We register? Yep. All right. I just wanna make sure. Thank you all for joining us. This was really fun and very energizing to finally get to share some of Quarcha with everybody. We've all been working on it for a while now and enjoying using it, but haven't really introduced it to a cohort of learners. So this was really fun. So I hope you guys enjoy exploring it and please don't be strangers if you have issues or struggles with it. You have my contact information on my website. There's a contact form there. And then I think all of our contact information is probably all over the place. So feel free to get in touch. You can share this. Feel free to share Quarcha with your teams, your lab mates, anything like that. We're not trying to keep it a secret, but it is under active development. So we're trying to make it better. So great. Fuck everyone. Bye everybody. Bye.