 I suppose we can, we can start now. So, welcome, everybody. And thank you for, for attending this, this workshop, which will be a proposal for efficient reproducibility, reproducibility in art. So, we would like to provide a sort of journey from, from patchwork to project. What does it mean? So, first of all, there's something about me. I'm Corrado Lanera. I'm working at the unit of biostatistics epidemiology and public health at the University of Padua in Italy. And I work with, with ARRA since 2014. And these are my, my contacts. So you can, you can write me or reach me as you, at your convenience for, for any question about the material or, or, or other. So, with me, there is Andrea Pedo. I don't know if you want to, Andrea, if you want to present yourself. I'm a young medical doctor. I'm working with Corrado, the same unit of biostatistics epidemiology. And I'm collaborating on him first on a couple of projects about mechanical ventilation and artificial intelligence. And now we are working on some of the projects of the unit. So, some words about, about this workshop. There are the, the slide you can, you can find the slide online at the URL I reported here and I reported also in, in the chat. So you can explore, explore them. And for questions, you can use the question and answer facility of, of the, of zoom and Andrea, we will manage the question and any case and interrupt me anytime. You can, you can ask some, some, some question at any time, if you want to speak directly. So, you can choose if you prefer to write questions or, or, or, or ask them directly. There are two ways in which you can follow the, the workshop. One is on your local machine at the end stage, the workshop will be one stage after another, and I will create with you a single project from the, the beginning to to the end. But at the, at every stage, there are a snapshot of a sort of snapshot of the project clearly I create it live with you. So, so maybe there could be some, some difference. But at every stage, you can, you can restart from, from a similar or equivalent project using this, this line of code, filling the stage name with the appropriate, the appropriate one. So, if you don't want to, to run them locally. There is also a positive cloud workspace that has inside all the, the project. So, you can simply go to, to this link, and, and start with, with, with, with one of them. That we go through, through the materials. Clearly for, for a positive cloud workspace, you need a pre positive account. And I don't know if there are someone of you that would like to use this, this resource that does not have a positive cloud account. This is a free account. And by this link, you go directly inside, inside a workspace in which you, you will find all those projects and you can use any one of them. Not super powerful, but it is free for, for you. Also, if you, if you like to explore the whole project. This is the GitHub link to the repository, in which you, you can find all the materials, all the slide, all the links to the, the staged project we will, we will use. Okay, so after this initial setup, we will start with an introduction that will be mainly, that will be mainly by slides. And next, we, we, we have a little discussion about how is our vision of, of the pipeline to, to, to set up a project or to develop incrementally a project. And next we, we start a step by step project development, starting from, from a cleaner project, but, and follow all these, these stages, motivating time after time. That's why we, we could need some of these, of these functionalities. So, at the end, we will just show you a template that is available that we, we develop that incorporate all the functionalities that we highlighted with some more. Someone more, someone more that we explained to you. Okay, so, and so we conclude with, with some capitulation of summary of, of what, what we have, we have, we have done. Okay, so, I don't know if there are some questions about, about how to, to, to follow the, the, the workshop or. Okay, so we will start with, with an introduction on, on just a little, maybe funny or, or already known story of what has soon as possible mean when we develop a project. Usually, when, when someone asked us for a, for a, for a project for some analysis. They always need that they are ready as soon as possible. So, we started off getting the raw data, and maybe if they have some something that is not perfectly right, we directly fix, fix them inside the, the, the raw data, and next we, we start doing the analysis. So, after some, some, some moment, maybe we, we get some updated data that we, that they overwrite the previous one. And we need to, again, to fix the, all the, the, all the fixes that we made, and, and maybe there are some more to, to fix, and maybe possibly we don't know if we have forgotten someone. But we need to, at some, at this time to, to update the analysis and this would mean to overwrite them, or maybe just to save, just to save space or, or to have a, to have a backup. So, we made a copy of our old, of our old analysis and we renamed the current one as a version two of our script. But after some time and we, we find some bug, and we fix them, and so we save a third version of, of our script that can be named analysis version two. And, and also, we find that because it is the third run of, of our analysis. Some computation are quite slow. And so we, we try to rewrite them with some exploration and we made, we, we pass a few hours to, to, to made our script something weaker, and we save another version of, of, of our script. So, at the end of, of the day, we, we, we send our report that we hope it is correct, just before the deadline that as, as anyone know the real meaning of as soon as possible is yesterday. So, the problem is that sometime after that, our boss returned to us to say that they have found a bug, and just for this, for this, for this workshop, I call it a box. So, that is something like, so about that is reported by the, the one who asked us to provide, to provide the analysis after we, we, we give them our, our results. So we decide to not give up with the, with the analysis. And during the, the nighttime, paying attention to not falling asleep so with many coffee, etc. And so we code and code and code. And finally, we, we, we create many version of, of our files. We have many names in our, in our, in our, in our scripts. We create a new version of, of plots, fixes, and, and after, after all this, this work, we, we sent our, our, our vision, but we waited some, and at the end, our boss returned to us with, with another box. So, they decided to, to make them doing to another one. So we, we, we, we, they, they switched the project from from us to other, another analyst. And they started to, to, to, to do the quite the same thing we did yesterday, but even faster, because now it must be as soon as soon as soon as possible. So, the problem is that the, the other people cannot understand quite anything of our work and clearly is not our issues. And so they, they, they try to, to, they prefer to restart from, from scratch, because it is easier and faster. And so they also add other folders, other files, other version of, of, of those files, other plot and other fixes to, to the project. So just in time, as we did yesterday, they, they send the, the project, but finally, also then find a box. So, the boss this time, understand that it would be better to, to make the analysis by themselves. And so the problem is that they don't recognize where to find the data where, where the, the analysis are inside the scripts. So, at the end of the day, nowadays, they try to ask some help from, from the main provider of, of solution today. And, but the, the answer as, as we, we can, as we can read also relay in a template that should be filled with the, in order with the, with the analysis and everything. And this is the end of, of this little story. And I don't know how many of, of you have similar, have similar experience. I work with, with analysis and, and other people and we, we are working with the clinical data and medical doctors always have urgencies, always need something and the analysis made by yesterday. And so this pattern. So I saw this pattern or some variation of this pattern, many, many, many times. So, what is, what can, can, can we do so we say, okay, we are good, our programmer, and depending on what is your point of view, I hope to not, to not be too much. So that this opinion is, is not always shared by anyone, but maybe you can decide to, to, to, to go with, with the tidy verse and hope that with tidy verse everything should be, should be good and this pattern will, will not be repeated anymore. So the issues, what is, is not the structure of the tidy verse, I hope, I think that quite everyone of you have seen this chart before, but the problem with what happened in, in our, in our story is the hidden part of, of, of this chart. So the problem is the, that the raw data that you will import, maybe they can change with the new data. And the problem is that maybe those data should be kept private because maybe you cannot share, you cannot share them, but maybe you should need to share the code for, for your analysis without sharing the data. And the problem that you can find the box with function, so even your function would, would change, and maybe other people can ask you to, to change the, the function, or if you are not the, the, the single developer, maybe other developers in the same project can, can change, can change the, the function. So for the visualization, they can require updates. And when you create the model, also the model need to be updated or fixed, and maybe they are not only showed on our console with some results but they should be used in a real, in a real environment in production. They should be validated by someone else. And, and maybe when you provide a report, you, you don't need to simply create a report that have some evidence that you do the thing in the correct way. And maybe someone would ask you to, to have the ability to reproduce, maybe your, your, your results, and if is one of your collaborator, or maybe your boss. If someone that have actually access to your private data, they should be allowed to, to, to effectively execute your, your code and your, your pipeline to create your final results from the road data. And so all of that. Also, all of this is, is, is made by program, and maybe if you are using our, maybe you are using some packages, and also packages can be updated. I don't know how many of you have had issues with the package update, maybe since the sending of the, the analysis and the requirement for a, for a revision, maybe six months later, or you make a project in pose to restart to with with it two years later, or maybe someone else restart or continue the project two years later, and all the packages are different and you cannot create which package was used in, in the project. And so, as we know in, in our story in the last story, it was simpler and faster, and maybe with the similar outcome to restart everything from, from scratch. So, the, the main issues is here is that there is something in the, in the chat that is the time. So time can change the data can change the function and change the report environment, the object and also the people working with, with, with the project. So, the main, the main issues here is, is, is facing the problem that when we create the analysis, so when we conduct a project, we, we, we should not think that a project has a static object that we develop, but we, we need to consider, we need to consider time. And when we, we hear from, from our bosses made it as soon as possible. The problem is to, to have a result, have a result in less time, in, in less time as possible. So, the point here is, is centered in, in, in, in this sentence so don't waste time, but the problem with the don't waste time is, is another question. So, who's time. It's your time that you need to don't waste my time, maybe, if you consider me an expert to ask some, some, some, some suggestion or to fix some something or maybe the people you, you're asking help in stock overflow or someone else, but you don't need to waste your boss time, which, which, which time you have to, which time you have to save. So, in my opinion, when, when, when someone asked you to, to, to do something as soon as possible, you have to save the project time. For us, the project time is the sum of all the iteration you made until you have a correct version of your analysis and each iteration is the sum across all the people in your team that are involved in, in the project and for every one of them, the time they spend on every task, it is required to get the, the, the, the end of the project. And you should also need to include the breaks, because you can ask yourself to, don't sleep one night, maybe, if you want to risk to, to make something wrong. But, but even if you, you are able to, to, to, to be asleep, to, to, to not fall asleep for one time and maybe maintain the concentration for a single night. You definitely cannot avoid to sleep for a week or for a month, or maybe for more than one or two days. But I think that just more than two days, one days is, is more than enough to have the guarantee that even if you, you, you, you are awake, the program skill, that's the ground of zero level, quite, quite, quite fast. So, if you don't focus on as soon as possible, what we, we, we can do to have a definition of, of, as soon as possible is the, the collection of, of the set of, of all the strategies that you can, you can adopt, and you can, implement to minimize the overall project time. So, the first things that I have learned in, in my experience is to start slow. Because even if you want to, to have the perfect project, not all the perfect project need the same level of complexity or not every project need a great structure of control and development and maybe you have to run a single line script and you even didn't know, you don't need to, to create an overall project so, so the suggestion is, is to start slow. And what does it mean, it means, it means that if you need something, do it, but don't do, don't do things that you don't need it. Because if you do something that you think you need, but you don't really need this could be, could waste your time for two main reasons, one is that there are some possibility that this functionality will, will be never used. It is a poor waste of time, because it will never be used by anyone, neither by you, but there is another option that you, you anticipate a different needs so you provide a solution that is a wrong solution to, to a similar problem that, that will, will, will happen. So you, it is not only a waste of time, because something you have implemented will will not be used, but maybe you have to fix it to make actually running in the correct way, when, when you need it. And, but if you, if you got with, with these strategies, you have to also to, to, to start slowly, but keeping in mind that you need to prepare your project at you at and and yourself to add features to add new components. So you, you, you, you, you need to start to, to think about a project, maybe, even if it is a single file project, keeping in mind that there is the possibility that, that tomorrow, it will become a, maybe a greater project, or maybe your team will be become made by you, but maybe from by two people, not only, not only you. So, in, in, in, in, in this, in this workshop, our aim is to, to step by step, develop a project, starting from, from scratch to, to a project that will project structure that we think could be quite robust and reproducible. And maybe that can be easily understandable by other people and other collaborators. And, and that it is as easy as possible to be fixed if some bug is discovered, and also that can get the private data with, with, with your, within your project, keeping the option to, to share all your code in a, in a, in a way that it is easily executable by someone that have access to, to the, to the, to the data, but clearly it will cannot be run by someone that cannot have access to the data, but anyone can, can, can read it and, and explore it. So, the, just one, one warning. So, clearly, we, we have an order in our journey to this, to this project, we make some choice with the, with the tools that we, we use with within this, within, within this project. Keep in mind the, the, the, the, the, the first objective, so if you need it, do it, and only if you need it. So, you can change the order, according to your need on your next project, and you can change tools, maybe if they are improved, or if you prefer to use other tools. And, and also, maybe I provide, I try to provide some, some basic usage of, of, of the tools we, we, we have chosen, but this workshop is not about those tools. So, for many tools, we, we, we don't explain how they work, or how can you become good at using those tools. And my suggestion is to start, maybe to use them and clearly study them, but, but maybe even more important to use them. Even before to, to start to, to study them. So, this is really a learning by, by doing, and by experience, and you have you, you, you create your opinion about which tool is better. So maybe you, you completely disagree with the, with our choices of the tools that we, we presented and you would like to use completely different tools, you, you are free to, to, to, to think about that. The only thing is to, to try to start to think a project that has something that has stages, and is something that leave a long time, and that the time will change the project and you should be prepared yourself and all the people, all of your team, will be of your future team that people you don't know to understand what, what, what you have done as quickly as possible and as clear and as possible. Okay. So, we, we, we can, we can start. You can switch on your microphone if you want to interrupt me at any time, use the question and answer to, to, to put any question. You need to. And so we can, we can start with the, with the, the, the creation of, of, of, of a project. If you, if you would like to have a ready to use a very project, you can run this line of code or go in the, in the same project on, on positive cloud. Okay. So, the first step in creating a project is to, to have a project. So start to, to think about your analysis has a project and, and so create a, a, a box. So, in this case, a folder in which to put maybe all you, you developed about, about that project. And then you can create a project with, with our studio that will be our, our, our, our choice to do in this, this workshop. So the, the main, the main point is that your project should be, should be independent on, on the position in which, in which, in which it is. In our, the, the package here is extremely useful to reach every position inside the project folder, starting from, from the root of, of, of the project folder. So, so in, in this sense, we, we will use here to be independent on, on the absolute part of, of, of our project. We can explore files within the sub folder independently on where the folder can, can move around our computer, or maybe on other computer if we, we share our project with, with someone other. After that, that is simply one of our personal choices and standards, we use a setup, our script, just to, to track the, the, the code we, we progressively run to, to, to create, to create our project. But to, to have the, to, to don't lose them, maybe to explore for a next project, or to have a ready to use draft of the stage we already did in the past, but also to make some other that be interested in how we have developed our project, maybe to help us in this development or maybe to, to try to use our strategies in, in the, in their project. And after that, we will, we will use RM in, in, in our project that is a package that create a sort of, of, of local library of packages inside, inside our project and using a very smart cache is, it is lighting fast to restore, restore the exact same version of, of the package of the packages we, we, we use the in our project in, in future execution, and, and also in, in the execution of different system, maybe. I use windows, one of my colleagues use Macintosh or, or some other operating system, Linux or, or, or, or other. And within a lightweight files and, and, and structure that don't need to, to share the, the complete, the complete packages folders, we can, we can recreate the exact environment in, in, in which we have developed and, and run our, our, our project. And after that, we can also activate the, the, in, in our project that it is an engine that the track, the, the, the version of, of, of our scripts and in general of text files, and, and it can be used to, to keep track of changes that we made during the, the time that pass, while we develop our project and, and, and also can, can help in, in, don't waste time or space in, in exploration, because as soon as we create a snapshot of, of our project we can completely delete code that are outdated because we, we will not lose them because we maintain it in, in our history so it allows us to make more exploration, more experiment on, and keeping track of them because many times we, we start to, to, to, or at least in, in, in my experience, I start to, to try some, some ways on implementing a solution or some other ways of implementing another solution. And I, I have lost in the past, those experiment because they are not tracked and I had a lot of files at, at, they create a lot of confusion in my mind so start to delete all files just to, to, to find that I deleted the wrong files on, or I have a very, I have implemented maybe a very smart solution that I have lost and I cannot restore them. We, we, we get, we, we, we save all of that and, and, and also we can, we can use the tab or other some similar, similar platform to, to host online project that are, that are tracked by, by, by Git so that also all our collaborators can, can have access to every stages of, of the development and can very efficiently contribute to it, keeping track of changes or, or, or updated. Okay, so, after we will create the, the, the, the first structure, we can suppose we, we need to, to, to, to have some, some, some analysis and, and also provide some record to maybe our boss or some other or maybe to write a paper with, with, with, with the, with the analysis. So we can, we can start creating and, and an analysis script and, and separate our markdown document. Now you can, you can use also, also part of we, we will not talk about it. There are other workshops of, of talks in, also in, in, in, in this our medicine that you can follow to, to explore how we, we have a question on, on the charts. If you can make a quick remark on the charts. Oh, I, I, I'm not sure how, what, what do you intend with the industry related, but in this environment is, is simply so it's not an environment. We create a sort of, of of, of a folder that contain all, all our project. We will stop at this level. Yes, yes. They are main issues that, so we don't talk about the level we, we, we miss, but the, the, the main issues that we, we, we, we, we won't touch is related to the Arab itself. So at least so we, we not, we, we not talk about all the system configuration. We, we, we, we simply stopped our, our, our, our solution at, at the, have and in, in encapsulating the, the, the project, the packages level structure. So oh, yes, yes, yes, yes. It is a, yes, it is local. So get it is, is an engine like and it keeps track of your project. So you can use with, with your local project without any issues. The tab is an online platform that hosts the project and, and provide the services and utilities for, for managing and deploying those projects also, but there are other platform. There are maybe on-premises solution that you can implement that use ways to host that you can, you can host the, the, the, the project. Okay. So, the first, the first issues here is, is that when we have analysis and, and report, we have, we have a problem that we have to our, our input data and we need to know where to, to put them and how to reach them and where to put them. And we need to put our, our, our output, output files. Okay, so we can start with the, with the first, the first, the first stage. I will also create, I will also create the project, but if you, if you prefer, you can, you can start from the big one we, we we provided. Okay, so, can you see can you see all my screen or only the slide were shared? Yeah. Okay, perfect. So we can, we can create a new project clicking here in the, sorry, clicking here in the new project button or in files, new project or here, new project as you prefer. And we can start with a new directory, maybe we are starting our project and we, we can give a name to the, the project we are developing, maybe reproach. And, and so, with within our studio, it is really easy to, to attach a kit and RM functionalities to our project, because they are, we can easily embed that and embed them in, in our project just to, to put those two things. Okay, so we can create our project. Oh, yes. This is okay, or maybe I need to increase more. Okay, perfect. Okay, so, so just to, to, to, to explore our starting project folder structures. We have something related to get that for the moment we can ignore. We have our, our files and folder related to, to the, the things that our, our project is an R studio, an R studio project. So, we explain those four files. And also we have our RM structure that is, it is composed by a sort of, of list of packages and that we use in, in our project and a folder containing some files, but mainly the, the local version of the local library, the local library for, for our project. So, as you can see, we have our version of our for my system here. And inside, we have the, the, the architecture and, and, and the, and the list of packages we have. So, as you can see, those two things are now aligned. So we have only the R inversion for point three point zero, and we have only the RM packages, and I have only the RM packages in inside my, my local, my local library. After that, we, we see that we have also an R profile files for the, if someone of you don't know what is the R profile, our profile is, is a, is an R script that is running just before that you have access to your actual R environment. And it is something that is, that will be executed when you activate when you, when you start every time you, you restart or start your project. When you open your session inside, inside the project, this, this script is run just before you, you, that you can have access to the, to the project and, and are itself. So, as you can see, the, the main things that it is done here is to execute the activation of RM environment, and, and facilities inside, inside your project and for the moment, this is the only thing that, that our project do to prepare the project for to be used. Okay. Another thing that, that, that it is important is that with, with the RM, mainly, you need only to remember mainly, mainly, only a single common, or maybe three, but maybe a single one that is the status one. RM status. I, I needed this, this time to explicitly name the package, I would, I would like to, to use, to use the function and with the double column. I access to the, the function that are inside, inside the package and so I can execute exactly the function status inside the RM, the RM package. So, maybe everyone you need to do about, about friends is, is suggested by rendering itself to you. So, maybe you, you don't need to remember quite anything. If you want to remember one function, remember this one, that is the, the main, mainly the, the only one you need to, to remember everything you can do, and we'll, we'll suggest you to do it, explaining even how to do it. Okay, so we can start creating our first script that we can, we can save in naming it 00 setup. So, to, to keep, to keep track, to keep track of, of what we, we did, or what we did in, in to, to create, to create our project. So, the first, the first thing we can, we can do is to, is to activate and include some, some, some, some package that we can, that we can use to, to help us in, in, in developing our, our project. So, maybe the first, the first option would be to start to install packages, and we can use the dev tools package, but the issues with, with this, with this line of code that that is that if we already have this package we can usefully install it again and again and again. And also, if we provide this, this, this project to someone other, we force the installation of this package in, in, in the, in, in another one system. So, this project is not a good option because maybe we can break some other code, updating, updating, possibly the, the, the package using Rambo this is quite, this is quite the mitigate, because we are, we are involving only the local library of packages that are part external packages from, from, from our, from our project. But anyway, one, one option is to try to ask the, the, the, the people to, to, to install, to install the package. So, and only if it is needed so we can, we can start with an if, no, why not space that goes for example. Okay. And ask if we want to, to, to effectively install our, our, our package. Oh, okay. And we, by default, answer, answer force. And if both the condition are, are matched, we can effectively install our, our packages. This way, this way, we can run this, this little snippet and ask us if we would like to, to, to, to install the depth to only if we don't have it in inside our, our local environment. So we can say yes. And if you would, if we don't like to, to install them for any reason, we have the option to, to say, to say, to say, no, I forgot, sorry. Okay, I think that, as you can see, in, in my system, the installation of depth of depth tools that required quite a lot of packages is very fast. And this is because quite, quite all of them are already inside the, the, the cash in, in, in my computer. And the very first time you install a specific version of a package, clearly all the installation must be, must be done by, by your system but after that, even in new projects. You use the rent and you use the same version of the package. You need just the time to, to make them link to, to, to its cash. So now, as you can see, inside the, our local library, we have all our package, all our packages here. But this time in our aren't blocked by we, we don't have them. In fact, those two things are should be synchronized. And, and we always have the option to decide which is the updated one. There are, there are three option or the package, the packages we have installed are the same of the packages that we want to have inside our project that are the one, the ones listed in the aren't blocked. Or, our library is outdated. So we have to restore the, the, the, the environment following the instruction provided by the aren't blocked directives, or maybe, as in this case, our, our library is updated and we need to update our, our list of, of, of packages that are used in, in, in our project. So, we can use RM status for that. And as you can see, I am now say that we have installed some package, but that they are not recorded in the log file. And it already suggests us to, to, to run this, this function to add this package in, in, in a, in the log file, if we would like. So we can, we can run it. Now, again, I am explained as exactly what it is going to do. So we are, it is going to change the no version of the package with the, with the aesthetics with a specific version of, of those, of those packages. So we can say yes. And we are simply updated now, our aren't blocked with the, the specification of the description of every package with the version and also the, the exact hash of encoding of, of the exact version of, of, of our packages, because even if the developer is not a very good developer and change the version without changing its number, this code will definitely change. We need that we, we know that. So I don't know that exactly which package you need inside, inside, inside your project. Okay. So, after that, we can start to, to create, to create our, our two scripts. And so we can, we can create the script for our analysis, and we can create also the hard markdown file for, for our record, we need other packages as, as our studio as cancer. Okay. And we simply say, okay. So here is our basic. We can save it with the, with the name. Okay, and we can clear all the content just keeping our Okay, so here, maybe we have said that we would like to use the package here. And maybe we would like to use the package that I diverse, for example, but, but as you can see, we need to install them because we, we, we doesn't have inside. So if you have those package in your system inside this project, there is no package called diverse, or maybe here. So we, we need to, to, to, to install them. But what happened that often you, you made some copy paste and say, okay, this is here, and this is here. So, just keep in mind that you are using some times to change every occurrence of the package name with, with the updated one, but maybe you can, you can do some error in, in this, in this copy pasting execution. So we can, we can avoid this, this kind of, this kind of repetition, and we can start to, to, to use, to use some, some function. So we can, we can install, for example, and now we can, we can copy paste this content, for example, and here we can put the package package, right. And here the package. So this way, we have created a function that automatically do all the, the job for, for us. And so we simply needed to please install. And with a single copy here. Okay, just, just keep look at this doc. This is only a, a, a sort of way to keep a name hidden. So if you look at the environment, if I have, if I have a variable. This is appear here. But if I have something named starting with the dot, it cannot appear. It didn't, it don't appear here. But we have it. So we can run this. And clearly, it doesn't ask us to, to install that because we already have installed it. But with the here that we would, we, we, we don't have now it has to install. So, yes. And, and as you can see, I don't know if you have seen the, the prompt. I have made a mistake because I put the comma here. So, also, I gained the option to correct this in a, in a, in a single, in a single place. And all my functionalities will be automatically, will be automatically updated. So I can also install the database or I miss the space now. So maybe. Okay. So, as you can see, now we have inside our, our library, the package here, here, but it is not inside our block. So, we can also we, we already know that we have updated, updated it. So we can also put here our. status to check, to check the status of our project. And as we already know, we need to have a snapshot of, of the project. So we can also put that here, whoever. Okay, yes. Okay. So, now we can, we can go with, with our analysis. Maybe we can restart our session to have a clean environment to work. You can do it pressing control. So, control shift at 10. And you automatically restart your, your session. So, maybe we can start to, to read some data, but the problem here. Okay, we have, we have created our analysis, we have created a sort of, of, of record, but, but now we need to. But now, we need a place to, to put, to put our, our, our data and, and, and our output. So, our suggestion is to, to create different folders. So, one for the road data, and maybe one for the actual data that we process it, and one for the output we create. So, this way, this way, we can, we can now put, put here some, some, some data we can create. We can create it by, we can use the, the, simulate the, the, the, the Palmer penguins data, just to have, just to have an example. So, to do that, we also need, we also need the, the, the Palmer penguins package. Okay. Oh, yes, we need to. Okay. Now we can, we can use our, the, the, the road data inside the, that's inside the, this, this package. And because we have already the, the, the kind of us, we can use the either function to write them inside our folder. Okay, this way, we get the, the, the data inside here. And, and now we can, we can read our data from analysis script, so we can import our TV row packages. Okay. And, and next we can start to we, we have a little preprocessing of, of them, for example, penguins. We can define our, our, our, our, our object. And maybe we can, we would like to have a uniform naming. So we need also the package Janitor. So we can put it here. So now, our package. So now we have access to the clean names. And next, we can maybe select only some variables. This is just to simulate some preprocessing. And now, when we have provide some preprocessing, we have to, to, to store our, our data somewhere, and we can store them inside the data folder. So we can write, yes, that is the, our data serialization format. So we have a sort of, of a file that any code in the our way, our, our data while CSV is a text format. You can do it. And it is simple, simply a text files nicely formatted. RDS, RDS is, is quite a same object, but it is not in a textual language, but in a sort of an R language that we cannot read but are understand. So it keeps all the, the structure of our object or the levels of our factors, et cetera. So we can put language and put it here. So just to, to provide a parenthesis. Oh, I execute all the, the, the consecutive line of code that create a single command with the control or command return. And you create, you run, you execute all, all the, the block of code that create a single chain of, of, of, of common. So with write RDS now I have my data here. And just, just a small, a small parenthesis, the difference with the main difference with the RDS and RDA that you possibly have seen if you use the save workspace function, or if you have the dot are see here, we have the dot are data that I suggest you to never, never save and never restore. But the, the, the main difference in those two kind of object is that the RDS file is a single file is a single object that here that is stored in a, in a way that are can understand and, and can store all the structure of the object, but you have to think about it like a single object so like a CSV so R should read this, this object and, and you can name it as you, as you prefer so you, you have to put a name on, on this object that when you, when you, when you read it, exactly as here, when you read a CSV, you have to put a name on, on, on, on the object that, that you have read, while an RDA is a, is a sort of, while an RDA is a sort of, of a box that in which you put our object and our object are their own name. So when you load an RDA file, you recreate, so it is, it is like a zip file, so you extract those objects from the RDA box, and you put it inside the environment, this is quite dangerous, because if inside the RDA box, there is an object that has a name that is equal of an object that you already have in a global environment, you overwrite your object without even knowing. So, just to, so think about this as a CSV. So, you have to read the RDS file, and you have to load the RDA file, RDA you extract and you take the object that, that are inside the box and put directly in your global environment. The RDS, you have to read them and assign a name to the object that you have, that you have, that you have read. So just to answer your question. Yes. We have three questions after that one. So please finish to answer this question then I will read you the others. So I read these two questions, one is about here, and the other one is about the working directory of the project, those are, we can answer them in the same answer, because we don't need to set the working directory, because it is the R project itself, the R Studio project itself set its own working directory. And we put here, when we define our path, just because maybe this script should work even not in an R Studio project, maybe. And also, as a, and also because maybe our script, our analysis script now is here in the main folder. But if we move the analysis from, from, from the root of our project inside another sub folder, maybe we would like to create a folder analysis, because we have more than one script with the analysis and we would collect all those, all those script inside the folder. We didn't need to change this line because they are always start, always start the path from the root of, of, of, of our project. So, we avoid the, the issues of thinking about where the analysis script is, so as soon as the, the analysis script is somewhere inside our project, this path always work, also always will will work. Okay, so there are another question. Yes, you can see that in the Q&A section. And this what happens if I update any package, for example in RENV. So, so, so for the package, there are, is a very good question. So there are two types of, of packages in this, in this sense, one type is the RENV package. And the other type is all the other packages. So, clearly, we can, we can, so RENV can update, can update the other packages, but it requires some different ways to update itself. So we have, we have two function. One is, one is RENV update that update all the packages that are known. And we have RENV upgrade that updates RENV. Okay, anyway, my suggestion is to always start the project with the last updated packages, the most updated version of the packages because they are the most updated. The one with the last bug already known fixed and there are the one that implement the last functionalities and possibly there are the ones that keeping that will last during the time, more in the future. But once you have started a project, the wonderful things of RENV that you don't need to update them. And my suggestion is, is as soon as you have not finished a project, don't update your packages. So start with the most updated version of all the packages. But once we have started, always keep the same, the same, the same version of, of, of the packages to avoid collision between maybe some other people with, with, with that work with, with you, maybe to update them or maybe some script that you have developed some weeks ago that work with inside your, your, your, your very same project, maybe you have developed a two script, one, two weeks ago and one now and the updated of the, the, the, the, the, of one package broke the, the, the previous, the previous script. So start with the, the most updated version of, of the package and never update them until the end of the project or, or if you don't need to do it because a very huge or important bug is discovered and fixed. Okay. So, we have created, we have created our, our, our data and we have stored a version of, of our data here. So now we can, we can use those data to do something. So, for example, we can do a plot, so make a block of example, a block. Okay. And we can put a smooth and we have this way. And we can also have another smooth, like the difference, put also some points for, for all those, put a nice team without the, that's end of the right part on the top. And just to know the, what I, what I did, you can, you can, so they, they, they automatically complete the command in, in our studio, you can simply use the correct consecutive matching letters on, on your comments. So for example, I can put the legend. And, and next I can go for pause and it doesn't matter what is in, in between the, the only thing that counts is that this is the only command that have this sequence in disorder. So we can be much faster using those, those tricks that are being called without spaces or all collapsed on that. So, okay, we can have our, our plot. You can see it. Okay. And so we are like that, that somewhere stratification counts, and we have to save this plot because we have to create our records. So, this is save our plot in NG. And we want to save our GG plot and where to put here. And we can put some, some configuration. So now we also have in, in our output, we have our plot. So this is quite nice. Okay. And so, after, after the plot, maybe we would like to, to have some analysis, highlighting those two, two wave ways of get the linear model results so we can have model that can create the assignment operator correctly spaces, space it with the alt and the dash, the dash, so simply dash. Okay. So, in our model is G. We have to save also our, our model so right at the S because it is an object model. Here, with the, with the, with the tab you complete also the path. Okay. Next, we go with the stratified one, maybe more. Okay, and we also write. Okay, so we have our mode and our second, second model. Okay, so now we have all our analysis done, we are, we are ready to go. So we can provide some section, some section on, on our, on, on our script just to keep track of what, what is done, we can use a control to use a section label so for example packages. And here we can control shift art and read and example. And here we can control shift art and here we have the. Okay, so now we have in this quick we have our analysis so, and we have stored all our object. So we are stored all our objects that are exactly the, the output of, of, of our analysis so we can now start to, to create, to create our record. So once we finish this section since we are already halfway through the workshop or maybe a 10 minute. Like to freshen up everything and start like again. Yes, of course. So, just to, to finish this, this, this stage. We, we, we need to create our, our record, but keep in mind that. RMD is useful to create our analysis in, in directly between, between our, our code and, and our description and text. But this way we have to separate, we could separate our, our, our analysis from our record, but we can do this because we keep it all in the same, in the same project. And we, we use in our report, the object that we create inside the analysis of the same project. This for the moment is quite a weakness of these strategies, but we will solve also this at the end so we can use record. Maybe the following is, and now we can create our chunk that as, as we have here we can use control alt E. Right. Sorry. And here we can have our, our path. So, output. Clearly we need to, to, to load the, to attach our, our packages so library here, because we have used here and library diverse. Okay. We have our summary for our, for our, for our model, but that's a simple model. Show a negative control sheet. Hi. Here, at the shortcut with the, with the top of bottom. Inside here. Okay. So my, and we have the other one. Let's show the, the other relations. And as always, again, here. Take our output. And so we have our picture. So we can meet our project, our report. Our report here with all the object that we have created directly within our project in the analysis script, and we have saved in the output of, of a folder of our project so that if someone asked us for a high resolution. Or if we have to need, if we need to, to reuse our model, we don't need to fit again, fit again that, and we can use this in our report as many times as we want without needed to rerun everything. Okay, so, so, at this point, at this point, we can have a break inside the break, just start to think about what, what should happen if some something changes so what if some input changes, what is some output changes, and what if you are to repeat your code many, many times as we did it with, with, with, for, for, in our analysis, maybe for save our, our object that we use multiple times or pretend to do it many, many times, or maybe similar plot, many, many times. And what if you find bugs inside, inside the aura or code. Okay, so we can have a 10 minutes break. Just to answer the last question in the chat. Yes, you can hide your code with, with the echo option. So if we set here to false every code will be hidden, or you can put here to have a global option and in a single make the eco polls to hide just this, this code from, from your, so if I can meet this report, just I have the output without the code of the, the first of the first time, while the, all the other one have the default option. And so I have the, the, the, the code of those chunk as well. Okay, so 10 minutes break. Okay. We can, we can restart. So suppose we have to something to, to change, or we have, we have multiple multiple instances of very similar functionalities that we would like to, to, to implement. And so for example, we can, we can take our analysis, for example, and, and say, okay, so maybe we would like to incorporate our storing function in a inside the function, because, because maybe we would like to, to, to, to use different input, and we would like to be more flexible, but sure to, to what we, we do. So, at the, at the, at the beginning, we can, we can maybe start with a control shift our function session. And maybe we can start to, to extrapolate our, our repeated code in inside the function. So for example, you can have a right object, we can function. We have our object, and a name we want to, to provide for, for the object, and maybe a sub folder or a sub directory of, of, of, of, so that the of our, of inside our, our project to store, to store their functionality. So maybe we have reader. Yes, our opposite here. We can use the glue that is already inside the already loaded install it. We have name. Yes, with glue, we can include variable inside the curly braces, and we get the, the output pay only an attention that inside your analysis script is perfectly fine to use the function name directly without explicitly explain, or declare the source packages, because it is a single script. And maybe you have restored your session, many times control shift at 10 and, and, and, and, and it is quite fine and make the code of more readable as well. On the other side, inside the function, always be the suggestion is to be always, always to be explicit about which function, which package a function, a function is because, because if you are not, if you are not explicit about about that, you can, you can maybe overwrite the, the main functionality, the main functionality of, of, of a package so a function of a package inside, inside of, of, of your code. So just, just to have an example, if just a very quick sample function. So now, if I mind you, I don't have the back so this is the first error so I need to, to attach the package, but suppose I have attached it. Okay. So now it works. But suppose that for, for some reason, I redefined the glue function. Like this. So my glue now return to because no one say to this blue, that it should be the glue inside the glue packets, and not that my global environment blue. So inside the function, because I don't know where to, I will use my function. I don't. So when, when I, when I write my glue. I don't know what happened before. I don't know what, what other thing I have, I have created another name for, for the same object because when I, I, I write my glue, I don't read with my eyes, all the, the code that is inside so maybe if here there are many, many lines. I don't know that this object is called inside this function. So inside function, always be explicit about, about where to put, where to, where to, where to have to take, where to take the function in which to take, to take the function. Okay. So this is the reason why outside function I use directly the name, but, but inside function I keep the package name. Okay, so. Okay, so for example from, from here from, from the object, I can write maybe our, our output, so right output, and I create our, with our object and name. And maybe we are right object, object, name, and in the folder, and I can return the object so that I can use the function in a pipe. And maybe I can have right data function, object, name, and this is right object, object, name inside data, and I will return my object. And also, I can use maybe another for the plot. This is quite different on some object. Name, I can use the last block to use the same convention as gg save. And now I can use my custom say, create another. So, as you can see the, the, where it is. So they save, they save the, the gg save function, take the name before of the object, while all the other writing function will take the, the object before, before the name. So, just inside our right function, we can create our custom function, take the name and the object and simply plot to gg save name and object with our path that is always output here. And we can also include our specification. And so that now we can custom save, and we can return the plot. Okay. So, so now we can have bright output, but because we have, we have returned the object and we take it as a as a first argument, we can also pipe it here. And we can, after that, we can write our plot, right. Okay. And so, even here in the model, we have our right output with the name mod, and also here we have right. And other issues rise here, because maybe we say that what happens with if the, the, the source of, of our data change, maybe all the analysis will change. So, maybe we, we would like to automatically created. Oh, all of this should be works so we can control shift F10 restart everything. We can see how Ram have something to say. Okay, there are those package that are that we didn't snapshot it. So we can snapshot them. Yes. Okay. Okay, so if we run, we don't need this anymore. If we run all of the script control a select all and control return run all in order. Everything should works. There's some function here that doesn't exist in visible. Sorry. Okay, so, control shift F10 control all control return. Okay, so let's say if the report is the same, we can connect it. Okay, we have our model and our plot. We, we say, we, we, we look, we look at that we have the updated version of, of our. We have store the updated version of the object. But maybe we would like to store with to keep track of all the version, maybe because when we provide the record often often. When we update it, someone can ask us from the previous object. So what if we are we are asked to provide the previous model. So we can put here some version, maybe, maybe now is 1.0 for example, and, and we can add, we can add some versioning on our, on our, on our, on our function. So for example, version. That maybe is not provided. And we can check if. We can use our version. We can hear here. And we can use to do. And we can use the beer. slash name underscore. So, yes. And so we can be there. Yes. Not that you have a small type in a line 15 in the package name of glue. Oh, thank you. Okay, so this way, we have saved also a version, a versioning name of our of our object, but we keep also save a standard unversioned name of our object. This way, our report will always call the same object with the same name and always is the last object that last updated object. But we, every time we update our analysis, we keep track of the actual object we use for this very, this very analysis. So we, we have to update that because we, we need the, the bear. And also here, and maybe here. It's, it's a, and also this is, okay, so now, as you can see, if I want to shift at 10 always restart your session to be sure that everything works. I mean, your console with the CTRL plus L, and everything is clean. So we can hear control all control restore. Oh, I don't, I didn't put the version in, in our saving. So, as you can see, all the version of the, all the version of the name of the object are saved and stored. And so we can hear, we can put our, in our report, we can need it, and it is still updated because all the naming are, are the last one. Okay, as we see in, in, in the function, as we see in the function, this is a highlight that we can, we can read, we can read the code to highlight what, what we are doing. So, try to extract, maybe the logic of, of, of, of your, of your analysis, maybe only in the pre processing, for example, you can extract them in, in the, in a function as well to, to keep track of, of what, of what you are doing and keep, to be more easy to read, easy to read the, your code. So, just for example, we can put the year, a function preprock that getting input, our raw data, and do exactly that. And in this way, we, we, we gain the first things I mentioned, so that now we simply say, when, when we read our analysis, we simply say that we have read our data, we have preprocessed that, and we have written that. And if we would like to know how we have a preprocessed that that is a different question, respect dot, what we did in our analysis, we can, we can go inside the function inside the function and, and check what we have, what we have done. And also we gain another option that as we have a function, we can, we can start to test it, maybe if we have found a bug. So, keep in mind that writing formal test seems to be time expensive, but always when you, you try to do. You find a bug, or when, when you try to see if an output of preprocessing is correct or not, you are always written a test, a test, but you are written them in the console, inside the console. So, the, the issues is not to, to, to, to have to write other code to test your code that seems time consuming, but it, it, it, it is simply to know how to test them, and don't waste your time to repeatedly write your test on, on, on the console, and also losing those tests, once you have seen the results. So, the package you can use a package you can use in, in, in your, in, in your project could be the test that package. So we can, we can install that. And from the test, the test that package, we can, we can, we can try to, we can start to write some, some tests so control shift are tests. And, and the, the, the, the, one of the, the, the, the, the problem, if we would like to say that of testing is that we have to load the testing package, and maybe other packages. And they can interact with the, with our script and we are with our analysis and with our environment that start with our package. So we don't need the test that package loaded and attached inside our session, but we need it only in when, when we do, when we do the test. For that, there is another package that is automatically installed with the deaf tools once that is the with our package with our package. And, and this create a sort of a box in which we can include the many different things. One of them is packets so we can use with packets. So we can say we would like here to use test that and the inside here inside this block of code that we will write here, we have the test that package attached, but outside here, the test package is not attached anymore. Now we can use the functionality of, of the test that package to create a report. So now we have to attach it to use them. But after that with the control shift that then we restart our session and, and we have a clean execution of, of our analysis. So with the reporter, we can use the default reporter, and inside here, we can test that we can use the test that functionality. So here we simply have to say what we want to test. So for example, reprock returns expected number of calls. For example, and one things that we have found useful is to divide the test in three main stages. One is the setup of of the test. One is the evaluation of our code. And finally is the actual test that we want to provide. So to test that the prep rock, we need an instance of the bureau and keep in mind that everything that happened inside the test that live and die inside the test that happened. So we can do whatever we want to hear that outside of the test that our environment is kept clean. So, leader, it's CSV. And we are 30 minutes from the end of the session. Thank you. Okay, now we go with the evaluation, and we have our DB reprock bureau. And at the end, we can have our actual test and call the be expect equal for. So, if we run, if we run this, if we run this code on the beginning of control shift of 10. So we have our version, our packages, our function. And when we run, when we run our test. Okay, when we run our test, we say that all the tests that we have, we have now just one is running, and it is faster. But as you can see, outside, we don't have the DB object, we don't have the bureau object. And if we explore the, our package stack, we don't have the test that package inside our environment. So everything here is kept isolated for everything from everything else. Okay, so, as always, we should always try to make a mistake. So, to see if the, if the test that correctly tested for our interest. And so, as we can say, if I have removed one variable, and that is, so the result is three and not for the text correctly is failing. Okay. Also, our tested. So, one thing that you can point at this, at this moment, is that clearly, this is a, so this is a quite horrible and confusing script. So, maybe if we ever had at this moment, so we have repeated our code and we would like to extract and, and you, we would like to use function to encode the repeated code. We would like to extract function to use different level of, of, of abstraction when we read, we did, we did our code if we find, we have found bugs, and we face ourselves in, in making tests, and we write test to be sure to not repeat. Every time all the tests we have already, already produced. All of that is quite confusing if we, if we arrive that in, in, at this point in, in, in, in the same, in the same script. Maybe now we, we, we can need some little, little structure that can be, that can be, that can be useful. And for, for that, we can simply activate a quite powerful functionality of, of, of our studio that is, sorry, a functionality of our studio and, and, and that tools packages and, and, and under other activity package. That is to have a description by this is a, is a, is a created, you can create that using the, they use this package. So, use this. The description, the use this package is automatically installed by the depth to package is a part of, of, of that, and the description, the description find is, is simply a file that is characteristics of, of our package development. We are interested in developing our package, but the, the wonderful things is that, if we have a description find correctly formatted that has used this to do. We activate all the functionalities of the package development, even if we are not developing a package. The only thing we have to care about is that the packages must have as they have a specific naming convention, but we can avoid that to check on, on that name convention so we can name our project as we would like simply using the exact name option force to use description. So if we run this, this, this line of code in a, in a moment we, we run this. We, we have our description file, I will not to open it because it doesn't matter on our interest what it is inside of that if you would like, if you would like to go deeper on on that you can open this description file. We don't know how it is structured, complete the, the fields that are inside but we don't need it, we, we simply need to have it in inside, inside our project. So, once we have our description file here, we can activate all the test that utilities, for example, so for example, use this. And we can simply do that, and everything is created for us, we have the test folder, and we have the test that folder, the test are scripted, and inside here, we will go our test but even that we don't need to know anything about about that, because we simply use other functionalities that are, for example, use this user to create functions in so script that contain functions in the current in the correct position so we can have functions. And, as you can see, it automatically create their folder and the function script inside, we can use whatever name we want. And, automatically, if you use, use this, use this, use test functions, it automatically create a test function script, correctly named, correctly put in the, in the right folder at, at every time, if, even if I don't, I close all of them. And I rerun user functions, it will open exactly the same file. And if I run, use this that function, it's opening exactly the same, the same file. So, we can extract and put all our function inside, inside the function script, for example. And we can put our test without all the machinery of wither and reporter simply the test. So it is even simpler inside, inside the, the test function. Let's say we can, we can completely, we can remove all those section that we, we, we added earlier. And we now simply have to, to load our, our functions so we can hear source, hear function. Okay, so, so that we have all of our function inside our script, we can put a tool here, if we would like to, to, what happened, why my people doesn't work. It's just a moment. I don't know what is happening. Some suggestion. I did some strange. I should come just moment or studio because I don't know what happened or analysis. Okay. I don't know what, what was happening. Okay, so we can put the tool here. And this way, we can render something to say, we have this different version of, of packages, and we have not record these files that it is used but not recorded so. Okay, so now we can, we can do our analysis control all return. We can see, we have saved another version of, of our data, we have saved all the version of our output, but our report still be the updated one and the same one in our case. Okay. Okay, so we correctly load or all our functionalities, but if we attach the library that tools. We have with the control shift T, T, we can automatically run all our tests. That is failing. Now we are, we see why, but we automatically run all our tests. Take a look in why we fail it because we cannot find the select function inside the preprock. And in fact, we, if we, if we go to the function file inside the preprock, the clean names and the select names doesn't use the janitor package expression and the deep layer package expression. We don't see this error before, because we, we, we execute our test inside the analysis script after we have attached those packages. So, while our environment for testing is isolated to the outside environment is not isolated inside from the outside. So, we have now to put a janitor and we have to put the applied and simply pressing control shift T, we automatically run the test. The good things about that is that with only a single key press of three keys, we are able to automatically run all the tests we have written all the tests we have written and we need to write them only once we don't need to rewrite them on the console to check the result every time and we, we run all of them every time so even if we change something that we don't expect that it would break some other functionalities. We always run all the tests that even the one that we, we, we don't think that are relevant. So, this is a very powerful, very powerful tools to save a lot of times, because we will always break, break something and this way we automatically fix our, our, our function. Okay, so that's it. We can, we can go on the further stage. And now we can, we can start to, to think about about data. What if we would like to have, we need to have data that are kept. Secure and hidden from the outside, but that we can keep able that we need to, to be able to show our, our code, we can, we can use. We can use environmental viable to automatically detect where our, our data is, is stored to have our environmental environmental variable, we can, we can again use a and use this functionalities and in particular, we can use, use this. Use this. Use. Are you wrong. No, sorry. Are you wrong. But we would like to have environmental variable only only at the project level so project. At the same time, my suggestion is to, is to put them inside the group of code, because in the same time, we want to use this to keep it private without any risk. So, you see, ignore. And we don't. Okay, so this way, and this come this get ignore, and the question that some of you made me before to have this environmental variable that have secrets inside so the real path in which are stored the data, or maybe some token or some password or some other kind of credential or secret in, in general, if we track this, this, this file in, in with, with get, we can, we have a high risk of this close our, our, our secrets so usually those two lines inside the same block of code with force to automatically run them at the same time. And inside the get ignore that is simply a list of files that are not tracked by by get inside our project. We add the, the, the correct line of code inside that so they get ignore so we are safe to, to put all our secret inside, inside our, our environment. So, this way, for example, we can put a variable, for example, that are all part. And we can have data path, and we can have output part variable, and, and maybe we can, we can use a secret, a secret, a secret path that could be, and I can put it everywhere in, in, in, in my computer. Outside the, the project folder, for example, I put it inside my, my, my project in, maybe in a, in a private space here. And I can copy the other one, or maybe I also can create and create them so for example inside our project. We have private space and we have data that are all and output. And here we have our raw data that are out completely outside of our project, and we have our, so I can delete, this is a previous run. I can delete everything. Okay. So, now, I need a way to, to, to, to use those, to use those information. And the way we can access to them is by the, the using of, of function since get them. So, if we, if we restart our session. We can start our session, and they are in your own variable are loaded. Yes, because of when we start the session, the first things that will be executed is that they are in your own, and the variable are stored inside inside the memory for for our session. Next, the profile will will run that is the aspect that we see below before and after that we can assess interactively or we can use or execute the code in, in, in our session. So, here, if we use since get them, and we name, and we name one of, of the variable that we would like to where we get the, the, the, its content. The problem is that if we use this directly, we can show it on the console, or maybe we can store them in inside, inside an object, and we don't want to do that. So, so we can create some functions that give us the option to get those, those, those variables so we can, we can, we can use this. So maybe we can create some utilities so use this use our utils. And we can use this. So we can execute those function and automatically inside the inside our folder appears the utilities function and inside the test that appear the test board for for them. The utilities. So, here, we can now put our function to to get the, the environmental variable so for example we can use get data row path function. Maybe we would like to go in a sub folder of, of our, our key then private folder, but maybe not. So we can use file path to define correctly, our, our path sees the 10th, and we have data row path. And can use. Okay, Colorado we are in the last 10 minutes of the regular session. Okay, thank you. Sub, sub. Normalize but it's useful to have part working on different system, and we need to explain if we want that the file already exist or not. So, if we make must work through. It means that the file and in the row data in the data row case that we want that the file is there or get an error on the other two options. So for the data and, and the output, we, we possibly don't have the, the, or ready the, the, the object that we wouldn't that we want to create so we don't need that we need the must work equal, equal false. So, let me just copy base that. And here we can have data must work. false. And here we have output. So we can also create some tests. I have already created them, just because I'm quite the late in time I copy pasted them. I created a test just to explain if the directory exist, and if the source to check if it reached the, the files, and if I get an error if the file doesn't exist in the case of of raw data, and the same for, for the other. Keep in mind that just with the control shift, the team. I have all the tests. Run it there, and maybe. Okay. And maybe I have all my tests that with the with the with error or, or, or, or passing. I get an error because with the test that I use another package that extend the functionality of test that including other expectation, that is, that is checkmate. Okay, and you can put just to let you know, you can put a particular are our file here. This is named simply set up inside the test, the order that's that setup, and this script is run first before all the tests that is running so here we can library. Checkmate, for example. And, and so now, possibly everything should work. Okay, because all the function are found were found. So, now that, that we have, we have all our structure. There is still the last option and, and, and the opportunity to, to, to, to, to improve. Okay, so it's still remain the, the issues that inside our record, we have load the object that we have created with with our analysis script, but there is nothing that guarantee the automatically linked from the execution of the test, and the, and, and the fact that the report markdown load exactly those, those results so we know that everything is inside the same, everything is, is inside the same, the same project and so it is quite good, but it is not a guarantee. There is, there is, there is the issues that we don't have a clear connection between what we have to execute, if we change, if we change something. We don't have an automatic way to, to, to create new objects only they, they, they, we need to update those objects and we don't have a direct link with, with what we have created and what we load in, in, in our report. So, so for that we can, we can use the, the, the target targets package, and just to let, let you know so, so we can add targets. Okay. So that even if you spend a lot of time the first time to, to, to install those packets with the, with random environment, only the first time you install the package you spend time, as you can see, in half a second I, I have installed the target package. We can use target types that expand the functionalities of, of, of targets. So, this way, we can simply use, we can simply use targets, and use targets is as simple as that. And, and everything is already set up for us. So it is created the main script that is already opened there with, with some useful instruction. And there are also, there are also another folder that it is hidden from here that is that manage all the, the machinery of, of targets. Okay, so, okay, so we have go out of time, just to, to. I would only, so I will only show you the final, the final results. So I keep this project here that I am. The only difference is that in the target, in the target, in the target script, I put all the, the calling function here, and every, every function get, get a name with, with, with the target. As you can see, I have created the object that I give a name to object that are created using one of the function we have defined we. So, we, we had linked with, with, we have also created a link with, with our record, with our record, and we can, we can create a function that save versioned, our versioned object. This way, this way, we can, we can simply call that, and here, and here, you say that you have all the, the graph of, of the, of the pipeline you have created with all the function and all the objects we have created up to the final report, and you see, if something had to run, and in that case, in that case, you simply tar, tar, run. Where is this run script? Tarmac, sorry, not the run. I'm with, with the late. And, and in with this, everything is executed only if it is required to be executed, and you have exactly in your report, in your report, you, you read exactly those objects. And so you have created in a perfect link of what you have executed and, and run it and what you have loaded inside, inside your record. Okay, to answer, to answer the question. And yes, you can use the code as you want. And, and also to, to wrap up everything. And you can use all, all our code, and you can share with, with colleagues, our repository, and also, also, we have already. If you know that you need all of that, all of that, we have provided a comprehensive template that is here at our unit. GitHub account that is lines analysis that is a template that you can simply push the bottom use template on on the tab, and you have all, all of we have talked about today, and you gain some functionality, other functionalities. And I hope also to have included some good instruction to, to, how to, to use and automatically configure the, the, the, the project using that that template, quite everything should be easy to, to, to use. Please open issues, if you find that it is not something that it is not clear. Okay, just to, to, to conclude, start easily, slow, but keep in mind that you want to have a final result that is good. And don't be a high speed. So not as soon as possible because if your result are wrong, all your speed goes down to zero. And keep in mind that the bugs will be there. And the greatest error you can do is to assume your code is going to be error free and do and check only what you really need to do and check. And use standard and, and current style to make everything invisible. So the, the incredible power of all of this structure, if you use it as a template, as, as soon as you use it in three, five, 10 projects, everything's become invisible because this is a lot of stuff. But you know exactly when to look and to see some folder or, or some, or some script and when I, I know that I want to go to some function, I go here, everything else disappear. Only they are folder. I see only they are folder. And here is the record, the test are all, all, all there. So the, the important things or have a current style standard and templates are making things invisible. So that is the most powerful characteristics of standards and, and template and that save at that save your time. So, these are just the technical recap of what we have done. And even if you don't use our template, if you find yourself to always use many structure and many, many convention create, create your, your, your, your, your own, your, your own template and go for as soon as possible, considering the project time. So, thank you. I hope that I would be clear and, and maybe useful. Remember that it is true that the best is the enemy of, of the good, but the only way to go faster is to go well. So these are two sentences for Voltaire and Uncle Bob that is with me across all my project and, and, and journey on my research activity. Okay, so thank you. And please write to both of us for any question, any suggestion, any requirements of improvement for our templates, give us feedback, and keep in mind that our proposal as tools are not written on the stone and are not, and they don't need to be shared. So, if you would like to use this that use this that or if you would like to use other things use other things. So, keep in mind only the process of create a project and maintain it across the, across the time. Okay, so if I can give a quick final remark. It's a lot to process it is true. But once you start to use them, it goes along really easily, like to follow everything and to have everything well organized. Okay, as that even I did a new computer science I studied medicine and I didn't have so much problems implementing it in my in my research projects so I think that's quite a pro of this approach like to combine everything at the point where the learning curve goes kind of flat and it's really easy to take up on those kind of libraries, libraries that fit so well together. Yes. Thank you Andrea and another thing. Remember that it is quite a lot, but you don't need to use all of that stuff every time, use only what you really need, and from that moment, you need it afterwards. And if you know in advance that you need all of that stuff, don't implement it, use a template. So spend the time, the first time or the second time to create your own template or to adapt a template you find from other with your needs and use a template and all of that lot of stuff disappear because you have quite one click away from a completely set up correctly set up a project. You don't need all of them, don't implement them. So start if you have only a single script and you have only to run a single execution, do only that, do only that but keep in mind that every time that you add something. So how that something could be maintained in the future with changes and implement a little structure that can be maintained step by step, one time after one time that you add the functionalities.