 Okay, hi everyone. Welcome to today's bite-sized talk. Today we have Chris White, who's going to be telling us about Gitpod and how it can be used for testing development and training. I'll let Chris introduce himself, but I hope you enjoy the talk. Hi everyone. Yeah, so my name is Chris. I work at Postdoc at University College London. I also help with the training materials at Sakura Labs, and this is particularly where Gitpod, so this basically a miniature computer in the cloud that we can use for development, is really handy specifically for training, but is also now used really frequently within the NFCOR community to test code and to make changes and use all the features of Gitpod, of GitHub, I mean, within this really nice environment that we can all use. So yeah, what is Gitpod? It's an open source developer platform that's using BS code, and it can spin up these miniature development environments in seconds from a Git repository with all the code you need so that you can actually do all the testing or whatever you need to do in your particular piece of code. And it's free to use, and I'll get to more details of that later. And all the progress you need are actually just a Chromium-based browser, an internet connection, and a GitHub account. So you will need to decide in using your GitHub account to get access to the Gitpod environment. So just to overview what I'm going to talk about today, I'll explain why Gitpod is used, how you actually get started, which is a very simple setup, and then we'll get straight into running an NFCOR test pipeline. Then we'll go into slightly more like deeper topics that may be more for developers of NFCOR code, which is editing NFCOR pipelines, dealing with GitHub and how to push simple branches, for example, setting up these whole GitHub environments and live rendering of HTML and to do things with the website. There's also a really useful task that we use at NFCOR. So yeah, why are we using Gitpod? Well, it's this really cool, it's the kind of ethos of NextFlow, the fact that NFCOR that you have these repeatable and reproducible pieces of code, but now we also have a way of testing them in the exact same environment between all the collaborators of a particular NFCOR pipeline. And it also helps just simplify a lot of the tasks that we're doing when we're coding with these particular pipelines. And it's really fast. There are other different types of environment like this other than Gitpod, but Gitpod's super fast and normally you can just go straight from a GitHub repository to this kind of VS Code window where you have a terminal, all your codes, and a really, really nice neat way of dealing with all your work. So yeah, to get started, we may as well just dive in so that we can show exactly how this works. So yeah, if you want to do this at another time, I think maybe it might be difficult to do it as well at the same time, but if you want to then go ahead. But all you need is the browser and your GitHub credentials. And to actually open any GitHub URL as a Gitpod environment, then you just need to add this prefix that's shown here before the GitHub repository URL, or you can install a Gitpod browser extension. So I will put all these in the notes afterwards, but you can download this extension and it will basically add a Gitpod button, green button to every Git repository and all it's doing is just making this prefix before the URL so that you can quickly access with one click, especially if you already logged in with your GitHub credentials, then it will be one click and you'll get straight into the working repository. And finally, I just wanted to mention that this is actually being done through a file in the Git repository called a Gitpod.yaml file. And this is the file that tells Gitpod what particular pieces of code that we need and how to set up that environment so that the particular code that we're interested in works. So, yeah, so we're going to just go straight into the live demo. So I'll explain exactly what it looks like when we go into this environment, how you can add extensions into the environment so you can add different add-ons. This is a picture from Gitpod. You can add Docker or you can add lots of different functionality within your containerized environments to test your code. I will explain how to use Git within this and also opening files like PDFs, etc., and live rendering for website development. So what we're going to do now is we're just going to take, oh, sorry, skipping ahead, we're just going to go straight into how to use Gitpod. So hopefully everyone can see this. So the example pipeline we're going to use is the NFCore test pipeline. I've already got the browse extension, so I actually have this link already. I could also copy the link address, open up a new tab, and just paste it in. Well, just for quick speed, I'll just click on it. So when you log in, this is all you need to do, and at this point it would ask you for your credentials. So for me it's not because I'm already logged in. So yeah, so this is going straight from that repository into now we're in Gitpod. And along on the other side you can see the Explorer set says these are all the files within your repository. You have a window here to look at visualization of code or PDFs, etc., and then you have a terminal at the bottom. You also have, should have extra buttons here, which I can't see. Anyway, there's all the things you would normally want to do on your own computer. You can also do here. And just to show you how this is actually working, the Gitpod.yaml file is not to kick the right one, but I'll go back. So this particular repository, we're in the master. So some pipelines, this is a very good example to show you. So when you're looking for a pipeline to look with running with Gitpod, you need to make sure this Gitpod.yaml file, because this is all the programs you need. If we click on that, it's shown that we have an image, so a Docker file from Docker Hub image is being called that has all the code that we need. And I will go a bit more into these Gitpod yaml's a bit later. So we're in this development branch. I also mentioned at this point, if you're actually making changes to NFCore code, then you'll be wanting to do this in the dev branch and not in the master branch. So now if we click it, we should get back to a Gitpod environment that I was expecting that should have all the code that we need to run this particular pipeline. So sometimes this can take a few seconds. So if we click on that, it will probably be more than that. But if not, we can move on to different other things that I can explain to you, but it should pop up reasonably fast. I get it, there we go. So now we are actually in the correct repository that I wanted to be in, but just move that away just to make it clearer. So you can see there's a Gitpod yaml file, which I was expecting to see, which has, this is the reason why we have the code brought into this environment. And then in the terminal, now if we type in next flow, it finds the tool because it's already installed within this Gitpod yaml file. It's told us that this container contains all the things we need to run the particular piece of code. And so the other thing that's quite nice here, if you want to actually test this repository, there's obviously the read me. To open this, this is not obviously rendered. There's a preview button so we can easily click things to preview the render to look exactly the same as it would do on the web page. It also tracks. So if you're, if you're actually needing to change something like in the read me, you can, you can add whatever you want and it will automatically change it in the preview on the right hand side. But I'm not going to change this at the moment. Anyway, if I move this across, so if we look at the read me tells us obviously how we can run this piece of code. And I just want to quickly show you that obviously it's quite easy to quickly run this particular piece of code. So we're going to run next low. And remember this is actually in a dev, the dev branch so if we run it exactly the same as this line, we would actually pull the master branch which is not actually what we need to run. So in the next row, we can just run from the main file. Then we can set a profile. And here explains how to set the profile as normal of next load code. And in this particular case, we're going to use Docker. The reason for this is that singularity is one of the few things I know that doesn't work in Git code. So if you need to test something with singularity, you'd have to do it outside you wouldn't be able to do it within this environment. Then we could set an idea. So everything you need to do here is very simple to do. Okay, so if you want to add a new folder, we could call it results. And this is the kind of way you can add a folder manually by clicking one of these buttons for new files on your folders. And then we can just run the code like this. So now we're running next flow and running this pipeline within Git pod. So I hope you can see it's quite nice that we could quickly get to running this pipeline and seeing how it works and begin to see the output. The resources on these machines aren't huge. There's only a few threads. So you can't do anything that's going to be using like eight threads, for example. You can run for maximum half an hour if you have the free account. But we'll explain later there's an access that you can get a limited time. If you if you need that for the particular project. And so yeah, if we just see that the program is running. And we'll see obviously all the tasks next flow running through. And then I'll just wait for it to finish. Shouldn't take too long this this pipeline for, I should have mentioned, is actually taking fast queue reads, it's running it through fast QC to get color control of those particular samples. And it's using multi QC to push them all together to make a really nice plot of all the different features, the quality scores of the fast queue reads. So it's a quick way of running a pipeline. And you can do this for any of the pipelines that have this get pod yaml. And it's quite fun to do it means you can just really test out a new pipeline see exactly how it works and see if it worked with your data. You can drop and drag your files into this explore area to bring files in. And there's also other ways that you, there's also normally here there will be a file in store I don't know why it's not here. I'm doing full screen. Anyway, you can drop and drag files in here. Yeah, here we go. Sorry if I have the outside full screen you have options here that you can save files to your local machine. You can open files and do all the kind of things you would expect to do in a normal computer. And the other thing. So now it's finished in our results file we have our results. We have the multi QC for example we have the multi QC report. And again you can use these preview buttons so that you can render these particular results into a nice file that you can see everything's going on. So here is the results of that fast QC result in multi QC. And so I think this is quite cool know that you can you can just quickly run these pipelines. And within this this particular environment you might want to change something so maybe you're particularly interested in changing a particular module, and you can still change everything in a get put environment. And once you change something, I'll just add, I won't actually push this to the branch because people won't be happy. But if I add on say, I don't know, perhaps, whatever I've changed here. If you click on the code here, you'll notice on the left hand side on the source control. It's noticed that one of the files within your repository has changed. And if we click this button, it said that we've changed the main.nf, and you can now use the get functions within get pod to add that particular change. And you can also then make a message for that particular change. And also down the bottom you can see that we have this development branch on the bottom left hand side. So if we click this, we can also choose to create a new branch. So if you've made changes to the code. This is a very quick way to add create a new branch and show that that, you know, so that we can push it to the push it back to the repository and then do a poor request. So I think that's everything I really wanted to show you on this very basic example. So let's quickly go to another example that it's quite helpful to explore, which is actually on the website. So if I go to the website and repository, we can also run it from here. And I'll just show a few more features that I think are really interesting to see. But we can actually live render the whole website. The whole website is changing and we can change the underlying markdown. And so this is quite helpful when we've been developing the website so hopefully this will quickly load. And I can show you this example. So I will show you in this repository that if we see the get pod.yaml file. And this is what I showed you before that you have. Well, there's many things you can put in these ammo files and I think the best way to learn about this is to go to the get pod website, but also just looking at other people's get pod yaml files you can see exactly what people are doing. Before I showed you image, image was pulling a docker container, but you also can initiate tasks. So this is so you can do lots of different relations. Commands related to Docker, you can open ports to expose ports so that we can look at web browser and share share a particular web browser and HTML rendering. And the other thing are the VS code extensions. So these extensions are all on over here. And you can make sure that each repository that you make into a get pod environment has all the different code linting services. Adding different particular pieces of code that are very useful to have each time you can add these extensions manually in this get pod yaml. And you can also set all the get hub settings that you would need to change. So just very quickly because I'm probably running over time. We can, we can set here in the remote browser, we can set this to being a live preview. Okay, no you can't. Probably have done something wrong. So yeah, so within this repository we're now making a live view of the actual repository that's changed. And then if I wanted to go through the code into the markdown, I can actually change the markdown and like I showed you before you can actually change what's happening in this live preview of the NFCOR website. I think this is super useful as well. The only last thing I wanted to show you was with the get hub button here. You can look at all the open pull requests. You can click on different ones that you're interested in trying to fix. I'll get rid of this for now. And it should show you the pull request and give you all the information it loads. It will. Anyway, normally it should you should come up with all the things you can do and get basically you can do within this get pod environment. I think this is really handy. So there we go. So you can assign you can do everything that you want to do and get but it's within this get out of repository and I think this is really helpful. So I think finally, I think I showed everything I needed to show there. And I just wanted to mention. Here is a few slides left. So I wanted to mention that also these get pod environments are used as a sense of the training. So Satish presents really well the training materials that we have two weeks ago. All of these repositories are now running a get pod environment with all the code that you need to run them. So if you need to next, so this is a really cool way of getting into get pod and the next flow that you can have this amazing environment in the cloud with all the tools that you need. So the cost, the free ones really cool is only 50 hours per month. You have four parallel workspaces, but you have this 30 minute in activity time. As a member of NF core get pod, get hub accounts, you can find a free professional open source account, and you can get this through the following link so if you click this. There's a, I think the form you need to fill in to be able to get a professional account for free, which is really nice of get pod to open source developers. And yeah, just to say there's not very many threads, and I think the maximum size of each repository the back 30 to 50 gigabytes in case you're testing and need more than that. But normally it's the resources are easily enough to do all these different testing things that we want to do. So finally just to say if you need help. There's lots of people in NF core that work with get pod and have lots of knowledge of how to run all these things how to make the yaml's for example. And there's also a section on the web page relating to get pod. And there's really great resources and videos on get pod IO itself. And so yeah, I think that's, that's everything. Thanks for listening. So it's really fantastic talk and really exciting to hear a bit more about get pod again. Are there any questions. So the options are to either put your questions into the chat and I can read them out or I will just mute the microphones. So you should be able to speak up yourself. So what are the security issues. Personally, I really, I really don't know. I, yeah, I would have to check that up for you. And yeah, I don't know about if it's, what level of security they have. Yeah, I would say for all of the stuff to do with NF core isn't basically no security issues is public code running on the public server. And if you want to do something secret on a private repo and maybe consider that. And of course stuff I don't need to worry. It's completely off your system. So it's more secure than if you're running it locally. True. Sorry, I was just going to read out the question. Yeah, one of the pros and cons of get pod compared to code spaces. Yeah, so this is a good point. I mean we did play around a few of these different things, but and get pod is super fast is done. I think it is powered by Kubernetes and it's is done in a much more efficient way and it means that everything's really quick. And you have these initialization step the reason it's quick is because it actually saves these layers of your of these get put environments, so that you don't have to keep installing everything at the same time. So that's like a second, a few seconds normally to get to these things. So it's mainly for speed and also they're a really great company that seem to be really they have a really active community of people that help you with your code. So it's, yeah, that's the main reason. And the next question is, is there a way to put a link on a document to open it and get pod that automatically takes the current branch into account. I assume so I'd have to check that again I'm not sure I'd have to try it, but yeah I think it must be, it must be possible. I just didn't, didn't share that. Oh yeah I was meant to share the links. So I'm just going to quickly. You can just prefix the file link with the get pod IO. Thanks. Cool. It takes into account your entire current URL so if the current URL you're looking at is on a branch. We've got open get pod in that branch. And the same with the little green get pod button. Also, most of the repos we have a lot of them now. So when you're browsing pull request get pod has an integration which will automatically add a little button into the bottom of the first description of that pull request. So when you're browsing pull request you'll also see a button say open in get pod, and that will open you directly into that pull request. Great. So there's another question here can you kindly share the links, e.g. the slack channel link here. And make sure the links are at least on slack and one or two channels to find later. Okay, unless there are any more questions that pop up quickly. I just wanted to thank Chris again for his fabulous talk. You can always catch up on this talk again on YouTube will post this very shortly after the talk. And we'd also like to thank the Chan Zuckerberg Foundation for supporting these talks. Thanks guys.