 Okay. Welcome everybody to this CTLP spring institute session. I'm John The Graves. I'm your host for this morning, evening, wherever you are if you're in person. So we're going to be talking today about how to create interactive computational learning experiences, learning Jupiter. So basically our sort of goal for today is to go over sort of briefly the structure and kind of the framework for what we're talking about here. That should only take about 15 to 20 minutes. I think I have done other sessions and there's some more information online. For example, my website about some of the bigger picture reasons why you might want to use this technology. We're going to focus more on the creating aspect of this. So just a couple logistical things. If at any point you have a question, feel free to raise your hand either physically or using the raise hand feature in the room here. You can type a question into the chat. We have one of my very talented RAs from one of the projects where we're actually building some of these notebooks. We're going to be talking about Pranche. You want to say hi. Pranche will be here if you're in the online session and you have any like weird technical problems that you can't really short out or sort out rather. Pranche can kind of give you a hand, move you into a breakout room really quickly and then get that sorted out. So hopefully that won't happen, but we're there to help us out as we go through this session. Any questions before we get started here? All good. Okay, so just to be aware, we have a bit of an experimental camera setup I mentioned this before so we have a, it's called the owl here. It's an adaptive camera system so as people talk in the room it will kind of move around and adapt so you can kind of see me here. I'm in the bottom corner and then we've got Ainsley and Joseph here. So we should all be able to take part. Okay, so let's get started. Great, so why, like big picture, why would you want to use this tech, these tools or this technique, what's sort of like the pedagogical and curricular fit for it. It's really flexible. So first of all, so I'm just going to use some common examples of situations where I think it's particularly useful, but I think, you know, really is hopefully you'll learn by the end of this session. The sky is really the limit in terms of what you can do with this. It's really how creative can you get. But some of the common things are, the idea of data is really important to lots of different disciplines, but data always has this problem where it's really really abstract. You know, talking about like the distribution of wages in Canada or something and that's not that's kind of hard to grasp what we're talking about, particularly when students are still building, you know, their intuition for some of these statistical mathematical or, you know, scientific concepts. But often you want to show them like this cool patterns and things like that, or you need to teach them how to do different kinds of coding, maybe you know basic coding operations to do analysis or computer programming or whatever. We want to demonstrate a new theoretical technique like an algorithm or something. And the problem with one of these is that, you know, these are major learning contexts and pretty much any course that deals with data or the systems that manage data or talks about the social implications of data. So it covers, you know, many fields in the sciences, social sciences, even the humanities. It's way harder than it kind of should be when you actually take it into the classroom because there's a bunch of sort of like, like, non pedagogical problems that interfere with doing something that should be, you know, it seems like a relatively straightforward thing to do. The first one and probably the most obvious ones your students need computers. George had the great questions like, what software do I need to install your students have to have the right software install on those computers. They also need to know how to use the software, maybe even code in it if that's something that they need to do. You also need to have some way to share all these materials with the students as well. All of these cause problems if you ever like tried to get students to install, I don't know, Python on 200 computers. It's a nightmare. Our students are really good at, you know, using cell phones and iPads and stuff like that, but a lot of them even students who are fairly tech savvy in the utilization dimension are actually basically completely tech illiterate when it comes to computation using like the idea that there's a directory and things go in it and stuff like that as often a foreign concept to many of our students. There's also an equity issue at play in many cases because if you're trying to demonstrate say a fairly high powered computational technique need a computer that can run it. Students, particularly students from disadvantaged backgrounds will come to UBC and they might have, you know, only a pretty basic laptop, and you'd be like, oh, you need to install all the stuff and it just won't happen. I've actually worked with students whose senior theses were basically limited or restricted because their computers weren't good, and they couldn't afford another one we couldn't find any sort of solutions that would solve these problems. So there's other dimensions that this kind of intersects with as I mentioned. I've talked about this before in other sessions you can find some of my presentations on the website if you're interested in sort of the bigger picture here. Let's talk about the solutions and static. So, in particular, what I want to sort of talk about is I want to talk about there's there's more solutions than this, these are all major points of failure there's a lot lots of approaches to sort of solving I'm going to share with you today the Jupiter based approach. And so what is this, why would we do it and how does it kind of operate. So rather than going like the details of all this I'm going to give you kind of like a hands on example from my practice. So we've been working for the last year on a project to try to integrate the system into basically all of our undergraduate econometrics courses which is a branch of applied statistics. I'm just going to kind of show you what these look like when we, you know, put this all together really quickly. So I'll drop this link into the chat here. You are playing along at home. And so this is our red page for our comment project here we have a large repository that keeps track of basically all of our statistics things it's currently a work in progress we're in the middle of a big cleanup right now for for year two. I'm going to show you what you end up getting out of these, these notebooks at the end so I'm just going to go here under basics, if you're following the law and I'll go to the preliminary material. I'm just going to go down to this introduction to Jupiter notebooks age because it's kind of basic. So this is like the final final output of a notebook looks like this isn't the interactive version I'll show you this in a little while but this shows you basically the structure of what you end up getting from these. So first they're a wave, basically they're, they're based on what's called the computational or sort of literate computation framework or literate computing framework. And what they do is they allow you to weave together different forms of content. So for example, you can see here at the top we've got you know, a nice title for our notebook. So this is the introduction to Jupiter notebooks here, we've got some information about the authorship so my highly talented RA Attica worked on this and then I read it mostly. And then got you know you can write these in different ways so for instance, we have sort of a standard structure we use for these because we use these in classrooms they have sort of like an outline from the material so what students will learn when they go through they use these particular books. We also have you know references and readings. When you get into them, you can have as you can see here there's lots of text that kind of talks about you know what the notebook is or introduces the theoretical concept or whatever it is you want to do. This is sort of an orientation here, and then you can you know you can define things you can look through them and then it starts walking you through actually how these things interact. So, this is kind of what you end up getting when you go through the whole process of putting together a notebook because you get a package that looks like this but the key benefit of the Jupiter framework is not that it generates, you know, HTML output but rather than this is interactive. So in order to make it interactive need to launch it on a Jupiter hub. So if you do want some inspiration you can check check out on the, the accompanying document for this one I have a list of different teaching resources people have built here and you'll be see that cover lots of different use cases this is one of them like you can see. So, for instance, like a bunch of examples of ones were like, you know, we teach. I don't know, basically, sexual tendency and stuff like that. Let's show you the sort of in action side of things, just to get started here so let me go back here so you have a different screen. So, there's lots of ways you can develop notebooks and you'll kind of get the flavor when we actually start building one. You can use them as demonstrations when they're really highly sort of hands off. For instance, we've worked with the Center for computational social science and sociology for basically people with no coding, no program or expertise or it's really just like push buttons make graphs show up manipulate the graphs with a few different like basic things. So that's really great. You can use them as projects, you can make tests or you know homework assignments based around these that's something you can do. You can like I just showed you can make a website out of it so students even if they don't have an internet connection they can just start don't even if they're not programming along they can read along and use it as a reference. There's lots of different ways you can do it I think there's, you know, it's ultimately about your teaching context. Okay, that's the preamble. Any questions before we kind of go further here. So what I'm going to do in this session. So it's the 15 minute introduction I told you so what we're going to do is I'm going to show you how we can use Jupiter here at UBC. And then I'm going to show you some of the basics of building notebooks, using the tools that are pretty readily available at UBC. So this includes making your very first notebook, some tips on basic markdown, which you probably already know but if you don't it's really easy to learn writing executable code, different kinds of embeds or images. Then I'll show you some suggestions perhaps for assessment or more advanced use which you can adapt and kind of go around and then we can just kind of do a Q&A. So that's the goal. So again, of this session, what you'll be sort of able to do is you'll be able to create a basic Jupiter notebook on UBC's Jupiter open platform or pretty much any other platform that exists because they're all pretty standardized. You'll be able to write markdown content in your notebook, which is cool because it looks nice and students can interact with it. You can code cells and run them, understand kind of what they're doing. You'll be able to see how interactivity works in notebooks, and hopefully be able to write a basic cell test. And then I'll discuss some of the different deployment options you have for sharing the students. Okay, so let's get going with the hands on side of things. The first thing you kind of need to be aware of is there are lots of different options for using Jupiter. Jupiter is a system. It can be installed on your computer. That's one option. You can do that. That's called a local installation of Jupiter. The more useful way to do it is to run it via what's called a Jupiter hub, which is where the main sort of server that runs all the platform is installed somewhere else. For example, here at UBC, we have this tool, which is UBC's Jupiter Open system. So Jupiter Open is called a Jupiter hub. And what it is, it's basically a hosting repository that you can connect to using a web browser that gives you access to the Jupiter notebooks framework and IDE and a bunch of different hurdles. We actually have two frameworks here, which we can talk a little bit more about when it comes to assessment. Jupiter Open is sort of the general purpose one, which is probably the one that when you're getting started, you want to use, and it's probably the one that you'll use in those courses. But if you have really specific use cases or what you want to do with your computational tools, like for example, you have really big data like GIS data or image data, something that there's a lot of information that needs to get processed. You may want a core specific, just because standard hub is for everybody, if yours is really demanding, you may need to set them up instead. The other one could be if you want a really tight integration with Canvas, like where it can push and pull things into Canvas, core specific hubs are more general for that. There are also a bunch of backup options. So, as I said, the Jupiter Hub framework is all open source, it's developed by the Jupiter organization. And so, for example, there is a non-UBC hub, which is still hosted and is you log in with your CWL, which is through, it's called Syzygy. It's hosted by the Pacific Institute for Mathematical Sciences. There's even one, if you absolutely don't want to log in with anything, this kind of acts as just completely on your own. But let me show you the UBC Jupiter Open one instead. This is probably the, okay. I'm going to drop this into the chat here. You're following along, or you can just Google UBC Jupiter Open. Let's sit here. Chat, please. I beg of you. Yes. So Syzygy is not, it's not UBC's Syzygy. It's Syzygy set up a hub for UBC students. So it's still hosted by the Pacific Institute for Mathematical Sciences. Yeah, you can log in through that as well. There are reasons to use UBC's Jupiter Open over Syzygy or another hub. The main reason is that it's hosted here at UBC. So it's on UBC privacy compliant computers. It's also supported, as you can see in the tech guide here. It's supported by UBC's very capable and actually just next door here. UBC's very capable LT team. So if something goes terribly wrong, you're not literally relying on Tim over in the math department to fix your hub, which is what Tim's does. Tim is awesome, but like, there's one guy and he's on vacation, you're in trouble. So this is sort of the maybe the better option. This is also kind of a little bit stronger here. I'll post it again in the chat. So if you go to this page here and you scroll down a little bit, you can see it on my screen here. This is the instructor guide. So they've got a bunch of information here, but the login is through this button here. So you click on this button, you'll watch the hub. If I already logged in, it'll probably prompt you to log in with your CWL. And after a little bit of a load up something that looks kind of like this. I'll just wait here. So just again, when you're on this page here, you're on this page here just scroll down and then click on the login via Jupiter open. It'll prompt you to log in with your CWL and then you'll probably see a little loading bar will go across your screen. So if you guys, it's spawning a connection to the hub for you and then you should get it. I mean, it looks like this. This is the point where I need to make sure I have my guide so I don't forget to do things. Oh, George just dropped, can you get that in back. Everybody okay. Anybody need a hand. You can also ping Priyanshu if you are if you get stuck. I will also mention that there is a link in the chat. I'll drop a link in the chat. All the slides and all the material I have sort of like a cheat sheet guide which is handed out into the in person people here. Which you can access through this PDF here if you get stuck or you want something for later on, but we'll go through everything on its own here. Okay, I'm just going to go back. This is very confusing moment there. Right. So what are we looking at here let's let's sort of orient ourselves to this so once you've connected to the Jupiter hub but what we're at what we what this launches into is it launches into a basic editor. The editor is called Jupiter lab. I'm not going to focus too much on all the features of Jupiter lab for a very simple reason and that this isn't the only way to create Jupiter notebooks in fact there are dozens of different alternative software some of which you might like more. The nice thing about this one is it's kind of what you see is what you get the output you're getting you produce in this one is exactly what the students are going to see when you share with them, but no book and the access it via their house. So that's one reason to kind of build stuff in this although you can get more sophisticated as you go a little bit further but let's kind of get into it so when you connect to the the hub it'll spawn the Jupiter lab environment which looks like this. And this is basically kind of like the workspace kind of like Microsoft words like homepage when it opens up. So in the center here this is kind of like the quick launch menu it's called the launcher. I mean you see it's got a lot of different sort of buttons here. These are all at the top here these are all where you can create or launch new notebooks. There's different versions here. These are all the different computational environments they're called kernels. So when I say kernel what I mean is a computational environment that runs a particular kind of software so you can see here. This one here is the Python three kernel. So if you wanted to make a notebook is that was going to run Python code you would use this one. If you wanted to make one that was going to run C++ version 14 you would make this one Java, R, Julia, Sage map there's there's a bunch. And in fact there are even more than this is just the standard kernels that are installed on UBC is Jupiter open hub. Does that make sense. So when you create a notebook it launches a particular basically language. It's kind of like where you would create them there's also some other options right if you want to you know just wants a console to do basic computation without having a notebook but these are the main ones you tend to work with the selling point of Jupiter is the notebook. Let's just look over here on the sidebar really quickly this is actually kind of the important part. So if you look at the top here you might not have it might have been hidden. This little folder is the directory it's your home directory on this particular server. On the computer right you're accessing this through a web browser you connected to UBC's Jupiter open server here. And so these are all the different files, you might not have any right now that are currently on your particular, your particular server. So you can see I've got a bunch of folders because I've been using this for a while, but this is kind of where things would go. Right click or the Mac equivalent or Linux equivalent control click on this to for example just create new notebooks or you can click on existing ones here to open up a pretty standard context menu like renaming deleting stuff like that. So just think of this as being like the file directory or file manager. Yes, just a bit. I don't see your memories in the bottom I don't see one online and I was curious how do you know they were sort of the basic memory education. Let me just take a quick look see it here. They got nothing on there but how do I know they, as soon as I have, they can see every way and that was some kind of resources. Yeah, I have two 2048 megabytes. So you might be in the. It depends a little bit you maybe as well let me just show you. You can tweak the settings for this if you look at the top menu. So the question was, I don't know how much memory I have. It should show up when you're in the notebook it'll show you the kernel utilization here. Everyone has the same amount so you see you have two gigs is standard. You can tweak what's shown in the settings at the top you can adjust the theme you can add different things so for example, and see there's like dark mode, for instance, dark mode is not good for presentations but you may like dark mode you can increase the font size and stuff so under settings. There's really a bunch of like pretty standard visualization settings for what this looks like here so things like this. Let me show you a couple other ones here. Okay, so file menu is this top one here I'll also just mention at the top bar in the file menu there's a very important button which is create a new folder. Okay, there's also this upload file this is how you move files from your own computer into your Jupiter lab account or Jupiter hub account. So if you wanted to upload will actually do this in a little bit this is where you do this. This is like refreshing the file browser something like isn't showing up that you think should be there that's how you would refresh it. There's a bunch of get integrations which you won't talk about too much today mostly for time reasons although briefly mentioned kind of the approach here so this is like if you have stuff on GitHub, you can load from GitHub directly, or if you just have stuff on regular get this is where it kind of shows up here, and we won't talk about too much today but if you know get you know it. I think that's important though so this little sort of weird circle, I don't know why weird esoteric symbols. This is the kernel browser or kernel indicators so as I said, when you are on the hub launch notebooks into different traditional environments running different languages, these are referred to as kernels. And you can start and shut down them and you can see what you currently have running basically in your account here. So if you, you know, it tries to persist your stop across settings, but sometimes you might not want this kernel still active. So you can like for instance shut them down from here, shut down my kernels because I don't need them open right now. So this is where you can kind of manage and get a big picture indicator of where your kernel kernels are working if you have a bunch of files with that. This guy's pretty straightforward, table contents when you're in a notebook so you can kind of jump around between headings which is mess. And this one you shouldn't play with but if you have it on your local thing you can add extensions to Jupiter lab which extend its functionality. This is something that has to be done on the hub so this is currently what we have available here so you can see like something for creating link hack and stuff, but we'll get there. Okay. I think that's about it. The only other ones I want to mention is just a couple brief ones in the in the menu thing up here. There's a little file menu, which in particular has stuff like exporting but we'll get to this this is where you like save export download things like that. The run menu is where you can run different cells, in particular in different ways, and the kernel menu here is another way of accessing this little circle menu with a few more options so you can like restart rerun do stuff like this, but let's see it in practice I think with all like tools it's better just kind of use them and then get familiar and kind of figure out where things are as opposed to trying to learn like all the different menus. But I think it's fairly, there's also pretty good help anyone there's tons of like good videos online so you can make a different help things. Okay, so let's make our first notebook. That sounds like fun. Let's let's do that let me make sure I have my sheet here so I can remember what I want to do, but so to create a new notebook. We're, it's actually fairly straightforward basically what we want to do is we, we click on the environment or the framework that you are interested in so I'm going to demonstrate using our why are because I'm most comfortable with it and it's the one I know the best so I'm less likely to make a mistake. But if you want to play around or experiment with for example the Python one that's probably fine too. But if you want to follow along very closely ours probably your best, best bet the two most well supported in the sense that like, there's the largest community around them are are Python and then Julia, but there are a bunch of different groups for example Sage math is like a math specific kernel that's very, very well supported as well. You want to do like symbolic algebra or something which is actually really helpful to demonstrate like complex factorizations in that. But anyway, let's get to it. So to make a new notebook I just click on the our menu here and it's going to create it you'll notice over here on the little, if I open up this, if I look at the file menu. You can see I've created the new notebook it's over here and it comes up with a terrible name it just loads as an untitled. So of course to rename it, what I would do is I would just right click on the sky in the menu here. And I'm going to call it an example. Now you can see the main changed here. So that's kind of a couple of things that's kind of important to pay attention to when you're working in a notebook. So this is like, you can see the launchers no longer here. It's now been in place with the notebook view. And at the top you got a bunch of different buttons these run different elements I'll show you these in just a second. What I want to pay attention to is this little sort of circle on the side. This is the kernel information for this specific notebook. And so you can see here this notebook is connected to the, it's on the R kernel. So what this means is that all the code cells and this will be evaluated using the R language or the R interpreter. So if you hover over here this circle is the kernel status indicator. So if you hover over to tell you what's happening right now. But basically white means I'm connected and ready to go black like fill black or gray means I'm running something these weight. And anything else means you have a problem in particular bomb means or curl crash, but a bomb. Yes. Okay, so notebooks are composed as a series of cells. This is like the basic building block of a notebook. And cells are basically just little content units. You can organize them differently but you can see here we've actually it loads up with one cell initially so if I click into this, this cell here. You'll notice my cursor is gone in here and it's highlighted the cell to show that I'm editing it. So computer notebooks have three types of cells. We can see the different types of cells if we go up to this little bar at the top here says select the cell type this also changes it so if I click on this you can see right now this is a what's called a code cell. There's also a markdown cell and a raw cell. So what's a code cell a code cell is something where when this cell is executed or run will execute or run the code within that cell will come back to this in a little bit. The other type which is called markdown is a cell that when the cell is executed or run it processes this, the text in it as what's called markdown text and we'll go through an introduction to this as well. Raw cells are one which are just the text they don't process it so when it's executed nothing happens to it most people don't use these very much, but they can be useful in certain situations when like you don't want to execute something. So for example you were like demonstrating markdown for instance. Let's start off by making a markdown cells we can see sort of like the writing element so to do that just make sure you're in the cell and select markdown from the list here. Make sense. Okay, so in a markdown cell you write in markdown markdown is a language. It's a very really basic markup language which you have probably already used before if you've ever used like old school forums like like VB forums if you ever used Instagram or Reddit, the markup that they use for formatting those kinds of fonts or text is markdown. It renders to HTML and it does support a lot of HTML stuff as well it's sort of well but let's take a look at kind of the basics here. So I want to make sure I cover the right things. Okay, so let's just type some stuff into our Hello World. You'll notice ours has a spell checker involved as well, which is nice. So when I type into it, it looks like this if I then go up to this little play button here which is the run or execute button. If I click on it it will render that cell as text. In this case it just looks like Hello World but you can get a lot more sophisticated. If I click back on this cell and then I double click on it I'll go back into Edit Mode here. So the beauty of markdown and one of the reasons why it's so popular is that it's pretty easy to write basic like standard human readable fonts and like our human readable texts, but also include a lot of basic formatting options in a pretty straightforward and easy to remember way. So good, I linked a couple of guides and stuff to markdown if you want to learn all the ins and outs of it, but for example, let's say we wanted to like give our notebook a title. We can use hashtag which it highlights in blue. We'll call this example notebook. And now when I go up here and I click on run the cell you'll see I get a heading here. Actually if I go to my little contents you'll now see I have a heading in my little contents that are here. So headings, so your standard like h1 h2 headings that you would use in like like word or whatever are just more hashtags. So section one markdown would just be to and now you can see I got a section one markdown part and I can like pop around the different parts of my notebook. Like forgetting on that. Does that make sense. I think this supports up to six layers of these guys but each one gets more and more. Make sense. So heading is pretty straightforward. Okay, but like what if you want some more text here like this is a notebook. Looks great. Also you'll probably notice it's annoying clicking on the play button over and over again. You can hit shift enter or control enter and it'll run the cell you currently as well. So okay, what's another thing that we probably want to do a formatting stuff. Does it automatically save. That's a great question. So if you look up in under settings here you'll see there's this auto save documents thing. So if you go to settings auto save documents, you can check that I currently have a disabled because I don't like it auto saving my documents. And then what it will do is every periodically I don't know exactly how it determines it but it will sort of periodically create checkpoints to automatically save stuff. Yeah, you can save of course you are happy with your notebook so far you can save it by the file menu and going save for good old control s. Make sense. So what's something else we probably want to do a group formatting something for students to read or something italics and stuff okay so you want to make emphasis you know make text look different. Next italics are pretty simple they're just one star asterisk or underscores that also works but so for example italics like this it will preview kind of what it looks like. But if you go ahead and render that it looks like proper italics without all the weird code formatting. Another natural one is bolding bold text okay bold text alter font size emphasis in bold okay well let's see we hit all these ones. So bold text is to asterisks. Can you guess what three asterisks is. Oh, I spelled it wrong. That's pretty exciting. Yeah, it's like the basics. So those are kind of your basic formatting ones here. Let's do another one. Let's say you wanted to indent some text like in a particular way for example let's say you have like a long boat. You can use this if you put a it's like the alligator mouth that in like the dagger it's above the period on your keyboard of using a standard US Canadian keyboard. So use the carrot, would you bracket. It will produce indented and extended text, which can be good for having sort of like call out elements or things like that that you kind of want to do. Okay, how about another thing that we all love and learn or love and use bullets. So if you put an asterisk at the very start of a line it'll turn blue. It's a bullet. So for example, and when I render this they show up as bullets to get indented bullets like second level bullets just add two spaces kind of like in Python if you use Python so you add two spaces or a tab. And then another bullet like this it'll turn green. And this is like, say, level bullet. And it'll look like this. Rowan has a question numbered bullets. You can probably guess. Start of a line, a number one, and then a dot will turn blue. The number two, number two. Now what interesting thing you can see this. I can't type. But notice, if you take a look here, it'll keep the numbers in order there's really no way to underline them it just it thinks they're all numbers here which is kind of undesirable but that's just the way that it does these things. So, you see it's like one to four things that they're numbered bullets doesn't really care what the numbering is. But it'll render is 123 which is sometimes a little counterintuitive. So, heading fonts bullets quotations. Okay, let's get a little bit more sophisticated so often what we want to do and we're showing students computational stuff is we'll need to show them code. So like actual code that's not just like running, but that's actually here you just saw it if you look at the bottom of my screen it just saved my notebook. There are two ways to do this you can either do what's called inline code, or you can do display mode code so I'll show you inline code first so in our the life. So let's say I wanted to refer to like a specific function. If you use the back tech. So on a standard keyboard where the tilt is above your tab, you get it's like mono space formatted. It'll look like this library function is important. And then when we render this it'll show up as like mono spaced kind of offset code elements. So this is often useful when you want to refer to something that's like a computational thing, or like an object that you want to have that kind of space formatting. And so again that's the back tech operation. Often, when we have like chunks of code, we want to like set them up in like an environment that kind of like they set off from the rest of it. You can do this with three of these back ticks. So tick tick tick like this and then another three to close the environment. So like here's my type. Yeah, kind of. And this will render you can see it kind of pops up in an inset environment that that looks like this. Some flavors will also if you put a little parentheses here and then you put a bracket around it may also recognize it as what type of language it is so you put parentheses and then the name of the language. Some Jupiter hubs will format it like it'll give it like colors and stuff syntax highlighting, but you don't have to do this. If that makes sense. So this one doesn't, but it'll be. Does that make sense any questions so far. This is like 99% of all the coding that I do in writing that I do. And what's great about this is that it's really easy to learn like you really don't have to learn any of this, like it's almost second nature once you kind of get into the swing of things. Yeah, it represents it like an indented snippet that's that's right Rowan. Okay, so I'm in. I'm in economics and we do a lot of math and economics so math rendering is something that's really important for a lot of fields, not all fields but let me show you how to do that so similarly to this like model space thing there's two modes for math in this. It supported using a framework called math jacks which is a very specific kind of latex if you know what that means. I want to make inline math expressions you use dollar signs, just like the back kick so this is a math equation, math equation. So I put a dollar sign here and then I write like, but another dollar sign and then it will highlight it. And right now it looks like garbage, but when you hit enter, it will. Mine is not showing up properly spaces one sec. So it doesn't change. There's something. Yeah. Why is my math not rendering properly. Okay, let's try the other one instead. Okay, that's better. Why is my math not rendered properly in sec. I have to come back to this. I'm not sure. Anyway, this is the standard format for using your math expressions if you're doing these guys. Does yours work if you have yours. Yeah, give it a shot. I don't know. It might just be my renderer. That's up here. I'm getting an unexpected symbol. Oh, yours is working actually. Okay, I don't know why mine is. It's fun. Yeah, it's just it doesn't understand you're in a code block, so make sure you're in the markdown block. Ah, yeah. Does yours work? Okay, so I don't know. Mine's being funky for reasons, whatever. Oh, it worked. Oh, really? Okay, I don't know why mine's being problematic right now. Maybe I'll make a new cell and see if it would look better here. Any tips pre-enchew if you try here. Okay, that worked better. I don't know why this one isn't working. Anyway. So we'll do it here instead. So you can see in my second markdown cell here. Dollar signs. This is like an inline version. Inline code. Or inline map rather. So it's inline map. And then if you want to make a standoff equation, you can do like, this is a display mode equation. You do two dollar signs. Like this on either side of this and then you put in here like, so also if you're familiar with latex, latex supports, this supports most standard latex, like any of the AMS map or AMS sim formats, which are like standard latex ones. Like, for example, you can do like, like pretty, you know, doesn't support everything, but you can do like, you know, more complicated expressions. Don't worry about this. If you don't know latex, it's not a big deal. But you can do like basic latex expressions and stuff that looks this. I don't know why my doesn't like working there. So it renders really nice. So math is important for many disciplines. So you can make it render nicely in math environments. This is quite important because often we want to sort of explain something or whatever and you can do this and students can also write this that they know latex as well, and that would be the case for more advanced students. That makes sense. So, latex is another important one you can embed math expressions into your thing. You can also embed images or URLs. So let's do that in another cell down here. So I'll make a new cell for my. Why would you want to put some stuff up across different cells you can preview chunks more readily. So there are three different ways to use URLs when you're doing latex or when you're doing Jupiter rather so like let's say we wanted to link to ubc.ca you can just do this like it will totally understand that and it will link and that will work. If you want to be a little bit safer in the sense of like you want to make sure that it really knows it's a link you can enclose it in like when you brackets. And then you can do this instead and it's going to render exactly the same in my case, it looks like this, but it'll render exactly the same little it'll look the same as both of them. You can also do, if you put in the square bracket, then you put here and render it like this instead. And this is this is the most preferred way to do it is you either do it like this or like this. So this is like where you have the human readable version of this and then this is the URL. The reason this is preferred is because if people are using a screen reader. So for HTML code, if you're using this if your students are using screen readers which is actually quite common, especially for students with visual impairments or even students who are like English as a second language where they're using like something to help them read through. This provides like a more wealth, this will read better in their screen reader. So this will this will show up properly whereas this will just read out like w double w double. Yeah, the second option is much more accessible so this is strongly like preferred this option and you can make it as spiffy or not as you want. Hello. You have to catch up quickly. Okay, you can also embed images and other things so in this exactly the same way you embed URLs into your document you can also embed different images into these things as well so in order to do this however you either need to have a URL to the image that needs it needs to be on the Internet or you can ask that URL to it which is usually not a good idea unless you own it, or you need to upload it into your folder here go up here. So we can see in our directory here this is where all the files that are available so we want to embed or link to some other content it needs to be on the Internet somewhere that we can access, or it needs to be in our Jupiter folder here. So let me show you so let's suppose that we want to upload a picture from our computer here so I'm going to go to my, going to click on this upload button. So if you click on upload here in the binder file browser, upload files, I've got a bunch of photos here I'm going to pick this picture of a rock which I like. There is a very famous rock right off point gray by the way you should try to track it down as a name it's a bullseye. So when I do that you'll see it loads up my you can see the image is now in the list of folders that I have here. So now it's accessible to me through this particular website. Right so now it's on my hub and I can include this directly into my thing by referencing the path to that image. So to do this, go make a website. They go here and make another markdown cell images formats are exactly the same as URLs so what you do. There's one small difference you put an exclamation point to tell us that it's a file to square brackets and then to open brackets. In the square brackets you type the alt text or like the caption for the image. And then in the non square brackets you put the path to this particular file relative to the current notebook you're working in so in this case it's in the same directive directory as me so I would just put some dot JPEG here. But if this was like somewhere else on the internet you would paste like the URL to that particular image or something. So if I hit hit control enter it will load the image in. So let's like go grab, you know, by go to like UBC's homepage. And then I like find a random photograph that I want to steal and copy the image address you can do basically the same thing here. Now this image here. This will also work. What I just wanted to do this is that if someone else moves this object or they change the permissions for this object, it's going to break your, or your notebook, or it's not going to break it but it'll show as a little like missing image, image, instead. What's the use of the caption. Use of the captions. Yes, this is the alt text. Yeah, so it doesn't show up directly underneath the image, but when a student is using a screen reader or something it'll show up as like the caption for that particular image, which is important. Yeah, so that looks like Rowan it looks like you were running the cell in the in a code cell not a markdown cell so for instance I think I can duplicate your error. So like, you have to make sure you're in a, in a, in a, in a markdown cell because if I do this it's going to explode, and I have to be in a markdown cell. This guy here, markdown. Now work, although this is not going anywhere. That makes sense. So images embedding. Pretty good. Okay. So let's move on to the next major cell type we got about a half an hour so we're doing good on time so that's like the offering part so, as I said this is part of the computational or literate computation framework and that you what you do is you leave together. And then you also weave that together with code the idea that the two kind of cross talk with one another so you, you, it's sort of like super commenting if you remember if you think about like a standard code file, but something it's a lot more accessible because you can kind of, you know interact with them. So if we make a new cell here underneath it, we want we'll make this a code cell instead. You can tell it's a code cell because it'll have this little square next to it in addition to saying code at the top. This is the kernel execution count, which I'll, I'll talk about here. So code cells are a little bit different than markdown cells, in the sense that they what they are doing when you run these is rather than just rendering them they are going to render the output what they do before they produce the output is they send the commands or whatever's inside that cell to the kernel that you are currently connected to. So this needs to be in whichever language we're using here so we're connected to our so we can use any standard our commands that we can run on this kernel. So for example, we could, you know, get it to do some basic computation so one plus one and then when we, you know, get run what it's going to do this will happen really quickly so just watch here you'll see this will briefly turn gray as it processes the computation, and then you'll see this will increment, find us what order this was run into be the first cell that was run, and then it will produce the output underneath it. So let's just try that by here. So, it was super fast because one plus one is a very quick computation. And you'll see here, this will say, you know, this is the third cell I've executed here on this current kernel. And it will spit out in this bottom part here so you can see that was the cell and this is the output. So if you're using a different language, this would hopefully work in your language because it probably can compute one plus one but every language is going to be a little bit different in the syntax that they use for these things. Does that make sense. And this is really where the beauty of Jupiter notebooks comes in because this is just like having a computer like a computer on that you're accessing where you have a connection to the our language. So you can do anything in the our language that you would do more generally. So or whatever language you're using, you know, Python C Sage math, whatever it can compute any of these things. So you basically have access to a fully functional computational environment which means anything you or an RA or someone else on the internet, who can program into it can run this on this, which gives you basically unlimited scope to create a different kinds of visualizations, interactivity, things like that. There's, there's just enormous ranges for what you can do and how you can do it. A point out a couple of common ones that people use a lot. So for example, like plotly is a really popular framework. Yeah, so one thing growing points out here, code and markup are mutually exclusive so they're different cell types. So although you can add comments in here just like you would in regular our codes, the downside, a comment that will be fine. This isn't going to render as a markup. Right now it will process the output as well as it can. It tries to do it sort of as intelligently as possible. So I'll show you a couple examples here so you know maybe let's do a more sophisticated code cell. So I'm going to load the tidy burst library, which looks like this and then I'm going to think this is the way to do it. This should work. There's no time. And you can put typos. This is also really good for teaching people to code because you can see all the, you know, it'll just get perfectly computed objects here. So you can see what I did here is I when I made a typo first and then so what I've done here is I've loaded the tidy burst package and it's spitting out the output and then I've also created a data object here which is this is a standard package in many computational language called the iris data set and it's like a bunch of flowers irises. And then we can you know if you wanted to look at want to look at the iris data set we could, you know, we could look at, let's do this head of iris here. We could do head iris here. We've got this thing that looks like this. It kind of tries to render these as well as it can. Right, so you can provide it with other post processors and stuff that you want to use within your tool set here. But now we can pretty much do anything we want with this you can compute computational stuff you can, you know, just going to make a quick graph here. You can do iris equals CPAL dot. Don't worry about this if you are coding along at home. Do whatever you want, y equals CPAL dot with other equals species. I can do. Oops, and I need to not. Okay, it worked. And I just have to display it. I'm not going to, you know, go over our coding because you can learn whatever you want, right. The point is, this has access to all the underlying functionality of whatever language you're working. So you can kind of dump anything you want into these notebooks and moreover, this might look really intimidating to students but you can also encapsulate these in other, you know, helper commands to make things more accessible to students depending on the level at which the students are kind of at in their particular, you know, learning or whatever. And there are many, you know, interactive frameworks that people have in different language. So for example in Python and R plotly is kind of designed for this thing where you have, you can create like interactive visualizations that kind of look like this there's lots of different options for this. So that makes, if you can program it or you can get an array to program it, you can put it into a notebook and you can make it accessible to students. That makes sense. So it's very, very powerful because you have access to pretty much anything you want. This also gives you the option to do kind of clever stuff that you otherwise might not be immediately able to do within a sort of standard markdown language. So embedding videos is something that we often want to do. And you can do this using the computational tool that you have where you can have it render the videos as HTML output and then it'll render it. I'll just show you there, it's in the, oh, actually. Right. This is what happens. Thank you. I'll sort of show you the, I always forget the code for this one. So I always have to look it up. So for instance, if we want to, let's say we wanted to embed like a YouTube video or something here. So I have an example for both Python and R. So for R you would do this. I already display, hold and display, underscore HTML. And then in this bracket here, I'm going to go to YouTube really quickly. I'm going to grab a video. Yeah, so Rowan asks our code cell, our errors and code sentences highlighted. It depends on what language you're using. You'll notice over here, there's a little debugger. Different, some kernels have debuggers associated with them are doesn't, but Python does. So it depends a lot on it does basic syntax highlighting so you can see here it does syntax highlighting, but it doesn't do on the fly debugging unless you enable the debugger for that particular one. So it's really not like it depends on what you're doing. So let's go to like YouTube, UBC. Hey, welcome back to UBC. Sweet. I'm going to grab here. Share. You can do it. I believe in you. Please load. Let's try it again. Why is YouTube the point of failure here? Let's try another video. Why are you like this? Yeah, I mean, let me try hard refreshing this. Well, what I need is the embed code for the for the video from from YouTube here. Try again. I'm going to just be under their share button, but YouTube doesn't want to load here. Let me try this YouTube embed code. Let me go here. I just wanted an example, please. I'm kind of stuck but I don't have an embed code. Anyway, the point is you can just dump in this particular embed code into here once I let me move it over here and I'll try to fix this. If anyone online has an embed code, if you can dump it in the chat pre-enchu, that would actually be a solution here. You can help me out. Thank you, Andrea or and George. Okay, let's grab this guy here. Thank you. Much appreciated. So I'm just going to grab Andrea's example here. I'm going to copy it. Then I'm going to go back to my example here. Then we just enclose inside the brackets here in quotation marks. Okay, now the one thing you have to do you'll notice it doesn't look like it's it's all highlighted in red. One thing you have to do is you have to you have to escape the other quotation marks like inside the embed code so you can see this is like the one that we're using in our code block. But then it stinks that like the embed is ending here. So what you need to do is you need to put a forward slash in front of all those oops, in front of all the ones that shouldn't. Can you just do a single code for the ultimate one? Yeah, I'm not sure. Usually you can. So if you do the, let me try it. I can do both single code so it's okay. I'm learning something new. Let's try that. Oh, game changer. That's so much better. Thank you. Thanks. Or you can escape as well. Let's hope this works. Yeah, I work. Thanks. We have a nice little video here. Great. Oh, Andrea, so he was saying our support. So what was your name? Stefan, Stefan. He was saying that you can, it supports both single and double quotes. So in our, I didn't know this actually this is awesome. It's like, both of these, or you can embed it in a single quote and then it won't like if they wanted to pure it one another. Both of them are recognized. So what I did here, if you look, I enclosed the embed string in single quotes because it has these double quotes inside it, which otherwise I would have to escape. Yeah. Good. It's awesome. Yeah, so you can embed other stuff as well. So if you use like h5p content. Yeah. You can use like h5p content, anything that you can embed kind of you can, you can do this in Python. It's actually even easier because you can do HTML. There's like an HTML magic you can process and take on but part doesn't support that. Okay, so that's sort of like, I think the big picture, like most important elements for these but let's talk a little bit about like student use and encapsulation because one of the things that we want to do when we're producing notebooks for student use or in teaching context is there's stuff that we want students to focus on and there's stuff that we don't want students to focus on when they're when they're doing something. So for example, this happens a lot in some of my things is we often work with real world data like data from the consensus or something. And that data often in order to get it to sort of the point that student where we get to the concept we want to be understanding requires a bunch of like pre processing or cleanup or something. Right. And you could you know have your first cell be like this gigantic monster cleanup cell that is fine. The problem you face is that you probably don't want students to see that. Another thing you can do is you can also have these cells here your code cell since they're interactive you can make them self tests or self assessments so for example if I have like, let's make a markdown cell here and we say, What is the capital of capital of Canada. And then you could say, you know, a, I guess like, a, Ottawa. The Toronto. And type properly. So it's always the hazards of typing in front of other people. So you could do something like this you could make questions or they could be coding questions to you know like what's the mean of, you know, what's the mean seatball length or something you know you can you can make questions, but then because this is interactive, you could have students say, you know, you could then tell them, you know, or you could put like a question mark here, answer. So you can pass something like this and you can have the students as they work through the notebook fill these in but moreover, you can also write a helper function that you know tests this particular answer. So for instance if we make a function in our just like let's call it. So we're like, s test one equals I don't know. Just one. So I'm just making this, and you can, you know, whatever you want to do inside your function so I can do like if answer, answer equals a print. Success. Else. You can do something like this and then you could have the answer down below you can have them run, you know. So you can do this and then when the students are working through the notebook they can, you know, fill in the thing. And then when they run the test it'll tell them whether they got it right or wrong stuff like that. Does that make sense so you can write interactive self test now of course there's some obvious problems here so the first one is, again that issue with encapsulation you probably don't want to show the students, the code that runs the tests in the notebooks. So the way to do this is to create a helper file. So, if you go over here, make like a new file. I'm going to call it tests dot are so it'll be an R file I can hold our files and then I open it up here. I've got our command again, you know, move the code I don't want students to see here into my tests dot our file, and then save it. Or if this is Python tests dot pi or whatever. And then if I go here, you can load it in our it's the source command source. And if I run this it should still run here right now, and you can put this somewhere at the very start of your notebook we usually have like a setup cell that we run at the very beginning of each notebook. And now this has encapsulated this as well but you could equivalently do it for anything you don't kind of want the students to see. So you could replace this with like show video one and it's just a function that just shows the video instead of like, all the, because it can look kind of students, especially students who are very getting started with code are often very intimidated when there's stuff that they don't understand. So encapsulating stuff by creating a helper functions and hiding them this is quite useful. You also notice that the logic of these helper functions can be as sophisticated or as, you know, rudimentary as you want so for instance you can, like, you can incorporate sort of simple correct incorrect answers you can automatically type in is like capitalization and stuff like that you can you can do a lot of that sort of like processing of the, the elements in these particular things. And this is really really great because the students work through the notebooks or you work through with them you can kind of have like points where you stop, you check in and you test your knowledge as you're going through. Right, maybe something like this or they could be you know, code based as well, you know, like, right and you could have like, you know, you can have a cell that they're supposed to fill in here and then you could write a test that that's whether they've got you know, the correct sepal length here. Does that make sense. So what's a sort of obvious problem if you're trying to use these for formative assessment. It might be a problem with this. Yeah, exactly like we have to provide students with these files so that they can actually run them and they can just look at the answers and that's not really great because I mean this baby fine. You don't really care about that that's okay. But often you really want students to work a little bit on the answers because you know especially if you're getting into the hands in for homework or something even just for completion. So how can you kind of go through the answers and just fill in all the correct answers than actually, you know, bother going through it. So how can you kind of handle this or what's a sort of like approach to handling this. There are several different ways to do it the easiest and probably the most straightforward way is to lean on the magic of cryptography, in particular using one way hash functions. The function in cryptography is something that takes an input usually rendered in invite code or something. And it computes a function of that input, which spits out a string a piece of text. That's very very hard to kind of go back to the input. It's rather called one way hash functions, the use for stuff like storing passwords on computer systems and stuff. You store actual people's passwords, you store the hatches of them. The idea of a hash function is that it's really easy to verify that someone got the correct hash but very difficult if you have the correct hash to work out what would have generated that. So this can kind of the this can provide you the way to kind of solve this problem here so let's say like you know we the answer is a really don't want to show them that the answer is a. We don't want them to be able to peek at this. So in our my pride in the notes, a couple of different options we can use the digest function so they go library digest. Then I do digest. Do the answer here. Yes, well, I believe it. I just a. We'll see here. I'll show you. Oops. Can't spell. So, what it does this digest function and what it does is it takes in this input and it can actually be any our object although it's usually best practice to test sort of the minimal correct answer. The one that's like the least prone to any sort of like manipulation by students. And so you can see it produces this big string here. Instead, this is the hatched value of a and you can see, you know, even if I, you know, if I pop in a different value like be here. We get a completely different output. Right. And so the idea is that rather than in your tests dot our function, which is what we'll do is we'll load that library and rather than testing whether the answer is equal to a will test if. Digest. Of the answer is equal to this guy here this string here, you know, here. Makes sense. So if I save this, and I go back to delete those guys don't need them anymore. I'll get a success but if I put in anything else like be just the wrong answer. I'll get a whoops instead. Right. And the students can't peek at the answer because it's just a big string of cryptologically secure information that goes into it now. One thing to be a little bit close enough when you are doing this if you're developing these notebooks in particular and you know you kind of put them in. You have to be somewhat sensitive about what you're asking students to put in the you have to be careful about this either by handling common mistakes that would compile incorrectly, or by being very clear in your instructions for students because for example if I put in capital a it's going to be wrong. Right. The reason is, we look at, you look at, you know, the digest of a and the digest of little a. Those numbers, the, the hashes of those are very different because although they're very similar characters, hashes try to not put similar things into similar hashes so they're not going to collide. So you have to be very careful that you are actually, you know what the answers they're providing are exactly what the correct answer should be because there's really no, no way for them to figure out, but this unless you provide some guidance for common mistakes and stuff like that. So it also extends to if you test more complicated our objects, like for example computations or things, anything that includes sort of arbitrary student generated text or input like the name of a data frame or the name of variable is going to be a problem because that's a different object. So for instance, like if you make a data frame that looks like you look at a data frame that looks like this and then a data frame that looks like this. So typically, like, from a human perspective, those are the same objects, right, they just have a different name from the variables, right, but if you do the digest of that data frame. Oops. You do the digest of that data frame and then you do the digest of the other data frame, you get completely different results. So you have to be somewhat careful and exactly what you are testing if you can use this method of a busting the inputs for these. Does that make sense. So, hopefully at this point we you've got a good sense of like how to build a basic notebook and some of the basic ways and the tools and objects that you can use as I mentioned, at the end of this she here I have a bunch of examples of different pedagogical approaches people have taken across a number of different fields is very different styles of notebooks different use cases which I think to be honest is probably the the best way of kind of learning about what you can do and what you might do but so said the sky's really the limit for this. So the final thing I wanted to talk about was yes. I have a question on the hashing so yeah digest. It creates this code locally that then, like, yeah, but if the students uses digest on their own computer or produce the same code. It will. So it's the input. Well, is as long as the input is the same it'll produce exactly the same code. But you cannot use it backwards. Yeah, but if I gave you this code, you would never be able to figure out what produced that. I don't know the option for which amnesty they could. Yeah, I just a and I'll get. Yes, eventually, but that way they could just guess all the four options. Because if they have unlimited attempts, you know you can solve any problem with enough brute force, so but it prevents basically brute force attempts that don't exhaust all the options. Okay. I want to share what. So what are you talking about, Ron? Oh, the digest code. Oh, yeah, it's, it's in the one second. Just one second I'll drop into the chat. So it's right here. Okay, so let's talk about how we can share these with students now. This is often the last of the final mile to actually using these in the classroom. There are basically two options. I'm not going to spend a ton of time on them today because they're barely self explanatory but they're also somewhat idiosyncratic to your particular situation. The first and the most straightforward way to do this is. So when you're using these in the classroom, what you do is you just download, you put these in a folder, you download all your notebooks onto your computer, you upload them to Canvas as a file. The folder, you just tell students to upload them to their own hub when you're ready to do this. This is the advantage of being super straightforward and that you basically have to do nothing. You just put them together, you put them on your Canvas site, download, upload, that's it, which is good. The downside of that of course is you do that, which means you know if you make changes or something you, you, oh, let's say there. You have this sort of like, it's not the smoothest way to do stuff, right? You would presumably like a better way to do this. So the other option is to use GitHub. So if you're familiar with GitHub, GitHub is a hosting service produced by Microsoft primarily for sharing code in different environments. The great thing about this is that what you can do is you can make a really, really basic GitHub repository and then upload your notebooks and all the materials there and sort of an organized way. And then you can provide students with what's called an NB Git puller link, or what it does is it goes to your public GitHub account with the repository of all your files, it grabs them, and then it loads them into the hub of your choosing. So you can actually try this. I'll show you on the comment website if you are so inclined. If you look here at our comment, oops, it's on the screen here. If you look here at our comment website at the top, we have this launch comment button, which if you click on has a bunch of options for where you can launch it. If you look down in the bottom here, this bottom, this is an NB Git puller link. So all this really does is it sends a command to your hub to pull from my repository where I've got all these notebooks living all of these particular objects. So let me show you what this looks like from the setup side of things. So if I go to my GitHub account, so you get a GitHub. There's my GitHub. Actually, I want my public one. So if you look here, we've got this little super basic GitHub repository where basically what we've done is we've just uploaded all of our notebooks, you can see them, they're all sort of organized into folders here. You just make a GitHub repository and then you upload the folders and files that you would like to share with your students. So you can see many of these examples also in some of the examples here, but all you do, like bottom line is just make a repository, and then you upload your files here. It doesn't actually require you to know anything about Git, which is kind of nice. Maybe a step to barber. We were just getting started. So make an account, upload your files here, and then you can generate a link through NB Git. You can read the documentation too as well, but it's pretty straightforward. What you do basically is you plug in the URL of the hub you want to launch the student's notebooks on. So, in this case, it would be Jupyter Open if you're having students use Jupyter Open. You drop in the repository URL for your particular one. You choose the branch which is usually main on new ones or master on older ones, and then you select JupyterLab as the environment that it would produce. So I grab my GitHub link here, and I paste it in here. That's my guy here. My branches is main. Then I go and grab the Jupyter Hub URL so you can see it's here. Then I paste that here, and then I click here. You'll see it has this kind of slick. It produces this long, long URL, but this is basically what you would click on to do this. You can kind of see if you go back to the comment web page, I have it to launch on different hubs because people might want to launch on other ones. So if I click on launch on Jupyter Open, which you can try, it will do this thing. It will synchronize the repository. It will load it up here, and you can see here there's a little folder. This is the student looking version, and you can see it should load this comic project folder which now has all these notebooks inside it on your own. And students can refresh this. So if you like make changes or something, you can just say, please reload the repository and it'll do stuff like this. So that makes sense. So this basically copies the notebooks from them. It's basically one click. This is nice. But as I said, there are other deployment options because the one downside of it being on GitHub is the fact that it lives in the open, right? Although it's still copyrighted, you can make, you know, you don't have to just read using an open license. It's still going to be on the Internet and people can copy and download stuff. And you may not want that if you're sharing something that might be sensitive or something that you don't really want to sort of floating around in the AI ether out there. Okay, that is it for our session today. I'm going to call the formal presentation part right now. But if you do have any questions, I'm going to hang around for like another 15 minutes or so. If you have any questions or anything, I hope this was useful. If you want any help with, you know, setting up your own notebooks. Andrea, I'll put together some step by step guide to doing the envy get puller thing and I'll send it to you guys after the session. That's something we can do. Yeah, we'll send it. I'll send an updated link to the resources.