 everybody. We are here for a demo workshop from by Maddie Griswold talking about spatial omics overlay, overlay of structurally proficient, profiled regions of interest on the GeoMix DSP tissue images. And so Maddie, if you would like to come on and share your screen. And again, for people on the chat, or if you have any questions, please put them in the chat. We'll take questions at the end unless you want to take them at the end or as they come. Maddie, do you care? Either way, it works for me. Okay. All right. All right. Go ahead. Hello, my name is Maddie Griswold. I'm a bioinformatic scientist at Nanostring Technologies. Nanostring is a spatial biology company. And so a quick overview on spatial transcriptomics if you aren't familiar. At the beginning of RNA sequencing, we typically had bulk RNA seek, which essentially means that you took your tissue, you put it in a blender, and you sequence that. So you could really see the homogeneous mixture of the tissue, but not much more past that. The next thing was single cell RNA seek, which most of you probably know. But in this case, you could pick out the fruits in the smoothie and get the genetic profiles of that. But you still didn't know where that happened in the tissue. And that's where spatial transcriptomics comes into play, where you can actually see the structure of the tissue here in this fruit tart, where the actual location of each fruit means something. And that's what we're trying to get to in spatial biology. So Nanostring is committed to open source tools. We are trying to foster a community of open source method development for Nanostring data. And we are doing this. We currently have a dedicated internal open source development team that I am a part of. And we have this team so that we can ensure high quality analysis for tools or for high quality analysis tools for all Nanostring users. And we're trying to standardize methods with the thought that since Nanostring scientists are also going to be using these methods, that it will be the best method to use by people downstream. So a quick overview of our ecosystem today. Our main package is Geomics Tools. And this is the S4 class that we use specifically for our DSP data. It's using a expression set framework right now. And this will get you from data loading through QC and normalization ready for you to do any analysis that you want downstream. We currently have over a thousand unique downloads and it grows every day. Our next package is Spatial Decon. This is a spatial cell typing deconvolution method for spatial data, but it was built using the DSP data. This also has over a thousand unique downloads. And we recently had a paper published earlier this year in nature communications describing the algorithm. But essentially we take each of the samples and we can get the cell type proportion and population in each of those samples, which is pretty cool. We also have a background modeling technique called GeoDiff that has a pre-print out right now. And the last tool that we have is Geomic Workflows. This is a vignette that goes through an end-to-end workflow on analysis. So from getting your out of the instrument to visualizing differentially expressed genes and a lot of other very cool analysis techniques. But even with all of these packages, we still are missing the image. So when you Google spatial biology and look at all the images, you're very the first thing that you notice are all of these beautiful images. Or at least that's what I notice when I look at these images. But when you look at the analysis, this here is a pre-print of a internal paper past this one image showing where the tissue is, where we were getting samples from. The analysis still sort of looks like an RNA-seq paper. So other than this one image down here. But the secret behind that image is that it was actually done in Illustrator or Powerpoint. So it takes a lot of effort to first make these little pie charts and then hand place them onto the image. But the community itself can see that there needs to be images in our analysis pipeline. Here in Surat, we're looking at a Visium data set and we have a very beautiful image there. And even yesterday, the team showing spatial feature experiments was showing that images are present on people's minds. But what happens when our data isn't spot based? Computationally, it's really easy to create a pattern of spots. But what happens if we don't have that data? At NanoString, our data can take any shape imaginable. These are five different ways that we can profile tissue. And it's really driven by what the tissue looks like. So geometric profiling essentially means that you can take circle samples, you could take squares, you could circle around tissue, like hand draw them, that sort of thing. Segmentation is a very cool profiling that you actually take this image and our instruments can take the yellow image and make that its own or the yellow part of the image and make that its own sample. And then the pink part of the image is a different sample. So we can take that out into biological compartments. And then this contour sampling technique is you take this pink section here and you want to go equal amounts away from the tissue of interest. So what happens on an infiltrate boundary? What happens if you get farther away from that or closer to it? That sort of thing. So with these types of profiles, you get very unique sample sizes and shapes. So these two data sets, on the left we have mouse E15 embryo and specifically zoomed in on the heart section and on the right you have mouse brain. We have very different looking samples. So looking at the heart, these are hand drawn regions of interest or ROIs that someone painstakingly went through and wanted to look at these structures specifically. And then here on mouse brain, this orange and yellow section are two different samples. So the yellow in this are neutrons and the other ones are the orange's neutrophil sections. So but these images come either directly off of the instrument or these ones specifically are from our Spatial Organ Atlas website. And they're using very specific software that is either hard to use, takes some time, is in a different language than most of the analysis is done in. So it's a little bit hard to actually do analysis or make a publication worthy image. So that's where Spatial Omics Overlay comes into play. We wanted something that was R-based which is where most of our customers are the language that most people are using. We wanted a fast solution and we wanted it to make beautiful figures that could work in any manuscript. So that's where Spatial Omics Overlay comes into play. Both of these images are on the same data set that I showed previously, but done in R are ggplot based and are very customizable. So here's that part part and here's the segmented R-wise. So excuse me, each of these are overlaid exactly onto the image, exactly where they're meant to be, and you can customize them to be whatever you want them to be. So an example of this is in that same pre-print paper, we wanted to highlight genes on tissue substructure and actually showcase these. So if we look at this first one, we're looking at tissue substructure which are highlighted or are colored like this. But if we look at EPCAM, which is an epithelial tissue marker, go back over here, look at the pink sections, you can see that the gene expression of EPCAM is only shown in epithelial cells. And we can go through that with all of the other tissue markers. So while you can show this in a box plot or any other plot that you can think of, it's much more striking when you can actually put it onto the image itself. So all of this is able to be done using the OME TIF file that comes straight off of our instrument. So this is our standardized file that comes from the OpenMicros V group. And in this OME TIF, it contains two files. It contains a pyramidal TIF, which is all of the images. It actually has multiple images that are at different resolutions. So if you don't need to be working with the largest resolution and want some faster rendering times, then you can work with a smaller image and it's the same data. And then the XML file looks something like this and it contains all of the information that you need for this. So it contains the mask coordinates, it contains the fluorescent colors and all of that sort of thing. So all of our data comes from this one file, which is super cool. So all of the data is directly from the instrument. There's no guessing on where it might be on the image, what this coordinate might be. So it's pretty cool. So we're going to move over into the demo now. Are there any questions before we move to the demo? So here we're loading in the overlay package and geomics tools, which is where our data is located. So the only files that you'll need for this package are the OME TIF off of a geomics instrument. Right now it only works on geomics, but with other parsing functions it should work with any other OME TIF out there. Then a lab worksheet that matches the samples in the OME TIF to the annotation file. This annotation file contains any type of annotation you want. So it can be count data, it can be what type of segment it is, where the tissue came from, all of that. So the first step is just to get your TIF file. This file is just a just a file path to your TIF. This is currently a function to download the mouse brain data from the spatial organ atlas, which is the data I showed beforehand, and it caches it so you only have to download it once. So now we're getting our lab worksheet and then we're reading in the spatial overlay object. Here it's extracting that XML and then it's going to be parsing it's both for the image data and then for the coordinate data, and then we'll be generating the coordinates from there. So while that's running, I will say if you have other, sorry to interrupt because I was just checking we didn't have, do you have links for where these things are if people want to kind of follow along? Yeah, do they want it in the chat? Yeah, if you put it in the chat, then I will get them a not quick but onto the website, but then I'll figure out a way to get them quickly to people in the room. So that first one is the vignette and then the second one is the package download. Is everybody on Slack and all? So while this is still generating the coordinates, if you have data that's not geomics data, then these are all of the functions that go into this wrapper. So you can make functions similar to these for your own OME tiff file. So now that we have the object, we can look at that object and it's a spatial overlay object. This only works on one slide at a time. So here's the name of your slide, how many overlays you have and the names of all of those for geomics data. We have to know what panel is used and then if any of your data is segmented. So there are, I'll show this a little bit later, but there are three different ways to plot your images and one of them is outline. This means that we are only taking the outline data, but if your data is segmented you cannot do outline. So this is what the object looks like. There's some other accessor functions, getting all of your samples, getting the slide name and getting some other metadata shown here. So right now we're going to start this because it takes a little bit of time. So right now we're going to plot the overlay data and we're doing it at a low resolution. So this will allow a little bit faster rendering times, but if we go back to this real quick, without adding the image all of the coordinate data is based on this largest resolution. So it takes a little bit of time without the image added because it's trying to plot all of the points that are available. So let's see. While that's thinking we can show it over here. So this is what that image looks like once it plots and this is again the low resolution image. So the edges aren't crisp, but if you do it high res you plot all of the points and it becomes a crisp image. So yes, we're still waiting on that. Like I said, it does take a little bit of time to run that, but we can see it on the knitted vignette. So the next thing we're going to do is add plotting factors because on this one we're just plotting the sample ID and that's not very informative to start with. So here we're reading in the lab worksheet which just a data frame of annotations and then we're going to read in a geomics set object which has all of our count data. Let's see it has more annotations like slide name and scan name and then all of our count data here. So from these annotations from the lab worksheet we're going to add segment type from the geomics set object. We're going to add the expression of calm one, gene calm one and then just to show that it can we're adding a vector called our ROI label. So plotting factors can come from data frames, matrices, geomics set objects and vectors right now and your plotting factor can be either a column name or a row name and it will automatically find that data. So if we look at the object again we can now see that these plotting factors have been added and if we look at them we can see the segment type has called ROI, the calm one, gene expression and then that arbitrary vector that we added. So I'm not going to plot this because we're still it's still going to take a while so I'm going to show it on here and here you can see that we're plotting by the calm one gene expression and all of these plots are ggplot based so you can customize them exactly like you would on any other ggplot figure and a scale bar is generated every single time that you plot the image if you want it. There's also you can do scale bar equals false and it won't calculate it and won't add it but here we're doing a scale bar that's a 30% of the width and we've colored it to green so it's fully customizable just like everything else in the image. If we go back to here we're going to be adding the image so we're going to choose resolution six in this dataset there are eight layers going back to this so actually it might be easier to see it on this one sorry for all of the flipping of slides. So in the OMETIF resolution one is the largest resolution and as you go down it gets lower in resolution and each of these images are half the size as the previous resolution so in our image if we can check the lowest resolution here and it will be resolution eight so for the next or for the continual continuation of this workshop we're going to use resolution six and then we're going to add the image onto the object so we take the object the tiff file and then whatever resolution you want. This addition of the OMETIF can happen at the very beginning of making the object so if we scroll up here as it's calculating the coordinates again um we go up to our read spatial overlay in this case we did image equals false just to show the what you can do without the image but you can set this to true and it'll do this same process that is that we're doing now. So it's recalculating the coordinates every time that you add an image it's going to recalculate the coordinates because the coordinates start off on resolution one and we need to scale it down to whatever resolution you choose so that it continues to match but once that is done we can plot another image this one we're going to plot by segment type we're going to put a scale bar in the top center of the image and it takes up 50 percent of it and we're going to color it a cyan color now that that's finished calculating the coordinates we can look at the image or the object again and we can see that an image is added the images themselves are magic pointers but in the show function we're showing where the image came from and then you can do any of the magic um functionality on that pointer but show image ends up um showing the image in your browser and then we can plot this like I said um this plotting is going to be faster now that it's at a lower resolution because there are fewer points on it there's that scale bar so that's the main functionality but there are a couple of um addition or things that you can do on top of that so one thing is to add visualization markers um these are um you know fluorescent images so it's nice to know what markers were used for that so the easiest way to do that is just add for legend equals true and it'll add a legend um on your image where all of the other legends are at um but sometimes it does get a little bit to read like this yellow one um but this is typically for more um playing around with the data and not really your publication ready data set or your publication figure and that's where Cal plot comes into play we're going to make up a spatial overlay plot and then we're going to add a fluorescent legend that looks like this so again you can't really see the the white and I apologize for that but um it has all of the markers in their colors and then you add the cow plots on top of the other image so this ends up taking a little bit of time to correctly place the box and know where to put it know how big to make it um but it does look a lot nicer when you're ready for a publication ready figure um any of you probably noticed that this brain um is upside down for what we typically um are used to so here we can flip the image um and it'll be saved into the object so next time we plot it we'll have everything flipped over the y-axis um but if you don't want it to be saved you can also just plot it um so we're going to flip over the x-axis on this one um so this data won't be saved but you can plot it in that way you can also crop the image so in these images there's a lot of black space around the tissue um that takes up a lot of space so here we can crop to the tissue it'll automatically find where the tissue is and put a buffer around it so here we're plotting that same cropped image um by ROI label which is the arbitrary vector that we put on the data and it's colored by um it's a brightest color scheme and then we can also crop by sample type or samples um so in this case we're going to choose all of the full ROIs um in our data set and we're going to crop it to include only the samples that we want so only these full ROI samples and so we're going to crop by that and the tissue goes over to where the see if I can get this in one view tissue goes over um to only show the full ROIs but we can also crop where we want any sample in this field of view it doesn't matter if it's the sample that you picked or not and we can do that as well and then show that so in this case once it plots we now have not only the full ROIs but the segmented ones as well and the last um bit that we have here is you can recolor the image um here we have to add the four channel image again and then um can I do that okay um this might take a little bit of time um because we're adding the image again but what here we can go back to this once we add the four channel image um this is the fluorescence data that comes out of it um and you can see the color code this color code is what determines what color the image um color the images so here we're going to change our fitzy dye is this one right here we're going to change this hex code to another arbitrary hex code and then we're going to change alexa 646 which is this one right here um we're going to change this color code to magenta and then you can change the coloring intensity as well that'll change here so after those changes you can see that they've been they've been done and the listed color has also been updated with that so then we recolor the image and this is the image that you get but the the color of the image should be the same color that comes off of the geomics machine if you've played around with the coloring on there it should be the same this is just just in case it's not the right color that you that you want so um and so that's the that's the package um right now um currently it's released on an internal website called geoscript hub um and there's a download tarball there and a vignette which are the links that I put in the chat um and we're hoping to get this released on bio conductor in the fall but in a couple future releases we're hoping to add graphing on top of images so using those x y coordinates being able to add graphs on top of that um we're going to make sure that this is compatible with our new cosmetics machine that's coming out this fall which is um at single cell resolution so this package should work all the way down to single cell resolution we want to be able to extract data um to put into machine learning applications and after yesterday's spatial feature experiment we um we think it might be nice to have a coercion of our um our object into that um for other people to use okay do we have any questions people in the room thanks for the talk uh have you tried overlaying segmentation results onto the image for example for the cosmetics platform that you are going to release haven't yet um all of the segmentation this essentially is the segmentation results um for our data at least on geomics so we haven't gotten to um the cosmetics data yet yeah because with a cell segmentation there are a lot there are going to be lots of polygons so it's going to be very slow so it could be very different yeah um the cosmetics data will most likely use the um I don't actually have a good picture of this the outline function um and so it just does the outline points but um I at that point it might be nice to optimize it and maybe go to like an sf package or something along those lines um to try and speed up times somebody else here I haven't seen anything online but we were able to get your links up there are now on the website of the bio bio c 2022 website for people to look at um we have some extra time I don't know if you had any live coding last minute stuff you wanted to do I don't think so unless anyone wanted to see um a specific thing off of this data hi I was going to ask does the package do any like quantitative analysis where you're comparing expression levels it doesn't um it just puts that um that data on an image for you okay great oh hey maddie maybe if you can pull up geomics workflows so that they can actually see that quantitative part it's in a separate so these are in a separate package and maddie's package is supposed to build on top of that quantitative analysis so you do your quantitative analysis um these packages have been released for a while and then you move on to maddie's package to to make your publication ready figures yep that is true so this is the geomics workflows that I was uh that I showed at the beginning or at least a screenshot of it this is a step-by-step walkthrough from uh beginning to end and so it goes through qc um and all of the qc types that we do normalization and then let's say that you want to go through your differential expression genes and you pick out the the gene that you want to show so maybe in this case you want to show off clic5 and you can show it between the two um two in this case tubules versus glomeruli and kidney um and show the expression over the top of your image okay we did get a few questions on the chat all right in the background um so one asked is this workshop on orchestra which I believe is no correct um I don't think so I didn't add anything to it so no but um the workshop will be the same as the links that I put in the chat okay yeah so they will just have to run it on their own I guess that leads to this doesn't need any everybody can run this on their laptop they don't need anything special fancy or large uh memory memory yeah it will depend on what image resolution you want to use um and what your instrument or your machine can handle um I have been running on a server and can't do bias resolution um but maybe your instrument or your machine will end up being at a lower resolution okay and then uh there's another question on and I'll read it out so everybody here can see it is but from Dana King apologies if this is a naive naive question question can you explain the magic pointer in relation to loading viewing the image yeah so the magic pointer is a um it's a package called magic that's um spelled like magic with a k or a ck at the end and so if we go into oops mu brain and look at the image um so it's I don't know the back end of their package but this is just an image pointer to um I believe your image file so it does it um it doesn't take up as much memory as uh actually reading in your image um and Ryan said in the chat that yeah it's the image magic bindings for r so yeah right now if you point or like if you call the image the pointer it'll just pull up the image there are no last questions last questions okay everyone join me again and thank you Maddie