 Many of you already know us, so we are a new bias academy. We are a group of European bio-image analysts and we have started organizing these webinars now already nearly a year ago. And there are around 25 webinars. We had collected many, many registrations, 14,000 registrations. And then you will find also videos on YouTube. All the webinars were recorded. And also there we collected over 40,000 views. And what you are here today is obviously NIME. So we have two NIME seminars, webinars, small series of webinars, one today and one exactly in one week. And I'm very happy about this. I'm also using NIME a lot. And it's a very useful tool for analyzing your data and also analyzing images. But of course, we have the experts here to present you that. So we have today here Jan Eglinger, who is a bio-image analyst at FMI in Basel. And then we have Stefan Helfrich, who is Academic Alliance Manager with NIME. And he is located in Constance. And so they will be the speakers today and in one week. And then we have also, we are a group of moderators, which will help you answering the questions. And this is Franca Voigt from FMI also. This is Romain Gué from EPFL, Julien Colomballi from Barcelona, Rocco Dantfono from Cric and me. And there will be, so if you ask your questions, there will be, you will see that we will pass them on to the speakers. They will reply to that. So we really hope that this is a very interactive event. And with this, I'm very looking forward to this, to the presentations of Jan and Stefan. So I stopped sharing and you're welcome to go. Thank you very much, Anna. So hopefully you should now see the first slide of my presentation. Really a warm welcome everyone. Thank you for joining and also for the organizers and the Reyes Academy for having us proposing the topic and inviting us and also sticking with us since it took quite some time to actually get this in our diaries. So what we'll talk today about is KNIME as Anna has already pointed out and also very specifically about using KNIME for image processing. I will briefly switch over to the next slide. You have already seen our faces. Jan, I currently can't see. Oh, yes, I can see your face. Wait for me, please. Yes. Hello everybody. There you go. Jan currently has been for a couple of years and still is a bi-image analyst at the FMI. I actually put a second bullet point on this slide for me because you might think, who is this guy? What does he want to tell me about image processing when he is related to an academic program and all this management stuff? So I've been doing this since 2017, but before that, I also did PhD in bi-image analysis and also worked as a bi-image analyst at the Biomaging Center at the University of Constance. So that's where my background comes from and already during that time, I have used KNIME for my image and data analysis. If you are on Twitter active, there's at Jan on the School Eglinge and at Stelfrich, feel free, reach out to us, mention us whatever you like. Jan, if you don't have anything urgent immediate now, I would just go ahead. No, please go on. Thanks. Good. So let's start with who or what is KNIME actually because that is always a bit mysterious when people start talking about KNIME as a software. Is it a company? Is it a cult? No one really seems to know. KNIME in a nutshell, there's a couple of things. So KNIME, first of all, is a company. We are, well, we're started out as a startup, as a spin-off from the University of Constance where our CEO is a professor for information mining and bioinformatics. So that was already 2006. So that was the first release of KNIME. So we actually have been around for some time now. We've grown to a little over 100 employees these days and have offices all over the world. Our headquarters is in Zurich, Switzerland. We have also big offices in Constance, Germany, Berlin, but also in Austin, Texas. And we're looking for people in Hungary these days as well if you're interested. So what do we do on a daily basis? We build software for data analysis. That's mostly two things. I'll go back to that in a second. The KNIME Analytics Platform, that's on the one hand side, that's our open-source solution. It is free. You can use it to build data analysis workflows. And then before that question pops up, because it usually pops up, how do we make our money? There's a second product, the KNIME server, that as the name suggests acts as a central place. You can collaborate in teams together. You can execute your analysis centrally. I will not focus a lot on that today, but I thought I would mention it anyway. So what is KNIME Analytics Platform? As the name suggests, it really is a platform for the entire, what we call data science life cycle, really from getting data into the platform, data transformation, all that weird stuff that we spend a lot of our time on renaming things, changing types of things and so on, to really data analysis, clustering, predictive modeling, but also interactive visualization, machine learning, artificial intelligence, really all that in a package. And the entire idea behind KNIME Analytics Platform is that it's built around visual workflows. I'll get back to that, what that means in a second. And the really nice thing about KNIME is that it's a generic platform that is extensible. And extensible in that case means you actually really can access data from a lot of different sources. And that's why we are here today. It's not only about text mining, but it's also about image processing, you can load your images in, you can put image processing there, but you can actually bring it together and really join data from different sources. I don't know, do text mining together with network analysis and get some images into the mix. And this is really what KNIME is great at integrating data from different integrating data, but also integrating different tools. So we do not try to rent the wheel every time, we have integrations with other great libraries out there, DECA that some of you might know, for instance, Keras, H2O, you can actually run Python and R code from within KNIME, and one prominent example here already, could also execute your image chain macros on images in here, just to drop that here already. Okay, what do I mean with visual workflows or visual programming? There's two and a half pieces to that. So when we talk about workflows in KNIME, let's first start and talk about nodes, nodes perform tasks very much like the group by node here. So this is really the atomic operation that is executed. This task, the task receive input data at input ports and return the results at output ports. I'll come back to that in a second to introduce that a bit more. But the idea here now really is that you're from repository of nodes, really take the nodes and to connect the nodes output ports to input ports to model data flow in your analysis and build workflows like we see here right at the center. So basically, for instance, connect output port of an Excel reader to the input port of the, let's do this one, to the input port of the statistics node down here, but that we tell the statistics node, hey, do your statistics on the data that comes from the Excel reader that has previously been read into from an Excel file into KNIME, basically. And that really, you do one node after another and you can build your workflows with that. One node after another is only half of the story because there's actually a third layer and you can take a couple of components and you can modularize things. You can put them together into what we, you can encapsulate functionality into so-called components. And they again, they look very much like the nodes at the top here, but inside of this component really runs a mini workflow and you can build those components yourself, modularize your own workflows. But I mentioned starting from scratch and one node after another, it's not necessarily one native node after another, it could also be one component after another, reusing functionality that is available in form of this component provided by us or potentially also provided by the community. So this really is, well, it's an artificial workflow where it doesn't really have a lot of meaning but it shows the entire breadth of the data science lifecycle that I mentioned before. So it's not only about images although we'll talk about that a lot today. You cannot only get data from Excel but you can also connect to a variety of databases. You can connect to APIs and get your data from that, transform your data, do analysis as mentioned, classification here, decision tree for instance, really build models based on your data, a visualization piece, the last button at least to provide for instance, right back into an Excel file the results or you can also really generate reports in form of PDF or PowerPoint that contain the data or the results from your analysis, basically. So that's that. Now in this platform, I won't bore you a lot more with slides, only two more and then I'll jump over into a live demo and show you actually how it looks like. So there's one thing that you'll notice in case you have seen that we put some instructions in the announcement of the webinar. If you want to follow along, click along a bit and you've already went ahead, downloaded nine, installed it. That is actually the very first thing that you'll see is on screen here. Now you will be asked for a workspace to select. This workspace really is a folder or directory on your hard disk that will contain the workflows that you're working on and it can also potentially store the data that you're working on, but we'll get back to that in a second. If you want to follow along, you can basically point it to any directory, although I would suggest create a new one and select it, sorry about that and select it here in the dialogue. And with that, let me jump over, stop the sharing and show you how nine looks like. So we have actually done a test before, but if you can't read anything, please also put that in the Q&A section. I am able to zoom in if something really urgently comes up. So this is what we call the work bench of the nine analytics platform. This is what it will look like once you started nine the first time. We'll have a welcome screen here that helps you get started, gives you some tips and tricks, not super important, but if you're starting out, it's actually, it's a nice gesture. We do have a couple of other views located around here in the work bench. I will walk you through those quickly just to give you an idea. So here on the left top hand side we do have the nine explorer and that actually really is a view on the local workspace that I've mentioned before really your folder with your workflows and potentially also your data. So there's a file and folder structure you very likely know that from the Windows Explorer, from the Finder and Neco S and so on. So I have a folder for the new BIOS Academy for instance in here. We do have the node repository down here which basically shows you all the nodes that are available for you to use in a workflow in your local installation. We're likely if you install nine for the first time you've only installed a very minimal installation. So this, the nodes available in the node repository here change whenever you install new extensions, they add new functionality, they add new nodes and they will show up in the node repository down here. We have the console for error messages and so on. And over here, we actually have a description view that if we select a node in the node repository, for example, the Excel reader, it will really give you an overview. What does this node actually do? We'll talk about configurations options and we'll show you that in a second and scroll to the very bottom. It will also talk about output ports and input ports which data is required and what are the results which data comes back. So all that is quite abstract. I would suggest that I'll just go ahead and start a new workflow, create a new workflow and right click on the folder here, say new nine workflow. And we'll call that new bias demo and click the finish here. And then you'll notice that the welcome screen actually disappears and we now have an empty canvas here right at the center of our screen. This is actually the workflow editor where you put nodes where you build your analysis workflows. So the idea here really is a lot in nine is about dragging and dropping. Bear with me, it's actually quite nice. It's not meant negatively. It's actually really if I want to add a read data from a file, for instance, I would say here in the node repository, I would take a look at that, okay, there's a category I own. I'm interested in reading data. I'll open the read category and in here, for instance, I will find a CSV reader. I can take this CSV reader, I can first read, hey, what does it actually do over here in the description? Is that what I want? If yes, you can take it and drag and drop this CSV reader onto our workflow as a starting point for our workflow with the data that we want to read in. One thing that you'll immediately notice is that they're down here. There actually is a warning sign that you have over that says, please specify a file, which makes sense. I've just dragged and dropped the note. The CSV reader doesn't know anything which CCD file should it read if they, I don't know which column delimiter to use and all this configuration settings. So basically in the background, we see and zoom in a bit to make that a bit clearer. You see that there's a traffic light that's currently on red. This basically means the note doesn't have, doesn't know about all those parameters that it needs for processing. How can we provide those parameters? We can again, talking drag and drop, right click on the CSV reader, the context menu opens up and right in here, we see the configuration dialogue as the very first to the configure entry, the very first entry in the context menu. We can click that and give it a second and then it will actually open up the configuration dialogue. Once you've used them a little, you will see reoccurring schemes here that parts of this configuration dialogue, they are reused in other notes. For instance, you want to select a file that is reused in a couple of other notes, not yet all of them though. So that will be the very first thing that we'd have to point to say, okay, where do we want to read data from? From our local file system, the hard disk can click the browse button here, can have some demo data lying around, can come on, click on that and import some contractual data. I'll just use that as an example here and click open and it will automatically try to figure out what are the correct settings to configure this CSV reader wherever possible we try to automatically pick some good default parameters. And you'll see a preview of the data down here. Okay, so once we have done that, we've basically provided all the information that the node needs and say, okay. And you'll see that the traffic light jumped from red to yellow, meaning all the parameters are available, but the note hasn't been executed yet. That is, it hasn't done its operation yet. It hasn't read in the data. That actually is an additional step. Right click, open the context menu and say execute. If we do that, you'll see there was very, very briefly visible a percentage at the traffic light jumps from yellow to green, meaning this note has been executed. And how can we confirm that? You can actually, again, open up the context menu and you will find in the context menu at the very bottom where it says down here file table for each and every output port where a note, so there can be multiple output ports as well. This just has one output port. For every output port, you will find an entry in the context menu here that actually allows you to click it, open up and get a preview, zoom in a bit, and get a preview of the data. And so one point here is, you can take a look at all the intermediate data while building your workflow. You don't have to re-execute everything from the very beginning. Can really take a look at that at every note. And one thing that I want to point out in NIME, we usually work with tabular data. That is very much like, you know, from Excel, CSV files in general, we have rows. They have unique identifier, row zero, row one, row two, and so on. We do have columns in here. Columns have a name and columns actually also have a type, integers, integer, integer, string. We support double types, but we also support image types, for instance. But the thinking is we are thinking in tables, basically. So that is very first and important take home message. If that is right clicking and opening up this output view, is, I mean, it's a lot of clicks to just see the intermediate results. For that, there's actually an additional view down here that you should see automatically, the note monitor, which is great because you can actually, it shows you by default for the very first output port. It always shows you the output data as well. So as you add multiple notes to your workflow, to build out your workflow, you can really click, click, click and take a look at the intermediate results. So why don't we do that? Let's take a brief look at the data here and say we do have an account length here and let's try to transform our data a bit and filter out the rows, all the rows that have an account length that let's say less than a hundred, for instance. How would we do that? There's a node for that. You actually don't know yet how it's called. So you can either not browse the note repository, the hierarchical view of the note repository or you can use the search functionality here. There's a button actually next to it. I wasn't aware that this is a button until, I don't know, I'd use now for a couple of years, honestly, this enables a fuzzy search, which in the very beginning, if you don't know what nodes are called, it's actually quite nice. So we are interested in filtering out some data just type filter in here. And we'll actually see, okay, so the very first node that shows up is a row filter. And now we can, the very manual approach is drag and dropping our node filter in here, our row filter in here, really connecting the output port of the CSV reader by drag and dropping the output port to the input port of the row filter, saying, okay, we want to filter data row-wise from the CSV reader. Again, red traffic light, right click context menu to the configuration dialogue because we have to define what we actually want to filter out. So what do we want to do? We want to, let's say we want to exclude rows by a specific value. What do we actually want to filter for? We wanted to say, okay, we wanted to filter the given account length. And now we have multiple options in here and we see, ah, there's a range checking. We can activate, we can define a lower and an upper bound in here and say, okay, we want to exclude all the rows with an upper bound of 100 with entries of less than 100 in the account length column. Say, okay, now it's not executed yet, can right click and execute. And already down here in the node miniaturial open up the regular view, it's easier to see there. We see that there were a couple of accounts with them length and less than 100. Now they're not in here anymore. So we basically filtered out the data and really have modeled the data flow by connecting input ports, I'm sorry, output ports to input ports of other nodes. Two things that I want to point out here sometimes it's, when you start out, it's very tough to know how to continue actually. What is the next node that I want to do? But very often you'll see, and also that's not only specific to nine, but also I don't know if you use imageJ for instance or cell profile. You do very similar things very often when you do your analysis basically. The good thing about that is we do have a workflow coach that you can see here on the left hand side that actually suggests you based on input from the community, what would be the next row, sorry, the next node? What does the community most often, which node does it put after a row filter? So very often that will be in 12%, that is a group by node that if we have time, we'll come back to that later today. But the workflow coach with the idea is that you're depending on what you select, it's context sensitive, that it actually shows you suggestions how you could continue, how people very often continue. And like 10% is actually, well, maybe not that significant, but there are cases where people in 80% of the cases, they just use one node because that's what you very often do. That's actually also really helpful as you learn about nine, but as you also don't know what nodes are called yet. Good. I think there's the second thing that I want to mention actually, if you find all that a bit cumbersome, it's very educational today. So I'm showing you all the clicks, mentioning all the clicks explicitly. If you're keen, there's shortcuts for a lot of things as in keyboard shortcuts, but there's also small shortcuts that make your life a little easier. For instance, you don't have to create a CSV reader manually, but you can actually, from the NIME Explorer, you can actually, I mentioned before that it can also contain data. We have some data in here. You actually take that and drag and drop it in here and it will automatically create a correct CSV reader node up here, and it will also auto-configure the node to the extent possible. And this also doesn't mean that you have to import all your data into your local workspace. It also means that, oh, let's see, second screen, let's hope this works. I do have a CSV file here that I drag and drop from my finder that might be a Windows Explorer, for instance. I can take that and really drag and drop that in here. It points to the directory and the file on your hard disk and it automatically created the node which makes things a little easier to get started. Good. I think for now, I should have covered the very basics. I've talked about nodes that nodes can be in different states as indicated by the traffic light below each node. Usually they're not configured. When you add them to a workflow, you go in, right-click, configure to set the parameters, jumps to yellow, and only when you execute a node you can jump to green and only then is the data at the output port available as well. You drag and drop from output ports to input ports to model the data flow in a workflow. And also implicitly, that's actually quite nice. And I should mention that implicitly by modeling the data flow, you're also modeling dependencies between operations. That is, you don't have to go in and as your workflow grows bigger, and Jan will show that in a second, that you add more and more nodes. You don't have to specifically execute them one by one, but you can actually reset a node to its configuration stage. I'm warned that I really want to do that. Yes, I want to do that. And you'll see now if we set the CSV reader node, it has automatically reset the raw filter node here. So there is a dependency. The CSV reader node knows all the downstream, all the downstream nodes that we are basically are dependent on its output data. What that also means on the other hand is if we want to just execute this raw filter here, right click, execute, really executes all the nodes upstream until it has reached a point where the data that is required is available. So you can really, at the very end of your work, so very one of the last nodes you work for, you can try to execute that and it will execute all the previous nodes. Good, let me briefly jump back to the PowerPoint presentation that is- Maybe if I can interrupt quickly, sorry. There was just a question coming in how to do about reading multiple CSV files if you need like multiple CSV reader nodes or how you can do that. I think we can quickly answer this at least theoretically. So theoretically and practically, that means you see up here the setting where it says I can switch the mode of such a reader from file to files in folder and then I actually don't point it to a file anymore but the filter options show up here and here I can define which files I actually want to read filter by file extension by some regular expression or wild expression that is matched to the file name basically. And then it reads in all the CSV files. Practically beware your CSV files should have the same structure as in ideally the same columns, then this works perfectly well, no problem here at all. If there's changes in columns, it might become a little more tricky and then you might have to reward to multiple CSV readers, which is also possible. And maybe just to add to that, there's also the advanced option of using loops to re-execute the same CSV reader with different parameters again and again, for example. Yes, jumping ahead, but very far away here. Exactly, but yes, that's true, thank you Jan. Good, let's briefly take a look. So you'll receive all the slides and have a couple of backup slides in here if you're curious explaining all the things that I've just shown to you live basically about the node repository, how to insert and connect nodes, which status is a node can be in configuration and so on. There is one thing that I briefly wanted to mention and I will actually jump to the presentation mode here is as you're starting out with NIME, I mean, enable the fuzzy search in the node repository one suggestion. But the great thing about NIME at the community also is that very often you don't have to start from scratch that is there's a lot of examples that you can build on. How can you access those? There is the NIME Hub at hub.nime.com and which really is a search engine for workflow examples for components that you can reuse modularized subworkflows basically in form of components. You can also learn about nodes and it's pretty interactive. You can drag and drop things from the browser into the analytics platform if you want to add a node. You can import workflows by just copying a URL and pasting it into NIME. So you really barely have to start from scratch and really take a look at the hub and look if someone has actually had a similar problem or it's just an example of workflow around. Good. There's no immediate questions. I will briefly zoom out a bit, take a look at the higher level again. What you see now is the NIME analytics platform and we don't only build functionality for NIME ourselves as in NIME, it is open source and we're really open by nature. Well, not really by nature, but you get the point. The idea is that people really can contribute, they can extend the functionality of NIME and that might be us extending the platform ourselves. But down here, you also see that there are community extension who also have a network of partners and that can also contribute and do contribute extensions adding functionality. And let's spare out the NIME server part here. I want to use this as a segue because I mentioned before that you can also do image processing with NIME and NIME image processing actually is such a community extension that is contributed by the community. So what does the NIME image processing extension actually do? It adds processing and analysis capabilities to NIME itself. Again, it's just a generic platform basically and the capabilities come in form of notes like I mean show the CSV reader for instance there's also an image reader that's added with the image processing extension but it also adds support for different columns types I shown previously integers, number, integers, strings and so on, but there's also support for images, labeling images and so on. And what that in the end actually means is that you can really build end to end pipelines. You're going in with an image to point NIME at your image files and maybe some experimental metadata in a CSV file do your image processing and NIME built your pipeline down here you extract some features and notes very hard to see but Jan will show a good example of that, walk you through it and then you can again use regular or generic nine functionality building a model from the data that you have extracted from image or I don't know doing an interactive visualization because it doesn't really matter to the visualization where the data comes from, it doesn't really matter if it comes from images or CSV files and the great thing about that really is you build a workflow once and ideally if your images are structured the same same number of channels and so on you pointed at new images, rerun the pipeline and just get out the results so you actually get batch processing almost for free not entirely for free, but almost for free. So NIME image processing is again we're really standing on the shoulders of giants here I already mentioned image J really a lot by means processing we have been an active member of the community we're using functionality from side job offerings and from input to from image J image J to and so on so NIME always is a lot about integration and also reusing functionality but then also contributing back if we are using things so behind the scenes this is what it looks like our image data are represented as input to data structures as the same as in image J to for instance a lot of the processing and algorithms uses the image J ops framework but we also add a couple of new algorithms in NIME and on top of that there's an image processing basically the nodes calling out to the to the functionality so we haven't been rented the wheel but we're also trying to reuse functionality which has some amazing perks and benefits that we'll I don't know maybe hear about today but maybe also hear about next week and with that there currently no super urgent question I would actually like to hand over to Janne don't despair you can't screen share just yet because I have to stop first and then you're actually allowed to share your screen and talk about an example Yes, thank you Stefan Let me share the screen one second all right so also thanks from my side to all the organizers for inviting us here and for setting the whole meeting up so yeah I as promised I would like to walk you through a small example case with an actual image processing problem so we selected for you today an image case of a multi-channel fluorescent image where our task is basically to segment the nuclei and then for each nucleus we have a doppy staining in the fourth channel here for each nucleus that we segment we would like to to only measure those cells that are positive for a given marker in our first channel and within these cells that are positive for this marker in the end we would like to count foci in yet another channel in this red foci channel here so we'll just start with a very easy step of segmentation and that probably covers the rest of the remaining time for today and we also prepared a little assignment on a similar dataset for you we'll talk about that later in case you want to hands on to try try it yourself until next week and of course as Stefan already mentioned batch processing comes almost for free in nimes so basically we also want to do this segmentation not only on a single image but of course on multiple images at the same time and then integrate the data from these images so before I go to the to the demo I'll also have the in the slides you will have the summaries for the new node descriptions so similar to the CSV reader we also have image reader nodes in NIME and let me just switch to my NIME desktop my analytics platform right away I will also just right click and create a new NIME workflow here I'll call it new BS image segmentation and if you look at the node repository down here I can make it a little bit larger I can look directly for an image reader and you see that there's multiple image reader flavors available here so that's one way to get the reader in of course the same with CSV files we can also do with any images that are compatible with bioformats or that you can open in Fiji and image j we basically can use the same read all the same formats because we are lying on the same underlying library the CIFIO and bioformats so if we open for example an lsm file here I can just take that file drag it onto my workflow and then I get such an image reader here and this image reader is already pre-configured to read exactly that image file oh sorry it opens also I can just cancel it as it is and right click and choose either execute or execute and open views if I execute this it will basically now perform the image reading which can take a long time if it's a large image and also if it's multiple images of course but it's reasonable well depending on the image size of course and now there so as usual for any node with an output port you will have the output down here at the images entry but in addition to that you will also have a second specialized viewer called the image viewer that allows you to actually watch or look at multi-channel images so I will just take this image viewer and unfortunately with 16-bit images that's a bit of an annoying issue with the overview here you often see only black squares that's one of the annoying things that in the community project time image processing no one has taken care of yet but you can just click into any of these thumbnails for 8-bit images there usually the preview works nice but for 16 images sometimes you see only black and now when you click on this thumbnail you see the preview and you see the dimensionalities here listed we have an x and y dimension so that's basically just a 2D image but it contains multiple channels we have channel 1 I can scroll through the channels so this one the first one is the one with our marker that can be positive or rather negative in the cells and then we have the second channel sorry the second channel has the foci that we want to quantify finally the third channel we ignore for our example today and the fourth channel is the one that we use for segmenting the nuclei also you can use in this image viewer you can use expand table view to browse through the tables we will see that later when we have multiple images so now as I said we don't want to only work on a single image we right away want to do batch processing of course now I dragged it from from my finder window I can also drag it if in case I have the data inside my workspace directory I can also drag the lsm file from here and it would open this image reader and now let's delete all of these I right click and delete of course you can also use the delete key for that and I confirm I just wanted to show you yet another way of reading in the images which is this image reader table if you drag this one in you see that it has an input port and an output port and that is why because the image reader doesn't take a given image as configuration but it reads the file paths to open from the from the input port so that's a file names port in here and then it will perform the image reading and give you the images that are being read so in order to have a suitable input for this image reader we need to somehow list the files in a folder so I'll just search for files and voila you have the list files folders node I hope you can more or less see it and I'll drag this one just in front here and I can right click and configure or that's the first shortcut just double clicking on this one we'll always open the configure dialog that's usually what I do and now we can choose to either give it a full file path in the local file system or if you have it in the workspace you can also have a relative to the current workflow let's say so then I would just create a path that is relative to the location where I've saved my workflow and now let me browse this it starts up in the workflow folder I go to the data file and on Mac I actually have to check the folder and then say choose and it would just give me the path relative to the workflow so just one directory up and data now it also tells me immediately that in this folder I have three files I can also filter for example for the file extension so you have these filter options here if you have more than just the images in the folder or some other metadata files you can filter that for them and then finally you click okay it switches to the yellow light as Stefan explained already and now if I right click execute I can check here in I know it's not in the node monitor here this one doesn't have them if I right click and go to the last entry I have the file paths here so now this column type is rather new it was I think introduced with 9.4 this is a special path type which is not yet supported by the image processing nodes so we basically have to just convert this path to a normal text string so that's another node that I still need now on the list files node I would have a path and there it is path to string that's what I want to have I take the opportunity to introduce another small shortcut when you have a node selected here that is basically active it has this bounding box around it then it's sufficient to just double click on an entry in the node repository and it will place this the next node after this active node and immediately connect it so it saves you a few clicks saves you dragging the node and then connecting this line so the path to string node it says it's not yet configured but it auto guesses the configuration so usually I don't even need to to do anything in the configuration and I can connect it to the image reader table and now I can configure this image reader table did I double click let me right click and configure and now it reads from the only available column named location I can essentially let it be like this in the other tabs you can have the option to also set subsets only if you only want to read for example several frames from a timelapse image acquisition or just a subset of a stack but for now we want to read the entire image for all the lines and now I will execute and then view the image viewer now you see that we actually read in three images represented in three rows and a single column of type image and I click on any of those ones if I just selected the image viewer and I can see here the the image that I see so before already the first one scroll through the channels and if you now look at the table of course it's nice to have a larger screen here but I voluntarily chose the small screen so that you can at least read the text if I click on any of the other columns I can just browse through through the images here and select any of these images and look at all the channels so that's the first step to get all the data in and now the well what we want to do now is segmenting just the channel number four so what we need to do first because we have a multi-channel image and we only want to work on one channel as we basically need to split the channels that's what you also would do in image j for example you would just split split channels so we have similar here we have a splitter and you have to scroll down to get the community nodes image processing where we have image and splitter so that's what I want to use now I can again double click and it will immediately edit to the last active node and now this one you can also read it in the in the node description on the right side it will allow us to split on any dimension of the image if I double click for the configuration dialog you have a few options to choose so you can choose if in case you have multiple image columns you can choose which column to work on you will always see a column creation mode here where you can choose whether you want to create a new output table or whether you want to actually just append a new column or replace the current column you're working on so that can be helpful from time to time and in the dimension selection in the case of the splitter you have to just remember that you have to choose the dimensions that you want to keep together so basically if I want to keep x and y and have a new entry for each channel I just leave the channel unchecked and only x and y which guarantees that I will get images that only have these two dimensions in the output if I click okay, run and well I will right away click execute and open views now to speed up a little bit now you will see that out of the one image per row I got four images with a bit of a cryptic title in here that's due to the underlying library mglip 2 that basically we have the if we have the channel index indices here 0 1 2 and 3 and I can click on any of those images and you see that now they have they're only x y images and don't have any active slider for the channel for example because they're just single channel images all right now we want to go ahead segmenting our fourth channel there's multiple ways of doing this of course we can just in the in this table select the column of interest now just for illustration purposes I would also like to to show you how to filter out a specific column that we don't need anymore I quickly mentioned that we don't require our third channel so I will just look for a column filter in this case if you search for filter you'll also find that and well if I double click again it will add it to my workflow and if I double click on that node I can choose which what configuration I want to have so in here I can choose the third of the four channels and just move it to the left panel in the red box here you have the columns that will be excluded from the output and in the green you will have the included columns so if I leave it just like that and execute this node then you will see in the output we're left with only three columns and now these these column titles if I right click again here they're still rather cryptic so I would like to get meaningful names for our variables of the three columns so basically the first channel I said was the the market channel the second one is our the foci of interest and the now third column which was the fourth channel would be the the doppy channel for the nuclei so I would also like to rename let's see if I find this there's a column rename right here so double clicking this again double clicking on the or right clicking configure on the column rename gives me another configuration dialogue and here I have the the existing column titles I can double click on one of them say change and I want to say a marker I do so for the other two as well I want to change this to the foci channel and again change the last one to the nuclei for example okay so that's just a bit of housekeeping to illustrate what you can do to make your your analysis more well to make it more meaningful in case you you want to read your workflow in a few months from now so you know what what was intended here now let's go to the actual segmentation so for the beginning I would like to just use a threshold a simple global threshold for the intensities as I would also do for example in Fiji when I just threshold the image we of course have threshold here as well in the image processing there's actually multiple thresholding nodes local threshold is also a local pixel neighborhood whereas global threshold is the default threshold that you would also have in image day so globally on the entire image that's why I want to use and double clicking on here we will have an option dialog here maybe I quickly jump to the presentation again so this is what we covered with the output use and splitter so this global thresholding node it creates a binary mask so basically it binarizes the image into white and black or into true and false pixels just by applying the threshold value that you can either define manually or with any of the automated algorithms that you might be familiar from image day as well and the dimension selection here is basically what any automatic calculated threshold is calculated on so if I choose an otsu or yen or whatever threshold it would be selected based on these dimensions so if you have a multi channel and multi time point time lapse image and you select the channels and times as well it would try to guess the threshold on the entire hyper stack basically sorry so we can do that and I can choose from several algorithms for the thresholding methods you might be familiar for many of them already in this case I'll try just the otsu method for example and I will say okay and execute an open views and now I forgot something very important I'm sorry for that as you see here now it applied the threshold to all three columns because I didn't tell it anything different so basically it defaulted to using all three columns as input if I configure this node again you see that there's a second tab here where you can actually select the columns to work on and I can remove all and just say double click on the nuclei or use these arrows to go forth and back and say I only want to segment the nuclei here if I click okay it gives a warning that you reset the node so basically if you remember I had just executed the node so in case you lose anything that was computed beforehand so it warns you that you actually will lose this first execution results so it switches back to the yellow light and you have to re-execute it and now it should work with only one and because I left it on the new table mode so it just creates a new table with only the results and we'll show you still in a minute how to join these results together with the original input data that we had okay so now if you see here that's of course far from a perfect segmentation you see these little dots and so on and so forth so it's very pixelated I guess most of you will immediately suggest to do some smoothening before doing the threshold so of course we can just apply now a Gaussian filter I look for Gaussian and if you scroll down to the image processing you have this one Gaussian convolution so that's basically the Gauss filter and now another nice functionality of NIME is that you can just drag a node onto an existing connection and it will introduce this node in between the two connected nodes so if I drop it here it's of course still asks for confirmation you are altering the existing connection between two nodes so are you sure that you want to do this? Of course we are sure and also the previous, the later node here it was executed already so it will be reset I have to confirm this as well if you don't like these messages you can of course select to never be shown them to them again so now we have introduced the Gaussian convolution if I look at the configure node here this node allows you to select several sigmas here for each selected dimension so for example for x and y we could actually if you want we can choose two different sigma radii for the Gaussian convolution or I can just put a single number and this single number will be applied on all the dimensions here if I apply, if I have 3D data that's another very nice thing in my opinion about KNIME that it really works well with 2D and 3D with almost no changes so I could also select xyz here it would not care if it doesn't have that in my images currently but if I switch at some point to 3D data it would also do the Gaussian convolution on 3D and of course if you have anisotropic data you could also do something like 552 to account for different set scaling for example but for now let's leave it to xy and convolution of 5 and of course I again missed that I wanted to only select part of our columns just the nucleus column and let's have a look at the result so this one is just the blurred version of the of the nuclei now oh another point here if you're short on screen state here we can double click on the tab and you can temporarily switch to full screen mode of that tab to have an overview of the entire workflow if you double click again here you're back to the original layout of the panels okay now we re-execute the global thresholder and we don't have to execute every node one by one of course as stefa mentioned I can just go to the last node execute it and all the dependent nodes will be executed first so now this looks already a little bit better I have at least smooth nuclei all good I have a binary binary image now to actually get a segmentation we would like to have something like labels for every unique cell so in image day you would do that with analyzed particles for example to get a measurement for every cell in other frameworks such as matlab and python this is usually called connected component analysis and that's what I call it what it's called here as well so connected component analysis down here in the image processing tab so we can run this one on the binary image I will double click to add it to the workflow and double click to open the configuration dialog here I actually can leave everything with the default settings there's options to to switch between the four or eight connectivity in two dimensions and respectively on the other dimensions if you select more here and of course you can you always have this recurring column selection dialog which I don't need to change because we only have the nuclei anymore here so I click okay and execute and open views and now you will see nice colorful segmentation images and if I click on these ones so you see they have multiple colors and in the in the table view you might notice that they have a specific column type called segmentation seg here so that's a specific type a labeling type that is brought in by the mglip2 library and it's actually a very flexible image segmentation type if you look at the well maybe I can point to it so if you look at this part of the screen here where it says value equals when you're in the background it's an empty label and when you're hovering over any of the cells you get the actual number of this label and the colors are just randomized so basically you can have here in the label options if you expand this here you can randomize the label colors into different colors in case you need to see some more contrast between some very similar colored cells so just for visualization okay let's quickly switch to the can I briefly interrupt you while you're switching over Jan had actually an interesting question he asks about duplication of images that in image j you always have to create a duplicate if you want to keep the the original image around do we have to do this in NIME as well or how does it work actually that's a good point so usually by default if you have this option of column creation type new table it will actually do this duplication in the most cases so for example yeah in this case every node would duplicate the image or just compute the result without changing the input image that's important when you work with very large data or large images and have a workflow with a lot of nodes chained after each other because once you want to for example save the workflow in an executed state with all the data it can lead to quite long saving times so you have to be aware of this on the other hand you can also change here replace which would however it would replace the column in the in the output table but not change anything on the other nodes for example if you branch in the workflow or if you're anything upstream of the workflow would not be changed so there's no chance to mess anything up by by changing the image by mistake maybe Stefan you want to add anything no actually not that was what was a good explanation so it is a bit I think I personally find it a bit easier in NIME if I just append things and end up with a usually end up with a column with a sorry with a table with a lot of columns but also to half answer Daniel's question about showing an image together with its split version and so you can select the append option and then you will basically the results will be added in a new columns you'll end up with a pretty wide table but it's good to to have an overview yep that was it yes thanks um yeah maybe also to mention so of course there's also ways to avoid that you duplicate and multiply your your amount of data uh so these are maybe a bit more advanced topics and also maybe I should quickly mention uh that's a very useful thing for at least when you work with images uh if you go to the preferences in NIME and general you can select show heap status and if you apply that you will see the amount of memory used by NIME currently in the status bar at the very bottom and I have to say that the memory management of NIME is really very very good so usually you don't have to care about having enough computer memory available because the node outputs when when when they don't fit into memory they are basically written swapped out to disk in the background so there's nothing you have to care about usually all right so we're close to the end of what I wanted to do today so we have created the single segments here there are ways to actually improve the result for example in here you see that some of the labeling still have holes because in the raw data we had low intensities here in the nucleolide or whatever it is so in the inside the nucleus nuclear we could also for example add a fill holes morphological operation we can do so this one if you if you click on the on the node and read the description it says fill black holes in binary images so the place to add this would actually after the global thresholder and before the connected component analysis so we can do this right here I just confirmed the three messages here and re-execute the connected component analysis right away it will execute fill holes and connected component analysis and you see that now we end up with segments without the holes similar sorry I will open this one again similarly you might want to get rid of very small segments this one like these two here so this we can do on several ways but the maybe the easiest here is a labeling filter labeling filter we can yeah if you want to test it I would leave that to you to play around with this for now to end this I would like to show how we can do a simple measurement on the cells so now we only have the the segments inside the table and I basically lost the the raw data already right after the column rename here so the Gaussian convolution I selected to do only on the last channel while in this node output we still have our three channels so if I want to go back and get these data in and join it with the with the segmentation results there's a node for it as well and this is one of the most used nodes when when manipulating data in general the joiner node this one allows you to join two tables I'll drag it here so from two input ports I'll double click to make this a little bit bigger and now I can choose the table I want to have from the input images and the table I want to have from the connected component analysis now of course so if you look at the at the error message here it tells you please define at least one joining method if I configure this node I need to set set something some criteria to join these two tables together maybe let's switch here so the joiner allows you to to join any two tables together of course these tables can contain the same number of rows in case you have unique identifiers for them it's quite easy you can just put them next to each other but you might also join want to join several well different amounts of data to each other so basically if you have a large table with 1000 rows and a small table defining your plate setup and the given given chemicals that you have added for example then you might join the corresponding correct line of this second table to each line of the first table so that's basically how that works there's some illustrations here for example in the left well this is a different non-image related topic here where you want to join columns of data and basically you join by the same criterion the customer key here is a unique number and you can join them together or you can join even on multiple criteria or on different on different columns and there's also different joining options I will probably not detail this now we can also talk about that if necessary in the next next week's seminar so for now I will just add a matching criterion and by default this one and defaults to a row ID for both tables I could also select any of these any of the columns that are in the tables but in our case the row ID is actually what we want because if you look carefully at the at the input table I'll just open this one so this one contains row zero row one row two it can contain actually any string value but the the unique property of this row ID is that they are really always unique values and they are enforced to be unique so we keep these row IDs in the connected component analysis table we always have them in front of each table in our workflow so we can use these row IDs to join together the the two tables so the left table and right table or the top input and the bottom input and in the column selection tab you see now for the top table I can choose which columns to include and for the bottom likewise in our case we can leave it on the default options to just always include all columns we do that execute and in our joint table we now have the images together with the with the segmentation so maybe I take this opportunity to hand back to Stefan so I would like to now share what I've done and well I will also show you to how to reset the workflow because that's usually when you want to save a workflow that you've worked on you it's a good idea with images especially to reset the workflow first and confirm this reset so that all the nodes are in the un-executed state and don't contain any data because if you save your workflow with all the images processed it basically saves a copy of all the output outputs of all the nodes which can be quite a lot on on your disk but of course while working on a workflow you can do that for sharing I chose to reset the workflow and then save and now I can use NIME Hub so I need to first double click to we redirect it to the login page sorry for that oh why am I not sorry I think I'm already in yes so if you have a login you can just connect to the NIME Hub here and share your data with publicly with others or also store data privately in a in a private workspace on the NIME Hub up to a certain extent of data so now I would take my public space and I have a work I created this morning a workflow group Nubias Academy and I will move my workflow into Nubias Academy here and depending on the size of the workflow it takes a little while to upload and it appears here in the NIME Hub we can also maybe Stefan didn't mention it yet I think you have the NIME Hub tab here okay that's funny now it's it crashed my NIME by clicking on it anyhow that might be a max specific problem now or related to my screen setup you can also go with the browser to hub.nime.com and if I now look for Nubias I should already find the workflow I uploaded immediately on the first position because there's not so many Nubias keywords on the hub yet and I can give a hand over to Stefan this URL and he can basically I will stop sharing my screen and I will let Stefan continue unless there's burning questions right now but I think we can wrap up in a few minutes. I don't think there's super urgent questions at the moment thank you very much Jan I'm hopefully already sharing my screen and I already found the workflow that you that you just uploaded and that actually gives me the opportunity to show you a I think pretty cool feature is that I can really go in and first of all on NIME Hub you can you can see a preview I'm getting a lot of information about this workflow but what I actually want to show you is I can take this URL up here do command c copy this and really select a folder in here in my workspace a command v for pasting and I can actually immediately download this in the background that allows me to import this workflow here can open it up and move it around for a second close it first move it into the workflows folder and then we can open it up again and now I actually have the chance let's see if it works execute here you might have I think I had the workflow one folder up actually so the the relative path is actually wrong that is interesting so if you move it to the session one folder it's actually good I had the wrong folder good then let's close it and remove it to this session one folder then we open it up again okay now it finds the files and I can actually go in and execute this entire workflow here so there's one question that I sorry about that I actually said I would answer that life and if there's any meaning in the crossing of the branches here in this particular case for a joiner there is semantics attached to top note and sort of top port and bottom port so it matters how you connect it but the crossing in general is just the visual thing thing doesn't really matter for the for the execution um what I'm going to show you now is a bit um a couple of words around best practices and documentation of workflows because that's actually super important and there's a couple of things that we can't change about notes and workflows that is our the notes or the names of notes they are fixed you can't change those but you see that as you add notes and that below here you'll find what we call a label and you can actually change the labels in here can double click and load all images space from folder x for instance so that's the one thing how you can document really assign labels to two notes the second thing is and as you browse through examples on hub for instance also you'll notice that we have this concept of in a full we call a workflow annotation you can right click anywhere open the context menu on the canvas say new workflow annotation it will add that and it basically adds this workflow annotation let's move it over here basically adds this annotation and behind behind the nodes we're currently in editing modes that where that's where the nodes are grayed out and click somewhere here and you'll see this annotation actually is behind the nodes you can also use this to document nodes in here and that's a bit boring already you can edit that you can change the the color of the borders for instance if you want to show multiple parts and highlight multiple parts in your workflow but you can also add descriptive text in here like this is the the segmentation part you can make things bold change font sizes and so on this will also be saved together with the workflow so best practice keep it in mind always document yourself others but also your future self will thank you for it so that's the one side of thing and the second side of things that I wanted to mention is I briefly at the very beginning mentioned components and that you're actually able to well height complexity and encapsulate complexity and nodes into components that again themselves behave like regular nodes that's what I briefly want to show you so for instance here let's select the nodes that are actually involved in the segmentation we can select all of them and do a right click here and say create component yes the nodes will be reset that is perfectly fine I can define a lame segmentation and we'll actually see that this looks again like a node itself has an input port output port name I can set a label and so on I can actually if you want to this is not six or anything you can go in open component open and in a new window it actually to zoom in again a bit that actually opens up this component you can take a look again at the the sub workflow at the nodes that we've wrapped to that we've wrapped together the nice thing one additional nice thing is first you can keep your workflows clean the second thing however also is that you can share components so I can actually share them with myself in my local workspace I can however also I log into the nine hub opens in a different screen the second that will be logged in so now I can actually say component I want to share this component to nine hub to my public space for instance and then you will actually be able to find this image segmentation once the upload is finished you can go to hub.num.com find the image segmentation and component on there and really from the hub drag and drop it into into your workflows and reuse it on in your own workflows let's see if I can I can find it in here there we go don't see my screen I'm very aware of that so there's a image segmentation down here that's the component can add descriptions here as well if I make that a bit smaller you can actually do this you can really there's a note I can here you can take that drag and drop that and reuse that in your workflows as well and the beauty of that is it keeps the connection to the version on the hub whenever I do an update and change things fix some things in there you can also apply that update but you could also say well I don't want those updates anymore or I want to do my own changes they are now read only basically you can go in say okay I want to disconnect the link and now you can change the component yourself again so the really is the components are functionality that you can also reuse we provide components ourselves and you can also put components share them with yourself reuse the functionality there and that I think actually allows me to jump the couple of slides and come to a short conclusion and we do have this session next week but allow me an intermediate conclusion here so nine image processing allows you to build image analysis pipelines built from images until statistics and visualization and we'll cover that part of our data analysis next week and it's actually quite nice because the workflow itself is kind of self documenting you can take a look you see the how the data flows in it and being able to also see intermediate results is actually quite nice while you're building out workflows and it makes debugging a lot easier to be honest and if you've taken care of building a workflow properly you can actually say I built it on one data file for instance build out my workflow and then just feed a couple of images to the analysis workflow and it will process all the images in a batch fashion and I think the nice thing he really is that's an interesting aspect and it's the same thing holds true for recording image in macros whatever you're doing to be able to reproduce things and towards reproducible science and so the ideal is that a workflow is basically it is a script in a sense it defines the pipeline and we make very sure that it will yield the same results when you apply it to the same data even if you have updated nine in between so that's the intermediate conclusion for now young feel free to to jump in if not I would go ahead briefly talk about the the assignment until next week it's really optional feel free to do it will be discussed at the at the next session we do have an entire workflow group that is a workflow that you can start with we've called assignment but also the data that is required to do the assignment the hides behind this link here that should actually maybe someone from the moderators can put that into the chat as well you can download it it is an in our file k&a arts nime archive you can import it into nime take the file back and drop it into your explorer or go the manual route is a file import nime workflow or if you have a nime hub account already you can do as I did before you copy and paste the following URL and just paste the URL into your local workspace and it will automatically download the the assignment so what is the assignment about you will find detailed instructions in the workflow step by step what you're supposed to do the idea is that there is that we have images two channel images multiple images in one in the data folder basically here the idea is to open all the images segment them according to the first channel clean up the the segmentation results with a couple of notes that you've seen today that Jan has shown but also feel free to explore yourself what is available in tint nime hub and also use the segmentation masks to extract the the size the area of cells but also the average the mean intensity from the second channel for each image in each file two channel images first step identify cells and then measure the intensity in the in the second channel we'll come back to this and discuss that take that as a starting point for the session next week I think that was it for now there are maybe a couple of questions in the q and a that we can briefly maybe take a minute or two to answer or also that would maybe provide a nice outlook for the things that we also show next week um do we still have a few minutes yes please go ahead so I think Stefan you had seen the question already um about using stardust or any other deep learning method and it's replacing the current segmentation by this one yeah so the the nice thing about that if I may really is that the component that I've just built I've wrapped the segmentation into one component basically the beauty of that is I can easily replace this component with another component or with another set of nodes basically so let's say there were the stardust component that you could reuse or maybe cell post component for example and you can really just drag and drop that onto the segmentation component that we've just built and it replaces the the current implementation so it's really it's actually amazing how fast you can iterate and try out different segmentation algorithms for instance and that is actually something we'll talk about next week yes and maybe if I may add for the answer to another question so do you need to install use programs on your computer like image j that that is used by NIME and of course if you share workflow you want to make sure it's reproducible so that is the dependent a bit on what tool you're using so in case of image j there is a built-in version of image j with the image integration in NIME that provides the basic functionality but if you point this point this integration to your local fidget installation to also use other plugins which you can do then of course you have to make sure that whoever uses your workflow also has these plugins installed and does the same thing so this is a little bit of something to take care of manually on the other hand for example for the existing python integration this is yeah if there there's now nodes that actually allow you to to take the same conda environment and define it in your workflow so that whoever receives and uses your workflow uses the same conda environment that's very handy in my opinion to have maybe Stefan you want to add something well there I mean it's the same story for python and r or any other external integration we try to not for let's pick python as an example we don't use a re-implementation of python for instance but you actually the since we are using a local installation of python on your machine yes that means you have to install python but that also means that you can use all the libraries I don't know second learn second image if you for instance wanted to you can use all those libraries as you'd use them in a Jupyter notebook for instance or in a regular python script so there's no limitations around that same thing holds true for r but yes the reproducibility there it's a bit harder to make sure that others have the same execution environment basically I think we answered that one life recommended way of doing version control for workflows if you share work when you share workflows on nine hub and you can actually create you'll be asked if you want to overwrite the version or if you want to create a snapshot from that but in general that is something that is less nine analytics platform and more nine servers slash nine hub related because the nine server also for instance it's a central installation of nine that you can also use as a repository basically and there you can also create support snapshots which are basically versions of workflows maybe just to add to that I've seen some people who are at ease with git to just use git to work to manage their workflow repository so basically the work the the folder that contains your workflows on your on your local disk and you can put some ignore rules for the data for example so that works more or less but of course it's you don't easily see the the diff between two versions in this case nine itself has a has a diff feature for different workflow versions but that is also more related to the to the server and hub functionality again yes good and then feedback microscopy very interesting question don't will not go into details if you google for nime and sila s i l a sila is a standard for lap automation we just i and johnny a colleague of mine work together with the folks that build the sila standard for lap automation we build a prototype in nime that actually allows you to get images from a microscope for instance but also sent back result to a sila supported um device also microscope plate reader whatever you publish a blog post about that in january so that should be easily findable google or use your favorite search engine sila in nime um you can read read up on that great i would say we would collect the questions anyhow and put the q and a's on the image s c forum after the next week's seminar so we'll still collect more questions next week good then it looks like we are on end thanks a lot was very very interactive very nice very nice features and please fill out the survey that the doc was sent around and yeah as jan said there will be the question answers uploaded and also the recording of the video at some point on the youtube channel of new buyers so yeah thanks a lot see you soon to a new webinar and jan and stefan if you stay a bit longer that would be great secondly stay safe everyone was a pleasure