 Hello again, hi We have a next session on now. We have Gagan and Simon They're both from the University of Melbourne and they're lucky enough to get to use Python and not awful languages Let's make them feel welcome. Just testing. Can you hear me there? Good good Master, what is the key for a good life? Well, that's easy my disciple be simple be open be practical and live in the present moment But master that sounds like the Zen of Python That is why Python is so good Ladies and gentlemen, thanks for having us here today. My name is Gagan Sharma and My background is in computer science I've been working in the field of neuroscience from last seven odd years and today. I'm here with my colleague Simon Salinas Simon is a mechanical engineer, but much more neuroscientist now Last year. We love the vibes at the kiwi paikan and at Wellington And we thought it would be a great opportunity for us. We can come here and present our work So today we'll start by telling you why we are here and a little bit about our Python journey Then we will introduce our work briefly. We will expose a Nigel image viewer customized to visually check brain scans We will also show you the benefits of navigating image headers with Pi Dicom This is very useful for handling medical images, which contain sensitive data And we will conclude with an example of how a few lines of Python code helps us to manage million dollar clinical trials That will lie on remote processing machines around the world We started coming to paikan with the motivation of learning and presented our work for the first time in Australia earlier this year And now we're very happy to show you an improved version of this presentation with our latest updates We're relatively new in this community. Gagan has been to paikan since 2013 and last year was my first time so we felt like outliers at the beginning, but this Well, since most of you guys are professional programmers, so This community is very welcoming anyway, so we're very excited to share our work with you I'm not a programmer. I barely knew what Linux was when I started working in this field But with time I became comfortable with mad lab and batch scripting and I noticed Gagan was coming back from this type of conference is very excited. So I asked him well why Travel so much when information is abundant these days and you can find pretty much everything on Google He laughed and forced me to join him next conference So I came to my first paikan last year in Australia then in Wellington and then I understood there's some knowledge and inspiration You cannot absorb by googling everything on an isolated corner So this motivated us to start using Python more in our work and implementing the type of tools that we heard about in this type of talks Now Gagan will tell you about the type of work we do and then we will explain how Python makes our life easier when handling medical images I will begin by telling you what we do at our work. This is us at the brain imaging lab we constantly receive images from the local scanners and From the sites around the world and we process this data to enhance its value and output the pretty pictures and data For the high-quality publications and for the clinical translation These are the images with which we work all the time and some of you already know what dichom is Dicom is a standard for handling storing printing and transmitting information in medical imaging The imaging part of it contains an image and information about is called error. It's the imaging of It's a language of the medical imaging equipment It's always worth telling the complex it is in dichom these are the images with a lot of information embedded into a file Dicom has many flavors as ice cream does In some cases inconsistencies in the tags can easily surprise you and it breaks your code Everything in dichom is a tag whether it's a doctor information patient information even the imaging data is referred as a tag Every development has a story and so does this one day We got approached by a clinician at work who needed a 3d data viewer with a very specific needs He wants to read the dichom and the other imaging formats. He wants it to be platform independent He wants to use it as a module in the language of his own choice And he wants the viewer to be quick easy to use etc etc. I Did not say no to him not because I'm an Indian I just thought it's a great opportunity for us to learn how to develop the 3d imaging data view But then why to reinvent the wheel when there are so many imaging viewers already available in the market? Mind it. These are not the good dichom views. These are the views with the great strength and functionalities But in this particular situation, they were not taking the boxes which was required So I start looking for the language so I could use it in my programming And we all know that these are the two dimensions where most of the programming language is full I'm always keen to use Python because I think it's easy to learn and use So I start googling and think hey where the Python will be suitable for my high-performance computing and the Google said yes Only if I can start with the code which is readable really I can barely read my own code Which I wrote three months before and On the top I need to merge it with some free. I mean cool pre-existing compile libraries We thought hang on if it's true for one minute what I can do It means that I can focus on the fun part, which is a science part here And also I can develop an application with less effort more impact without worrying about the nuts and the bolts of GUI So I start looking again. I found my abbe with my nice packages, but maybe I'm lazy. I thought no I can't do this Spartacus can't do that. I don't have that much time So I start looking again. I Found package called pyqt graph some of you may already know about this and the web page caught my attention We thought hang on this particular image here the brain image Looks like what we do all the time and maybe this could be adapted to our needs I start looking into the features. Of course the speed is the one it is very responsive It is portable and easy to install. It's suitable for the science and engineering But that's how my first baby steps started I downloaded it I start reading the example The code is so nicely written easy to follow. I never thought that before and This is all I did. I only wrote 24 lines of code with a teeny bit knowledge of a sci-fi a little bit numpy I injected this 24 lines code into original pyqt graph example This is the original example not mine code So I injected my 24 lines into this and I got this in no time running 3d data view Let's have a look at the 3d data view itself So I'm just gonna bring that Okay, this is the Can you see it? This is on the screen So I want you to focus on the three things first is the contrast level I have seen so many viewers struggling with changing the contrast level of an image so quickly this this tool is very responsive I haven't seen anything in python that responsive in terms of neuroscience and also This is a particular thing where you can view your data set at any arbitrary angle and this is great for science And you can go through the slices using the fast or the slow up and down arrow keys And this was great for our application at that time. Let's get back to our presentation so this is us using our viewer at work and We were able to review 400 patients each having a three different scans at three different time points We were able to review them visually in a very simple loop and we recorded the response of our clinicians Hence it's helping us to review the data for the publications. And then I feel like I'm a Spartacus Not a spy Spartacus I can use my pie so to cut all the difficult code and save the day for my work using all this 3d data view But then well 3d data view was very good It was impressive for the team But then another challenge came came up next month and my personal greed also kicked in I thought why not 40 data view and We just have shown you or we just already showed you how to read a volume But that was only one time point now We need to see the change in that volume over the time and then we can focus on a single region Slice frame to monitor the signal change which represents the blood flow as you can see on on the screen in the video Right now what is happening when the signal is changing the vessels are being bright enough and then they're coming to the normal intensity In trying to solve this challenge We came across a more efficient way to read images for viewing using the power of python's image IO package What image IO did for us it read our data set It shows which is the trickiest part in the 4d data set and it provides us with the numpy arrays and what happens after that All the 16 lines of my previous code got reduced to the two lines Only these two lines which I'm highlighting with a square Of course, I feel like another Roman corrector. Maybe I'm watching too much Netflix these days, but let's have a 40 data of your itself It's showing up. Yeah, it's coming up now So this is the 40 data set on the y-axis you can see that you can go through the slices in one volume And on your x-axis is where you can go through that slicing all the time You can see the signal change here or the graph represents the signal change here Which tells us that at the time point 40 where Simon is pointing around the signal starts changing And now at the time for 20 you can see the vessels are a bit bright enough This is where the signal was maximum and this tells Clinicians that how the blood is flowing in the brain over time. Let's get back to again our presentation This nine term it's not the time for claiming the glory But just to recap that all these sophisticated features were coded by someone else You don't have to be an advanced programmer You know you just use someone's efficient code to create a responsive and a powerful tool so that you can use in your day to Do work and that is what we love in Python We started showing you the veers, but before sure loading the images into the veers We need to show them in a way so that you can navigate easily But sometimes we get this one single folder thousand a couple of thousands of files in it a Complete mess and that is where pie die come another package comes in which makes our life easy now Simon will tell you how this package Help us a lot That's right now. We will show you a great Python tool. We discovered to process medical images These are the raw medical images coming from the scanner and here is where we have some flexibility to use Python for image processing After this we use specialized packages to take care of outputting these beautiful maps Depending of the type of information you want to extract from a brain scan We will show you how we use Python in the processing stage of this pipeline This is what you would normally get from the scanner. It's just that Dataset with disorganized files dumped into a single folder We use pie die come to output a nice folder structure that is that makes sense to scientists Sorted by patient names can date and type of scanner quiet I can mention die come alone is an extensive field that could easily take up a whole presentation What you need to know is that die comes were created with the aim to standardize images coming from different sources And that is done by embedding critical information to the tags These are the headers Gagan was talking about and we need to access these sacks to create organized folder structures We will show you how to navigate these headers in a Pythonic way Now it might seem like a mouthful of code We've got Python on top and bash on the bottom same result for accessing the headers, but Tell me which method you find more intuitive and I know the answer for this is obvious you're Python biased But even if you're in love with bash, you may find Python very convenient as you read the headers and they become variables straight away And there's less fiddling around with bash regular expressions To see how we do this in the command line So we start by importing Pi die come library then we assign We assign a die come image to a variable Then we read that file and now we have access to all of the tags For example DS thought Yep That's what I was referring the pixel dot. I'd selfies represented as a tag in the diet We also have Patient name for example or we can print patient name Scandate and place where it was acquired Again, we find it easier to navigate headers through Python structures as the bash alternative takes a lot more effort and string manipulation to store that information We need to modify these headers because medical data is often regarded as sacred data due to privacy legislations mainly in the first world Some tags of doctor's name patient age hospital address and other private data So it's important to de-identify it properly when sharing it with other sites Here's an example of how we do this on the command line again. So I'm going to de-identify the hospital name with a random variable and I also remove Patient name and now we have cleared those two tags The good thing here is we don't have to go through all of these tags individually, but we can make use of Piedicum libraries to strip them all in one go and also you can you can save the De-identified images without having to override the originals This was just one for one single image But this process can easily be scripted to scan the whole data set and output any folder structure Here's an example of how we run the code for sorting and de-identifying. I'm not going to go through the whole code It's available in bit bucket One thing worth mentioning though is that we were able to improve it after coming to Pycon last year and hearing about a couple of libraries Scan there to go through all files more efficiently and improve reading speed We tried this instead of always to walk and notice that it wasn't working the beginning But that Gaggen contacted the developer and yeah, I mean I contacted developer Ben There were some t-thing issues. We found those once we figure that out that code really become very fast with this new library That's right Then we tried also doc ops to simplify parsing arguments into the script and Found this one was particularly handy because you don't have to write the usage function But it gets written automatically for you as you specify the input arguments We're also having a look at pandas for handling data structures We just presented from the medical imaging perspective But the same concept of navigating headers with Python can be applied to any other imaging field We will finish with a quick example of a Python script that helps us monitor machines across Australia, New Zealand and Taiwan We have around 30 remote remote machines used for automated processing in several clinical trials And it's important to make sure they're running 24-7 With just a few lines of Python code we're able to detect signals coming from each machine and deliver an automated report into our inbox every morning and I know this is probably Very basic for those of you working in IT But we were quite excited to get a daily report telling us if any of the machines were down close to full capacity Or failing to connect to hospital service While we liked most with the simplicity and readability of the code giving us so much power with just a few lines And then after developing the 4d viewer We're able to deploy that very quickly into these machines with no fancy installation And we can check the quality of the data remotely through SSH straight away To summarize The scientific community usually has their own preference on which software packages to use for image processing But it's always possible to use Python to replace or preposess data more effectively Python provides readable code that can be easily adapted for custom image viewing and careful manipulation of sensitive data Lastly, we're very grateful for Python inspiration that allow us to develop three excellent tools So come to this type of conferences and absorb what Google alone cannot provide We would like to conclude by sharing a few lessons We learned from the main aspects of our workflow which we discussed today using the data processing using PyDicom and Image IO, the data visualization using PyQ Regraph and data management using the Python's API Well, there are still times when we have to use some other software packages to support these three very important aspects of our work We all know that it's not easy to change people process and the technology in a very well established workflow But we are getting there at work now We are trying to use every single opportunity where we could use the Python not because Python is a agile Fast and a portable tool, but also we strongly believe that that in these exciting times We can achieve almost everything with Python the good thing is that if you're very specific about your software or software tools You can still use Python because Python has the flexibility to join them together in short Python can be a glue with infinite possibility, but what next? Well in the next coming months we're going to be looking for our for our needs from processing and data management on the NIPI and I learned and XSNAP because these tools have a great functionalities for the kind of workflow we have but more specifically we will be looking at the Wispy for two reasons the first is that it is a very responsive and interactive library which Simon is just trying to show you on this screen and Second is that we can view our data set from any particular angle now right now on the screen is a 3d data set which is volume rendering is there and You can see the blood vessels here on one side the blood vessels are nicely fanning out But not on the other because there is a blocked vessel there What we need to do is that we need to add the fourth dimension here So that we can visualize the change in the blood flow in these vessels over the time Because we personally think that it is a great tool for visualization Education and research. Thank you very much. I like brains. Um, did we have any questions? Thank you great talk This is just a question. Well, um, I travel your question while other people are thinking of serious ones You had a slide and it had some languages on it and at the bottom there was one that's that language that everyone hates What was that for me? That was the machine language That was the machine language, you know the the old and aged machine language. That's what I assume that it's not my slide The source is written. We took from someone else. So yeah, I Know that this question came up in my team also somebody said which is like that language which you hate I said machine language The one which has a low-level writing and all those one zeros in the old and days Did we have other questions? Do you use ipython at all? Have you use ipython all the time without ipython? We can't learn all those functions because the type function gave us accessibility to all those plethora functions and we learned straight away This is the one I'm looking for if you're familiar with a command line is very handy, but otherwise probably the notebook is friendlier Going once going twice So Thank you again. Thank you very much. Thank you