 Hello, this is Sergei and this is debugging Galaxy tutorial. A last section, runtime error. So now we're going to be learning how to fix our runtime error in Galaxy. So sharing my screen and heading over to the tutorial. So the last error happens at runtime. That means that we don't have a failing test. Instead we have a bug report describing the error. Our goal is obviously to fix the error and our steps are going to be as follows. Step one, reproduce the error. Step two, locate the error in the code base. Step three, identify the problem as in what exactly is causing the error. Step four, write a test exposing the bug. The test obviously must fail. Step five, fix the error. Step six, ensure the error no longer occurs. Let's head on to the hands-on section. And part one is reproduce the error. So here is your bug report. In the user menu, clicking the data sets option causes an error message to be displayed on the page. Unquote exception in exposed API method. What do we do? We reproduce the error. So make sure you are in galaxy root, then start your galaxy. So obviously you head into your terminal. I head into my terminal, present working directory. I am in the galaxy root, or rather in the galaxy root of the repo that I need to be running at this moment. And according to the tutorial, I need to start it with run.sh. So run.sh, oops. If you are starting galaxy for the first time, it might take quite a few minutes. If not, then you should be able to see a running galaxy in just a few seconds, waiting. And here we are, starting server, serving on HTTP 127.0.0.1, port 8080. Heading back to the tutorial. So of course the process ID number will be different in your case. But now we can access galaxy. We can access it from the browser at this address. So I will just go ahead and open it in the new tab. And we have a running galaxy. Back to the tutorial to make sure we are following the steps. So to reproduce the error, we need to access the user menu. Now to access the username menu, we need to be logged into galaxy. So I'm going to, again, back to galaxy. I'm going to click the user menu. Aha, and I am logged in. So I will log out. I will pretend it's never happened. Back to galaxy, back to the user menu. The user menu does not exist. So we need to log in. If you don't have an account and most likely you do not on this version, you need to register. So I click log in or register. And at the bottom of the form I see, don't have an account, register here. So I click register here and create a galaxy account. Email address. It really doesn't matter what email address you use for this exercise, as long as it is a valid email address. So we will use, let's see, I think my email address exists in the database. So it won't let me use it. So I'll use something else, better example. Password. I believe the minimum number of characters is six, but they can be any characters. So I'll just do one, one, one, one, one, one, one, one, one. And create. We're in. Now, so we need to access the user menu and do something. Let's double-check. Now the user menu is available to us. Click user, then select data sets from the drop-down menu. You should be able to see the error message displayed on the page, okay? User, data sets, wonderful. We have an error message, unquote, exception in exposed API method. So this is actually very good. We were able to reproduce a bug. This means we are on the way to fix it. This means we know exactly where to start looking for it. In the wild, real world, it is not uncommon that a bug is not easily reproducible. So fixing it is not a trivial task. Heading over to part two, locate the problem. So to figure out what's happening, our best bet is to look at the galaxy log. So let's take a look at the terminal window from which we launched Galaxy. The question is can you see, can we see the error message? Terminal, let's take a look. So this is the Galaxy output log. And if you look towards the bottom of the screen, you'll see these very suspicious lines, which say file, lib slash galaxy slash web app slash and so on and so forth, line 129 in index. Raise exception, this should not happen. Exception, this should not happen. This most likely is exactly what we're looking for. Let's double check the solution in the tutorial. And indeed, you should see something like this and it gives us the summary, well, a screenshot of the lines we just looked at. So what's important is that from this error log, we can easily tell that the error happens in the datasets.py file, online 129 and the specific error message, well, it doesn't tell us much, which is often the case, but not to worry, this is a start. So let's head over to datasets.py. So back to the terminal, I believe I can get out of Galaxy. So I just clicked control C and I need the file, where is it? Here it is, file, lib galaxy web apps, Galaxy API, datasets.py. So lib galaxy web apps, Galaxy API, datasets and that will be line 129 and here it is. Great, raise exception. This should not happen. Back to the tutorial. So the good news is that the error happens inside an if else block, which narrows down our search to line 126. That line evaluates to false, causing the else close to execute. And of course the bad news is that we have no idea what exactly causes it. So once again, if we take a look at the terminal, here it is line 126, which is if stir as bull or string as bull, it takes the argument is trans.app.config.get and in parenthesis it's show datasets, comma true as a string true. If this evaluates to true, then the return value will be something else, which obviously is not happening else. So if this value is not true, as in if this value is false, then the exception this should not happen will be raised. So this is a good start. Time to head to part three, identify the problem. So we will investigate using PDB, which is our trusted Python debugger. PDB is great. It offers a wide range of functionality. It's exceptionally useful for debugging at runtime. I encourage you to read this documentation and let me click the link and show you what it's look like, what it looks like. It's not scary. And again, it's exceptionally useful. Here it is. It's on python.org. You simply head into documentation then the standard library. You simply search for PDB or debugging and you will see the link immediately. And this will take you to the documentation for the PDB module. So the documentation is not long and in fact you will need to read it once it'll take you 20 minutes. And then the debugger commands right here is all you need to navigate code bases of arbitrary complexity. So that's PDB. But let's head back to the tutorial. So the functionality offered by PDB is broad, is impressive, but we're going to use the simplest functionality it offers which is the built-in breakpoint function which simply drops you into PDB and lets you use it as an interactive Python shell. So first of all, let's add a breakpoint statement to our code right above the if else block. Back to the terminal. So here is our if else block. And again, we are in the data sets.py file. And we're going to add breakpoint right here. And again, breakpoint is a Python built-in function which will drop you into PDB as soon as the interpreter, as soon as the execution reaches this line. So we've added breakpoint. Let's double check the solution in the tutorial to make sure that we're doing the right thing. Solution, yes, we did the right thing. Now we need to restart Galaxy. However, here's one thing. To use PDB, we need to enable Galaxy's debug configuration option. In general, it may be a good idea to enable this in your galaxy.yaml configuration file. So what you can do is you can head over to config and there will be a galaxy.yaml.sample file. You can copy that file into galaxy.yaml and simply uncommon the option you want to set and set it accordingly. Of course, for the purposes of this exercise, it's enough to set the Galaxy config debug environment variable when running Galaxy. So what we can do is Galaxy underscore config underscore debug equals one and then run it. So Galaxy config debug equals one and run on a sage. Waiting for Galaxy to be served. And we're good, it's serving. So back to Galaxy. Here's our Galaxy. Just to make sure that it's running correctly. Yes, and we retrace our steps. So we go to user, datasets. Something different is happening. We don't see the error message, but definitely something wrong is happening. Loading datasets, the datasets have not loaded. So what we do is we head over to the terminal from which we launched Galaxy and we take a look at the log or on the terminal. And we see exactly what we were hoping to see, the PDB, the Python debugger prompt. There it is, PDB in parenthesis at the bottom of the terminal. In case the captions in the video might block this, let me just do this so you see this PDB prompt. Heading over to the tutorial to make sure we are doing exactly what the tutorial expects us to do. So at the bottom of the log, you'll see something like this. And yes, that's exactly what we're seeing. We are at the PDB prompt. From here, you can execute Python code interactively. So let's explore what's happening in the next line. Well, let's head out to the terminal. So if you look at the bottom line of the terminal, though the line right above the PDB prompt, it is the line which would have been executed if we had not added the breakpoint. So string as bull in parenthesis trans.app.config.get and arguments. So let's explore, let's see what's trans. Trans is an instance of a Galaxy web transaction. What is trans app? Trans.app is an instance of the universe application class. What is trans app config? Trans app config is an instance of a Galaxy app configuration. Finally, what is get? Well, it's a bound method. So it's a method of the Galaxy app configuration class defined in common configuration mixin. All right, back to the tutorial. So we've done some exploration. We got some context. Again, yes, we could have typed trans.app.config.get. We would have gotten the same result. But what we did, we explored a little bit. Now we know what object and what object is calling what object or we have a much better idea of the code we are going to debug. So that was useful. So let's take a look at the definition of this get method. And again, the method is defined in the common configuration mixin, which is used by galaxy.config.galaxyappconfiguration. How do we know it? Well, we got it from the definition of the, well, not the definition, but from the string representation of the get method right here. So let's head on to libgalaxyconfig init. And I don't want to exit the Python debugger. So I'm going to create a new terminal window. I'll keep it here. Go into the same directory right here. And I will take a look at that config class definition. So libgalaxyconfig init. And in the config, I'm looking for, let's see, once you find it, look for the definition of its get method. So I'm looking for the common configuration mixin. Let's find common configuration. Here is the common configuration mixin. And I need the definition of the get method and not this. Here it is, on line 480. Luckily, it's a very simple method. So it takes key and default. It has two parameters, key and default. And the only thing it does, it redirects this to self.config.dict.get a similar method. So let's see what the tutorial says about it. Once you find it, look at the definition of its get method. When you find it, look at the method signature. What is the meaning of the argument true that is passed to the method in datasets.py on line 127? So datasets.py, it's the file we were discussing when we were looking at the Python debugger. So why don't I open that datasets.py side-by-side with the config? Where does it live? Well, if you look at the PDB output, it lives in, here's the line, lib slash galaxy slash webapp slash galaxy slash API slash datasets.py line 127. That's the line we've already looked at. So let's open it side-by-side. Lib galaxy webapps, again, galaxy, API and datasets.py line 127. Here it is, string is bold. So the question in the tutorial, once again, what is the meaning of the argument true that is passed to the method in datasets.py on line 127? So trans.app.config.get, we pass show datasets as the first argument and true as the second argument. That's on the right screen of line 127. The method, this method get is defined on line 480 on the left screen. So you see the true is in fact the parameter which is called default. And this default is passed on line 482 to self.config.get as this second argument. So heading back to the tutorial, let's open the solution. And this is exactly the code we were looking at in the terminal. So the argument true corresponds to the parameter default. In the methods body, we see that the value will be passed to self.config.get. Now self.config.dict is a Python dictionary and we can discover that by looking at the code for an extra five minutes, you will find it very easily. So it's a Python dictionary and you call its get method. And we know that the get method called on a Python dictionary the second parameter will be a default value that will be returned if the key is not in the dictionary. So back to datasets.line 127. So now we know that true is the default value that will be returned if the key show datasets does not exist in the config dictionary. Take a look at this code in the terminal. So here it is, right side of the screen, line 127. If this key show datasets does not exist in the config.dict dictionary, then this code will return this string true as a default value. So this is something we have already established. Back to the tutorial. So moving on back to PDB. Let's make sure we are in the right terminal. Here is our terminal where in PDB and according to the tutorial, what we'll do is we will try to experiment a bit. Let's call trans app get config without the default argument. So show, what was it? Show datasets, I believe. I will check that it's indeed, yes, show datasets. Aha. So when we try to get the value of the key show datasets and don't submit a default, we get none. And none will be evaluated as false. Well, it means that that key does not exist. Now, what if we execute it, we pass true with a string true as the default and we get true as expected. All right, progress. So now we see that show dataset does not exist and we get the default value, which is the string true. Next step, string to bull or stir to bull. Again, back to the tutorial. And why do we care about string to bull? Because if you look at the very first line above the PDB prompt, you will see that this is the line at fault. So we were looking at the argument pass to string as bull, which is trans.ab.config and so on and so forth. Now let's take a look at how string as bull behaves as a function. So string as, first of all, is it a function? Yes, it is a function. Now, what if we pass it? Let's see what the tutorial wants us to do. The tutorial wants us to pass it the value true. Why are we passing it the value true? Because that is, we have established that the argument we are giving string as bull is trans.ab.config.get show datasets true. All this evaluates to the string true. So we're just simplifying this. So that's what happens. And we pass it and we get false. And this doesn't look right at all. So I think we found it. So exit the debugger to exit the debugger. We type Q, enter. We're back in Galaxy. Control C to exit Galaxy. Back to the tutorial. Head over to the function definition. You can tell from the import statement at the top of the file that it's in LibGalaxy until in it. Let's take a look at the import statement first. So heading to our terminal. We don't need the config anymore. So I'm closing. So now we are in, we are in datasets.by. We are interested in finding where string as bull came from. So go to the top of the file. And there it is. Line 21 from galaxy.util import string as bull. So I'm heading into galaxy.util. I'll just open it as side by side again. Galaxy util string as bull. I don't see a module. So it has to be in the module. In fact, I don't need the left side anymore. So I am in galaxy.util the init module. And I'm looking for the string as bull function definition. And here it is. Line 103 back to the terminal, back to the tutorial. And try to figure out. So head over to the function definition. We have found it. And try to figure out what, if there is an error. Well, let's take a look. So what are we looking for? We just called this function and we passed a string value true. And we got false as the return value which may absolutely no sense. So can we figure out what's going on here? Well, if the argument we passed is in this tuple and the tuple contains true, yes on one, then return true. But we got false. Why false? Can you guess what was the value we passed? Let's take a look at the tutorial. At first glance, everything seems right. It always does. And if you think of the value we're passing true, it is not in the true, yes on one tuple because true starts with an uppercase t. So apparently the developer did not account for upper versus lower case. So most importantly, do not fix this error. First, we're heading into part four. And in part four, we'll write a test exposing the bug. So this code is not covered by a test. Therefore, it is imperative that you write a test to expose this bug so that it doesn't happen again. In this case, we have a simple, completely isolated function. Again, here it is, string is a very simple function. It has zero dependencies, so it's very easy to test. A simple unit test will do. So start with a blank, let's see. Our unit test should go into test slash unit slash util slash testutils dot pi. And we need to start with a simple blank test function. Then in that test function, we will call the function on the test stringable and assert that when you pass in the value true, in an uppercase t, it returns true. Let's not think about what other cases we need to test. Let's just do this. So we are going to head, we're going to open the test util module, which is located in test slash unit slash util slash test utils. And you might recall that the first function test to control characters is from the first part of this tutorial, the test unit fixed the unit test value. So now we're going to add a new unit test. We'll call it, we're testing the stringable function. So we might as well call it test string as bold. And we need to assert that when we call it with, with the argument true, with an uppercase t, it returns true. So this should do it. Back to the tutorial. Run the test, oh, let's run this test. To run this test, as you remember, you need to be in Galaxy root. We are in Galaxy root. You need to have your virtual environment activated. So we will activate it now, dot space, dot then, then activate, and we have the then prefix. And now we use by test to, I suppose, execute the, run the test in the entire module. So by test, test slash unit slash utils, test utils, and it failed. Did it fail? Yes, it did. Good. Now it's time to fix the error. Why is it good? Because this is exactly what we expected from it. So part five, fix the error. Can you think of a simple way to fix the function so that the test passes? Well, let's take a look at the function. Once again, lib, Galaxy, fill in a module, string as bull, here it is. Can you think of a way to test it? Oh, to fix it. Getting back to the tutorial. One way to do it is to simply convert input value to lowercase, and that indeed will do it. So if string dot lower, so what it is going to do, it will take our string argument and it will convert it to lowercase and then test against this tuple of lowercase values. So casing of the argument is no longer relevant. This should pass. Let's double check. Let's run it. Now let's run the test. Uh-huh. And it did not pass. I don't know exactly why it didn't pass because it says that string as bull is not defined. And what we did is we assumed, oops, we should open the test. There it is. So we called it a string as bull. We assumed that it's imported. It's not imported, but the util module on line six is imported. So this should fix it. And now it should pass. Now it passes. And now since we got the wrong error, let us for just a moment unfix the error, this one. So this is what our file looked like before we so cleverly fixed it. Now let's rerun the test. The test fails, but now it fails for the correct reason. Assertion error, assert false. This is what we expected. So now we can go back into the function definition module and fix it once again. So again, we add the dot lower method, save and quit and rerun the test. Now everything passes and everything passes for the correct reason that we fixed it back to the tutorial. So one way to do is to simply convert the input value to lowercase. That's exactly what we did. In fact, that is exactly what the real function does. Check line 966, you'll see almost the exact function named string as well. So line number 966, let's take a quick look. So our function is here on line 103 and line, let's do it this way, 966. 966 and here is string as bull on the right, line 966 in the same file. So the solution is exactly the same, dot lower. This is the real function. Why we didn't alter the real function? Well, because the real function is a utility function that's used all over the galaxy called base. So essentially galaxy would have exploded in our face. It wouldn't just create this little bug, it would not run at all. So run the test, now that you've passed, we did it. Heading into part six, ensure the error no longer occurs. Now that we have the string as bull function covered by test as a bonus, we can safely do some minor effect, right? The test will prevent us from breaking things. Can you simplify the function's code? Let's take a look at the function. It's line 103, I believe, yes. So let's take a look. Again, the function is covered by a test. So now we can play with it. Now we can make it better and not fear that by making it better, we are going to introduce some other bug. Well, let's see, if something return true, else return false. So how can we make it better? Well, there are several ways, but the most obvious way, if you think back to your computer science model one days or to the first book on programming you ever read, you can just do return this. You don't need, if something is true, then return true. And if not, then return false. Just return this there. Much better, elegant, beautiful, simple, a pleasure to read. Back to the tutorial. Is that the solution? Indeed, that is the solution. But one more thing we need to do, we need to save and we need to rerun the test. And of course, something happened. What did that, what did we do? Aha, we very cleverly added a return without thinking about the end of the line. And that was a nif clause. So it ended with a colon. Left the colon and we're going to delete it. Right and quit, quit, rerun the test. Now everything is fine. So again, having a test cover function before you do anything to that function, no matter how true that is, is super useful. And I really made that error now. So back to the tutorial. Finally, head back to your local Galaxy and verify that the runtime error no longer occurs. Most importantly, remove the breakpoint statement you added to the data sets on. Getting to the terminal, the data sets where in Lib Galaxy, web apps, Galaxy, API, data sets. I don't remember where it is, but if I search for breakpoint, I'll find it. There it is. I'm removing it and I'm saving it. Start Galaxy and repeat the steps from the bug report. Well, run.sh waiting for Galaxy to be served on Fort 8080. Holding fingers crossed, almost there, not yet. And it's being served. Heading to Galaxy. Here's Galaxy, user menu, data sets, fingers crossed and it runs. Congratulations. You have fixed the runtime error. And this, by this, you have completed the last section of this tutorial and arguably the most challenging section. Congratulations and thank you for working through this. Stop share, stop recording. Thank you.