 So the next section, we're going to take another look at Rapture. You've seen the builder and you've seen how it works, but we're going to take a look under the hood and see how it works on the inside. And that will help a lot when you get stuck, when it's like, uh-oh, child process executed abnormally, what the heck do I do, right? So we're going to go into the next section and kind of show you how it works and how to debug it. And in order to explain it all, I'm going to work through a simple example. I'll call this the Fermi function tool. Basically it evaluates the Fermi Dirac factor of order one-half, which needs two parameters. It needs a number, which is the ambient temperature, you know, degrees Kelvin or Fahrenheit or whatever, and also a Fermi level, which is the Fermi Dirac, sort of where the Fermi energy sits, and that's in electron volts. That's a number in electron volts. So those are the two inputs, and when you click, it's going to produce a curve as an output like that. And this is one of the examples that comes along with Rapture. And if you looked at the example, you'd be like, how does it do that, right? You put in two numbers, click the button, curve comes out. How does that work? Well, we started with the builder, and we built the skeleton program and the tool.xml file, just like you guys have been doing. And if you look inside the tool.xml program, you can see all of the bits. There's no magic. You can see there's a tool definition, there's an input section, and an output section. The input section has the two numbers that we defined, and the output section has the curve that we defined, too. Alright, so all the stuff is in there and all explained. Everything tells you everything you need to know about the tool. So how does it work? When you type Rapture, Rapture looks for a file called tool.xml by default in the current directory, and it reads that file and it generates the interface. You told Rapture there's two numbers, it draws the controls for two numbers. If you told it it has units of Kelvin, it draws a little thermometer there. And it does all that, just based on your description. So the tool.xml gets you started. Then when you click simulate, Rapture realizes, oh, okay, I got to run this other program to get the output. So what it does is it produces what we call a driver file. And it gives the driver file a weird number, 1827 or 6598 or some weird number just so it doesn't collide with any other file on the system. In case two people are running at the same time, you both produce a driver file, they won't overwrite each other because they all have weird names, right? So Rapture produces this thing we call the driver file. And if we take a quick look inside the driver file, and you put it up side by side next to the original tool.xml file, it's almost exactly the same. Same stuff, except over here on the right, you can see there are a couple of things that got added. So in the tool.xml, if you do a diff, you'll see that the original tool.xml had a number with a default value and another number with a default value. But the driver file adds a current value. The current value is the value that you want for the run that you're doing right now. So every time a driver file gets generated, it takes a copy of the tool.xml file and it sticks in current values for each of the controls. So in this case, we're running a simulation where the current value is 77 Kelvin for the temperature and 200 milliolectron volts for the Fermi energy. Those are the current values, those are the values for the run, okay? All right, so I've got a driver file now, just like my tool.xml, that gets fed into the program that you're actually running, your program that you're gonna write. And your program will generate another file, we call the run file. And again, the run file has a really weird name, 12703129. Because again, we don't want the run file from one run to overwrite the run file of someone else's run. So we give them all unique weird names. So the run file, let's take a look at that. The run file is just like the driver file. You remember the driver file had the current values in it. The run file also has the current values, but it adds all the outputs. When you looked at your original tool.xml and if you looked in the output section of the tool.xml or the driver, you can find that there is a curve and you can see information about the curve. But there's no value for the curve because we haven't run yet. We don't know what the result should be. So we have a curve here now in the run file that not only tells us all about it, but there's actually a component section with x, y, and there's a bunch of numeric values in there for the run file, right? All right, so your program takes a driver file, reads it, does its work, and then saves out the run file. And then the run file gets loaded up by Rapture into the graphical interface. And Rapture looks for all the results it can find and plots them. The other thing that Rapture does is it normally takes that run file and whisks it away for you into a special directory. I don't know if you noticed, but when you get into a hub, there's always a data directory. And under the data directory, there's a couple of directories, one called results and one called the sessions, right? So, and then under that, there's all these weird number directories. So under data, Rapture, every time you run, will take the result file and store it away into your results folder under whatever the number of the session is. Every time you run, you get a different session number. And there's an environment variable, dollar session, that tells you which session you're in right now. If you echo dollar session, it'll tell you your session number. And that's where your results will go. So every time you're running Rapture, it's storing those run files, the results away over there. And that's really good, right? Because let's say you were doing a bunch of runs one day, and it's like Eureka, I found the cure for cancer, and then your workspace crashes. And you're like, damn it, where's the cure for cancer, right? I know it's in here, I know I simulated it somewhere. Well, if you go back into your data results, you'll find all those run.xml files. And the run.xml files, remember, not only have the results, but they have all the inputs that gave you the results. So you've got every run.xml file stands alone. If you look at it, you'll see all your inputs and you'll see all your outputs. So you know exactly what led to the cure for cancer. All right, now let me put this together one more time. See, I'm like a magician. It's like no magic here, nothing up this sleeve. I'm not even wearing a sleeve, and nothing up this sleeve either. I'm showing it to you one step at a time because when you try to debug these programs, you're gonna wanna go through it the same way. One step at a time, there's no magic. So I've got a tool.xml file. Tool.xml tells me everything about the interface. Tells me about all the inputs and all the outputs, but it doesn't have any values. It doesn't know what the value of the inputs is, and it doesn't know what the value of the outputs are. So when I run Rapture, Rapture reads the tool.xml and it generates an interface and it puts in default values. But it's up to you to change. You could set the temperature to four degrees Kelvin and the Fermi level to one EV or whatever. Set the values however you want. When you click the simulate button, it produces a driver file. The driver file is an exact copy of the tool.xml, but it has all the input values. Know exactly what the temperature is and what the Fermi level is. Then your program reads that, and it uses those values to generate results. Generates a run file. The run file is an exact copy of the driver file. Has all the inputs, but also has the outputs that you generated. So by the time you get all the way around, you started with just the interface, but you added the inputs. And now you have the inputs and the outputs in that file. And that file gets loaded into the graphical interface so that you can see it. And it also gets whisked away off to that directory that I told you about for permanent storage. Now at some point you may bump up against your quota on nano hub. You're like, I run on a quota space. And there's actually a little cleanup function that you can do when you get your quota. It'll tell you, uh-oh, you're out of quota, do you want to clean up? There's a way you can purge files. When you say purge on nano hub, what it does is it goes back and finds the oldest run files and gets rid of them. So as long as you don't have the cure for cancer sitting around in there, you can go ahead and purge those runs. It'll get rid of all those old run.xml files. You probably don't need them. But if you ever did hit the cure for cancer and want to go back to it, you can always go into your data results directories and you can pull out run files and do stuff with them. So, all right, so that's how it works. And the good thing about this is that we can grab, we can get in and we can simulate each piece. You don't have to run Rapture. If you have a driver file, you can debug your program because you can take the driver file, you can get in and run your program in MATLAB or in C language or whatever debugger you want. You can run your program, debug it and then generate the output and check it. So there's nothing magic here. If we look inside that run file again, just to hammer it home, the tool.xml only has the tool definition. The driver file has the tool definition but adds the input values. And the run.xml is the best one, it has everything. Has the tool definition, the inputs and the outputs. Everything you need to know about your run. You can actually reload things later. If you had a particular run file like this that was the cure for cancer and you pulled it back out of your data results directory, you're like, man, that's a really important one, I'm going to keep that one. You can always go back to it later and say rapture-load and give it the name of that run.xml file and it'll reload it. It'll reload it into the GUI so you can see what the input values were and you can see what the output value is too. So you can always reload an old result and take a look at it if you need to. Just by doing that rapture-load. You can even treat the run file as if it were a tool.xml file. You remember normally rapture looks for a file called tool.xml, right? When you just say rapture. But you can say rapture-tool and give it whatever name you want. You want to call your tool fred.xml? You can do that. When you're saving out in the builder, you can call it fred.xml and you can tell rapture-tool fred.xml. In this case, I'm saying rapture-tool and I'm giving it the name of a run file. And it'll look at the run file. Remember the run file has the tool definition in it. It acts like a tool.xml. So it will load that up and it will run the tool and go from there. In that case, when I'm using it like this, it'll ignore the input values and it'll ignore the output values that were already in the run file. They're there, they just get ignored. So you can reload old results and you can sort of treat your run file as a tool if you want to. It's kind of a strange thing to do, but you can. All right, so let's take a closer look at exactly how your program gets invoked. So I've got this driver file and your program's gonna get run and it's gonna produce a run file. And how exactly does all that happen? How does rapture know where your program is or what it's supposed to do? So your code is gonna contain the rapture library in it. And we saw that in the skeleton program, there's some stuff at the top and there's some stuff at the bottom of the skeleton program, right? So that's like this yellow stuff that's kind of wrapped around the important part of your code. So there's some rapture library calls in there. Also, the whole program gets invoked according to the instructions in the tool.xml. If we look inside the tool.xml in the tool section at the top, which we didn't really look at before very carefully, you'll notice there's a command section. And in this case, inside the tool where it says command, you can see this gobbledygook, python at tool, fermi.py at driver, blah, blah, blah, what's all that? That's the instruction that tells rapture how to run your tool. First word says Python, it's gonna run Python. So I know this is a Python program. If it was Fortran or if it was Matlab or something, it would probably be different. But this is basically the Unix command line that rapture needs to run, to run your tool. Now there's some weird stuff, the app tool and the app driver. Rapture expands those things, those are like placeholders. Rapture doesn't exactly, well, you may not know exactly where your tool sits. But if you just say app tool, rapture will figure out where your tool is and automatically put the directory name in there. So wherever you see at tool, rapture says, yeah, where's that tool.xml file? That directory, lab one or wherever you guys put it. And it'll take that directory path and substitute it in place of that tool. So it'll say, okay, I'm gonna invoke Python on slash app slash your tool slash current slash fermi.py, that's the file that I wanna run. It's sitting in the same directory as your tool.xml, right? Wherever that is. And also at driver, you remember I told you at driver, the driver file gets a weird name. First time you run it, it'll be driver 157927.xml. Next time it'll be something else. And you don't have to know because you can just say at driver. If you say at driver, it tells rapture, hey, just put the name of the driver file there on the command line in that spot. And whatever the name of the driver file is, it kind of substitutes the current name and then runs that. So rapture uses that command as like a template. It substitutes in at tool and at driver, it builds a command line. And if you type that command line at your Unix command prompt, it'll run the tool just exactly the same way that rapture would. In this case, it's running Python, it's feeding in the fermi.py script, and it's passing in the driver as an argument to your Python script. So that's it, it's basically a template for the command line that it's gonna invoke at the command prompt. Now, inside your program, in this gobbledygook that rapture generated for you in the builder, it imported rapture, blah, blah, blah. One of the first things it does, it says rapture.library, sys.argv1. Sys.argv1 is the Python way of saying, I wanna get the first argument off the command line. And the first argument off the command line is the name of the driver file. So that's how that gets passed in. If we had made that the second or the third argument on the command line, then we should make it sys.argv2 or three or something, right? So there's a little matching pattern here. Whatever is in the tool.xml for the command line, I'm gonna assume that that argument is gonna get passed in and I'm gonna use it inside my program appropriately. And the builder just generates all this stuff for you. It generates that command instruction and it generates the sys.argv1 line and everything matches up. But if you go mucking with it, you can mess with it and add flags in and stuff like that. Just make sure that if you change the position of the driver file, you need to change that sys.argv1 too. So that's all that's all that's happening is that I'm calling Python and passing in the name of a script, I'm passing in the driver file. The driver file comes in as the first argument to the Python script and then Rapture goes on from there, reads the driver file and it goes from there to figure everything out. Load up all the values and go from there. At the very end of your script, at the very end it calls Rapture.result, the builder put that in again right before it exits. And this is where it writes out the run file. So if you accidentally delete this command or you forget to put it in your program or whatever, you won't generate a run.xml. You have to call that right at the end right before you quit. And it'll generate the run.xml file and then exit. Otherwise your program really didn't do anything, right? So you have to call that at the end. So there you can see the driver file coming in as an argument. You do your work in the middle and then at the very end that's where the run.xml gets generated. And you've got a simulation. Another thing I want to mention is the correspondence between the XML when you're trying to read it and what you call it in your program. So in your program, if we looked in the tool.xml, in the input section, you might for example see this inside input there's a number with ID equals temperature and inside that there's a current value 300 Kelvin. But I'm talking about that inside my program. You can use the syntax instead of all the diamond braces and gobbledygook it's a little simpler. You just say input dot number parentheses temperature dot current. So that whenever you have like input dot number dot current, those are like the tags in the XML. And whenever you put parentheses around something, it's like the ID part. So this is at the top you can see input dot number parentheses temperature dot current or like right here. The way you read that is go into the input section. Find the number that has the name temperature and get me the current value out of it. And that's what Rapture does again in the pre-generated code. It's Rapture generates this skeleton for you because it's looking for that number control for temperature, the number control for the Fermi level. It's getting the current values and going from there. Similarly, in the output section we said in the output there's a curve named F12 and in the component XY section store my XY data. And that if you looked at the XML is going to be the same thing. There's an output tag, there's a curve with the ID F12. There's a component section and inside that there's XY. And inside that we see XY, new line, XY, new line bunch of XY data. So this is how you translate between the XML syntax with all the diamond braces and what you use in your program when you're referring to those same things, sort of a one to one correspondence. And if you're familiar with XML or XPath and stuff like that, it's very similar. We just use a slightly different convention in Rapture. All right, so hopefully now you're all set for the next lab assignment. Congratulations, you've inherited a sort of working but sort of broken program and it's your job to fix it and debug it. So I want you to make a directory now for lab two. And in that directory you can type W get and that big long path. W get HTTP colon slash slash. I hope I did this right, I'm making you all type it. nanohub.org slash infrastructure slash Rapture slash raw dash attachment slash wiki slash boot camp with a capital B and a capital C 2012 slash ex2.tgz. Hooray, I did it right. Again, let me highlight the man that you guys will type. There it is. Okay, then the next thing you do, tar xv zf ex2.tgz. Isn't that funny? Unix is funny, isn't it? It's like alphabet soup. The tar program is the old tape archive program. It's like a zip program, right? Xv zf, x says extract, z says it's compressed, f is the following file, and then ex2.tgz is like, it's like a zip file. It's how you unpack it. And if you say that, then what it will do is unpack, it'll make a directory called ex2 and you'll have a main.tickle and a tool.xml in there. Tar xv zf ex2.tgz and then it'll unpack it. Anybody with me? Anybody able to get that far? Come on, raise your hands and be proud. All right, good. So now, now here's what we're trying to do. So I have this program here that is a tool.xml file, but you notice the command here is empty. It's like botched somehow. So there's a command here that's empty. So if I try to run this, it won't really work right in Rapture. And I want you to walk through the process of messing with it and get a feeling for how it works, and then eventually fix it so that it will work. Eventually I want to say Rapture, and I want to be able to click simulate and make it work. But right now it says, hey, command's empty and it's all goofed up, right? So it's given me problems. So here's the assignment. First of all, I want you to take the tool.xml and I want you to create your own driver file by hand. You remember when you click the simulate button, Rapture creates a driver file for you. You're going to do it by hand. You're going to copy and the Unix command for that CP, copy, copy tool.xml to driver.xml or whatever you want to call it, and then edit it. And inside the driver file, I want you to put in your own current tags. I want you to by hand edit the integer controls and set up a current value for the two integers and a current value for the operation, whether it's add or subtract. So you're going to build a driver file by hand and then I want you to run the program by hand. Tickle shell main.tickle driver.xml. I want you to explore what happens when you run the program by hand. You're going to run this is the program and this is the driver file that you made up by hand and if you did everything right, it will produce a run file, right? Like any good Rapture program. I want you to do that, try it out, put in some values, take a look at the run file that gets generated. Because you're doing this by hand, the run file will be sitting right there when you're done doing it. You won't have to go find it somewhere. It'll be sitting right there. And once you get that far, then think to yourself, all right, here's the command that I'm using to run by hand. Now go back and edit the tool.xml by hand, put that command in. You remember the business with the at tool and the at driver? You got to figure out all that at tool and at driver stuff. Basically take this command string, put it back into the tool.xml and put it in with all the at tool and at driver stuff. You see what I mean? So you're going to do all the steps that Rapture is doing automatically for you. You're going to copy the tool.xml to a driver file. You're going to insert values, current values into the xml. Make sure you put them in the right spot. You're going to run and produce a run file and then you're going to edit the tool.xml so from then on Rapture knows how to do it automatically. And you're going to ask a lot of questions. Okay, so let's go through the solution that we had for the lab assignment here. And you remember, first thing we did was download this tool. We got a tool.xml and a program that you've never seen before and your job was to look at it and try to figure it out and debug it. So it's a tough problem because you didn't write the code, right? But it happens all the time because either the last student that your advisor had wrote the program and they're giving it to you or it's the guy in the cubicle sitting next to you that's like, what's wrong with my program? So you got to figure it out. And the way you figure it out is by going through the same steps that Rapture does. When you bring up the Rapture tool and you click the button to run Rapture, Rapture builds a driver file. It runs your code with the driver file. It produces a run file and then it shows the results. And so the point of this lab exercise was to get you to go through each one of those steps. So we started out by taking the tool.xml file and copying it into a whatever you wanna call it. I called it driver.xml, whatever we wanna call it. And then we edited that driver file by hand and everywhere that we saw a control like an integer and an integer and there was also a choice in there, each one of those had a default value but then we also wanted to go ahead and add the current value for each of those controls. The current value is what Rapture uses to run your program. So we edited the program and made a current value and then ran it by hand. All right, enough talking. Let me show you what we wanna do. So I'll start clean. I've got a main program and a tool.xml. And if I look at the tool.xml, remember I told you it didn't have a command. We'll worry about that in just a minute. But other than that, everywhere in the program where I've got an integer right here, I wanna add a current value. So I have to say current and slash current. Tags have to match in xml. And then I can set this value, not necessarily the default, whatever I want, four. So the current value for that first thing is now four. And then I'll go down and I'll find the next one. Here's another one. I'll set its current value to seven. Okay? Now if you keep going down, you'll see that there's a choice here. We haven't talked much about this yet, but this is how you express a choice in Rapture and there's a couple of options. There's an add option and a subtract option. And you notice that the default is add. So I'm gonna set a current value. And I can set that to either add or subtract. I'll set it to subtract. All right? So those are the three values and for each one of them I set a current value. And I should write that out. This was the tool.xml. I should write it out as driver.xml, right? Oops. Let me get out. Okay, so now I've got this driver file, which is an exact copy. I should have copied it first. And it's got the current values in it. Now I can run my program and feed it in that driver file. And what it's gonna do is my program's gonna read the values from the currents and it's gonna produce a run file. Let's take a look at that run file. First we'll look at it by hand. If I type vi and you remember tab is your friend because it completes as much as it can of the name. All right, so in the run file it looks just like the driver file. Down here in the input section you can see my current value. That's from the driver file. A little bit further you can see the other current value. See? Same stuff. The difference is that the run file also has an output. The run file has not only the thing that's in the output side, the integer, but it actually has a current value for the output. So that's what the run added. And we can view that if I type rapture dash load and then the name of that run file. Again, tabs your friend. So if I say rapture dash load on the run file it shows me that my current values were four and seven and subtract and that the result was minus three. So rapture dash load gives you an easy way to look at a run file and debug it if you've got a run there. All right, so that's all good. So if I were debugging this program if I were trying to figure out what's wrong with my program let's say it's got a bug in it. I'm gonna go ahead and put a bug in it. I'm gonna like mess up my program here. Change, I don't know, delete some characters and save it. Now if I run my program by hand tickleshell main.tickle with my driver file it'll give me an error. And I can see what went wrong. You can see the command right here, dollar IOT output blah blah blah that's the line that I just destroyed, right? So you could get into your program and run it and do it all by hand and figure it all out. We'll be doing more of that as the day goes on. So I'll go back to my program and I'll set it right again. Fix that line and save it out. And I can try it again and now it works. So whenever you're stuck with Rapture it's not like you have to fight with the graphical interface, create yourself a driver file and try it out. By the way, if something goes wrong when you run your program Rapture will automatically leave you a driver file. So again, if I break my program on purpose delete a few characters, save it. If I run Rapture and I click, okay. Oh yeah, first I gotta solve that problem. Then I'll show you that. The same way that you run it by hand you're gonna put the same kind of command in here. So tickle shell is the program I'm gonna run. I wanted to run main.tickle and I wanted to run some driver file. But if I did that, then there are two problems. First of all, when you go to run this program it may or may not be able to find main.tickle. Where the heck is main.tickle? I don't know, right? So we have to tell it where that script is. And second of all, what's the name of the driver file? Well, it's not gonna be, if I just say driver.xml it's only gonna use that one driver file. It's never gonna use the current driver file. So you have to change that. You remember the syntax that we used was if you say at driver it treats it as whatever the name of the driver file is. So it'll do a substitution. And we also say atTool. And that gets substituted with the name of the directory that we're in right now. The name of the directory that contains the tool.xml that gets substituted into atTool. So in general, whenever you're running this program you can always look at the command section in the tool.xml that shows you what Rapture's gonna run. Rapture's gonna run tickle shell. It's gonna look in the tool.xml directory. It's gonna find main.tickle and it's gonna pass in a driver file. That's how it's gonna run. If you were Python, it would start with Python. If it's C language, it's slightly different. When it's Matlab, it starts with Matlab. There's always, so you can always see the program it's trying to run by looking at the command. All right, so now if I run Rapture, it'll almost work. Oh, child process executed abnormally. So when something like that happens you notice there's a bunch of driver stuff here. Let me just clean up and get rid of a bunch of stuff. Clean up and get rid of the run files. I'm nice and clean now. I run Rapture again. I try to simulate child process executed abnormally. But if you look, there's a driver file there. That's the driver file that got generated when the error occurred. And whenever there's an error, Rapture leaves that around for you so that you can debug the problem. If everything works, the driver file goes away because you don't need it. But when there's an error, it sits there and then you can say, all right, what does this tool do? Oh yeah, it runs tickle shell on that program main.tickle using the driver file. So I can do it by hand. I can say tickle shell main.tickle and then give it the name of that driver file. And I find the error. This is how you debug. And then I go in and say, oh, I see main.tickle line 35. There's a problem there. So let me go into main.tickle down at line 35 and fix it. And now if I run Rapture again, it works. And you can try add and subtract and all the different flavors and convince yourself that it works. Now you notice I still got that driver file sitting there from when it failed. So if I don't need that anymore, I can get rid of it. I'll clean it up. Oops, remove the driver file. So now it's all cleaned up and everything's ready to run. Okay.