 Hello and welcome to this course on program analysis. This is a course at the University of Stuttgart. Usually it's a master level course, but if anyone else wants to just watch this or maybe participate in the course, even if you're not at the University of Stuttgart, you can of course do that. This course is going to be about program analysis. What is that? Well, we'll see in a few minutes, but in essence, it's the technique underlying many of the tools that software developers are using every day, tools like compilers and for example, bug detection tools. In this very first episode of the course, we'll look at some basics. We'll talk about what this course is actually about. And I'll also go through the organization of the course so that you know what to expect. Before doing any of this, let me just briefly introduce myself and my research group. So my name is Michael. I am a professor at University of Stuttgart since about a year now, so since September 2019. And before joining the University of Stuttgart, I've been at a couple of different places. So at some point I started my computer science studies at Theo Dresden, then went to Paris for two years to also study engineering and then joined EPFL at Lausanne for my master thesis. I did my PhD at ETH Zurich in Switzerland and then went across the Atlantic to visit UC Berkeley for a postdoc. I then came back to Germany where I've been an assistant professor at Theo Darmstadt, then went another time to sunny California to join Facebook for a sabbatical. And since about a year, I'm back in Germany as a full professor at the University of Stuttgart. The research group that I'm heading here is called Software Lab. So we are interested in all kinds of tools and techniques to make software better, where better can mean a couple of things. So it can mean that software is becoming more reliable. For example, because you eliminate bugs, it could also mean that software is becoming more efficient because you, for example, identify parts of the program that are not as efficient as they could be or should be. And finally, it can also mean that you're making programs more secure because you eliminate vulnerabilities that attackers could otherwise exploit. To do this, we are working on techniques for program testing and for program analysis, which is pretty much the essence of what this course is also about. And we're also very interested in machine learning where we apply some neural networks and other deep learning techniques, for example, to also analyze software. We always have thesis and job opportunities in my group. So if you're interested in these topics and want to dig a little bit deeper, then feel free to just contact me by sending an email. This very first lecture of the course will consist of three parts. The first one is the one that you're currently seeing where we'll look at what this course is actually about, why this is interesting to you, and also how it can help you even beyond the knowledge gain and the academic credit that you will get for it. Then the second part will be about the organization of the course. So I'll talk about what different parts this course is actually composed of, how the grading works and what to know about these different parts. And then in the third part of this first lecture, we look into some of the basics this course builds on, so things like grammars, abstract syntax trees and control flow graphs, some of which you've probably already seen somewhere, but this is just to make sure that everybody is on the same page. All right, so let's start with the question what this course is actually about. So the title is Program Analysis and this course is about program testing and analysis. If you've ever developed a piece of software and I'm sure you have if you're attending this course, then you have already done some of this while you were developing software. So what you probably know is a manual form of testing and maybe a semi-automated form of testing and also a manual analysis of programs. So what do I mean by that? By manual testing, I mean that you're just running your program, you give it some inputs and see what happens and see if it's behaving the way you expected. And by semi-automated testing, I mean using tools like JUnit or maybe Selenium where you can test particular units of your software or maybe test an application through its user interface. By manual analysis of programs, I mean things like inspecting your code. So not running it, but just looking at the code and trying to understand what it's doing and why it's maybe not behaving the way you want it to behave. And there also mean things like interactive debugging where you're stepping through your program looking at intermediate states to understand why it's doing what it's doing. Now this is not exactly the focus of this course, but it's related. The focus of this course is on automated testing and automated program analysis. So the idea is that not you as a human have to look at and analyze the program, but we're looking at tools and techniques that do this automatically and then understand something about this program, for example, by pinpointing bugs or vulnerabilities. So why do we need all of this? Well, first of all, the main problem is that software has bugs. Every piece of software that is non-trivial and interesting has some bugs. So for example, there was a book by Microsoft a couple of years ago that mentions that there are between 0.5 and 25 bugs per 1,000 lines of code in delivered software. So this is not some prototype that someone has hacked together, but it's actually delivered software that still contains a lot of bugs. Now, these numbers are per 1,000 lines of code. Real programs often have thousands, tens of thousands, hundreds of thousands, sometimes millions of lines of code. So this is actually a pretty high number. Why is that? Well, the problem is that bugs are pretty hard to find. So there has been a study that looks at bugs in the Linux kernel. So a piece of software that many people have looked at, that many people are running and where you would assume that there shouldn't be that many bugs, but even in this pretty solid piece of software, it still takes on average 1.5 years from the point in time where someone introduces a bug to the point in time where this bug is actually found and then it takes again a bit of time until the bug is actually fixed. So bugs are pretty hard to find. So it would be good if you had techniques that help with this task automatically. Finally, all of this would be no problem if bugs would be harmless, but actually bugs can cause pretty serious harm. So here are just three examples, some of which you may already have heard about. The first one is the Ariane 5 space rocket that unfortunately did not really make it into space because due to a software bug, it exploded before it really reached space. In the middle you see a satellite image that shows the Northeast blackout, which was a huge power outage that happened in the United States and Canada a couple of years ago and that left millions of people without electricity. Again, due to a software bug and on the right you see an image about a Thera25 radiation machine. So that's a device that is supposed to emit doses of radiation for medical purposes, but unfortunately because of a software bug, it was emitting too much radiation, which then had unfortunately the opposite effect of what it was intended to do. So it was actually threatening the life of people. So now that you're convinced that bugs are a problem, let's have a look at one possible way to address this problem. And this is program analysis. So program analysis means the automated analysis of the behavior of a program to, for example, find programming errors or to optimize the performance of a problem or to find security vulnerabilities. So the term bug here basically means everything that happens in a program that is not supposed to happen. So it can be just a traditional programming error where the program is simply not doing what it's supposed to do. It could also mean that a program is much slower or much less efficient than it is supposed to be. So that an app, for example, becomes unresponsive or it could also be a security vulnerability that allows an attacker to exploit the program and gain some privileges that that attacker is not supposed to have. Now what does program analysis do to prevent these situations or to at least detect these situations? Well, usually if you have a program then you feed an input into this program and as a result the program behaves in some way and produces some kind of output. Now what you wanna do in program analysis is to gain some additional information about this execution of the program. So for example, you want to see that in this execution some misbehavior was exposed or that some vulnerability was exposed. Ideally you do not just want to do this for one input that leads to one behavior and one output but ideally you wanna do it for multiple inputs maybe even all possible inputs so that you can then reason about all possible behaviors and all possible outputs of this program and in some cases you may be even able to say that some class of bugs may not occur in this program at all. In program analysis there are typically two ways to approach this problem. One is static analysis, that's what you see here on the left and the other one is dynamic analysis, that's what you see on the right. So in static analysis the basic idea is that you're analyzing the source code or byte code or maybe binary of a program but you're not really executing it. Instead the analysis reasons about the code and typically tries to consider all possible inputs that this code could have and that means that it's over approximating all the possible behavior of the program. We'll see in a minute what exactly this over approximation really means but basically it means it's considering more behavior than is actually possible. On the other hand dynamic analysis executes the program and then analyzes a concrete execution of the program and that typically means that it is considering only one input so the current input that is given to the program and that also means that it's under approximating the possible behavior so it's not looking at all the possible behaviors but just the behavior triggered by this one input. To give you a few examples of tools or techniques that you've probably already used. So an example of aesthetic analysis is everything that happens in a compiler. So compiler is for example type checking your code or it's looking for particular kinds of errors and this is aesthetic analysis. The same is true for lint-like tools so lightweight bug detection tools which for example maybe just executed by your IDE and highlight a couple of bugs in your code. As examples of dynamic analysis that you may already have used there are automated testing techniques. So if you have anything that automatically tests an application for example the monkey testing tool for Android and then this is actually a dynamic analysis and as another example there are profilers where you look at an execution and try to understand which parts of the code are fast and which parts of the code are not so fast so that you can then optimize that code. So let's make these concepts a little bit more concrete by looking at some examples. So what you see here is a piece of JavaScript code that calls the math.random API to get a random value in the range between zero and one including zero but excluding one. And then we have this variable called out which eventually will be printed to the console down here and depending on the value that is returned by math.random we may assign a different value to out than yes. So if r the value returned by method random is smaller than 0.5 we will say out will be zero sorry will be no and if r is equal to one then we will assign maybe to out. So now the question is what are the possible outputs of this program and at this point and also other points in this lecture I'll ask you to stop the video for a second and maybe think about it yourself before you continue. So what are actually the possible outputs of this program? So now that you have thought about an answer let's have a look at what a program analysis might say. So one answer a program analysis might give is an over approximation which could say well the behavior could be yes or no or maybe. So it's basically giving all possible outputs that seem to be possible on one of the different paths that you can go through this program and based on the limited knowledge that a program analysis may have about what math.random is actually doing this sounds like a legitimate answer because the program analysis may not know that math.random may not return one so we will never take this branch and this path is actually infeasible and because it does not know about that it believes that out could be maybe and therefore it's giving this over approximate answer of yes, no or maybe. Another possible answer that a program analysis could give is yes. So in this case it would under approximate the possible behavior for example by executing this program once and seeing what happens and it sees oh the output is yes so it's telling you well the possible output is yes. And finally there's a third option and this is what you would like to ideally get from a program analysis and this is what is called sound and complete answer which would be yes and no because these are exactly the outputs that this program may actually produce based on the different path that it may actually take. So given that this path down here is infeasible yes where we do not take this if and no where we actually do take this if are the two outputs that may actually be produced. Now to understand why most program analysis do actually not give a sound and complete answer but either under approximate or over approximate let's have a look at a second example. Again this is a piece of JavaScript code and again the code is calling the math.random function and then it takes the return value obtained by this call, multiplies it by two and then outputs the result of this multiplication. Again the question for you first is what are the possible outputs and again please maybe stop the video for a second and think about it. And then we look at the possible outputs that a program analysis could tell you. So again one way to address this problem would be that a program analysis gives an over approximate answer. For example it could say well the output could be any value. This would be the case if this program analysis does not really know what math.random is returning so it does not know that the values are in this interval and therefore it just says well it's some value multiplied by two so again some value that I don't know anything about so I just can tell you that any value is produced. Again there could also be an under approximate answer which for example could be any value in this range of zero and two. For example the analysis may execute the code once and then see that the output is 1.234 and it will give you this answer as a possible output which of course would be an under approximation because it's missing some other possible outputs. Now could we get a sound and complete answer here? Well in practice that's not really possible because it would mean the analysis has to give you all the possible outputs and in this case there are practically infinitely many values in this range of zero and two that's the analysis that the program may actually output and exploring all these possible outputs and enumerating them is practically impossible. This is actually the case for most real world programs and the reason is not just that there are many different values the reason usually also is that there are many different paths including potentially infinite paths because of loops. Now let's have another look at this question of under approximation and over approximation by looking at things a little bit more abstractly. So let's say we have a program called P and we're given inputs to this program and let's denote an input by I and then what the program will do is to have some behavior which we call P of I. Now let's look at the space of possible behaviors that the program might have. So let's say there's one behavior if we feed input I one into this program let's say there's another behavior if we feed I two into this program and then let's say there's yet another one which is P of I three. Now what you would like to ideally know about is this set of possible behaviors. So this blue circle here shows the set of all possible behaviors and this is what we want from a program analysis ideally. Now there are different reasons why we can't get this and you've seen one we'll see more throughout this lecture and what you typically get in practice is either an under approximation or an over approximation. So an under approximation would mean that we only get a subset of all these possible behaviors. Let's say this red circle here which includes the behavior triggered by input two and input three but not the one by behavior by input I one. So this is an under approximation and this is what you typically get if you're for example, testing your program. So if you test the program by executing it with some finite number of inputs then you do not see all behaviors. And the same is what you get from most dynamic analysis because most dynamic analysis just reason about the execution that they are actually seeing and this means they can only look at a finite number of executions and then typically miss some behavior. So what's the problem here? Well there may be some behavior in here which is one of the possible behaviors but that is not seen by the under approximation. And in practice this means you could have false negatives which for example could mean that you're missing some bug. So if you're testing your program but you've missed a bug during testing this is exactly what is happening. And then the other option not under approximation is over approximation where you consider more inputs than those that are actually possible in the program. And this is what most static analysis are doing. So this sounds like a good idea but of course there's a problem with behaviors like this that are outside of what the program can actually do but that according to some over approximation are possible. And if you have a tool that is based on over approximation then this tool may report false positives which for example, if your tool is a bug finding tool means that you may have some benign warnings that look like maybe there's a bug but then if you look more closely it's actually something that cannot really happen during any of the possible executions of this program. All right, so now we've talked about static analysis and dynamic analysis and the pros and cons that they have. Let's now talk about the third kind of technique that we look at in this lecture and this is test generation. So test generation is something that can be seen as an enabling technique for dynamic analysis because a dynamic analysis requires some input to actually run the program. And what test generation is doing is to automatically create these inputs. So instead of needing a human that is using the piece of software or maybe someone who writes inputs for this program a test generator is actually automatically producing inputs. So there are examples of this on all levels of using a program. So for example, you can think of unit level test generators where an input typically means a sequence of method calls that could for example occur in a J unit test and you can of course generate these inputs and we look at some techniques to do that. You can also think of this at the UI level where you have some application that has a user interface and an input for example is a sequence of events triggered on this user interface. For example, you can think of some Android app where you're automatically clicking on particular locations or swiping and so on and the sequence of these events is also an input that a test generator could produce automatically. And finally, you can also think of other kinds of programs. For example, if you have a compiler then the input to this program is a program itself. So in order to fast test or randomly test a compiler a test generator would automatically produce programs that are then given to the compiler to see if it behaves correctly. Finally, let's come to the question how all of this helps you. So one thing you'll get out of this course is of course some academic credits and some additional knowledge of things you didn't know before. But beyond that, I think there are three very concrete outcomes that help you in your work as a software developer. One is that after having taken this course you'll be able to use program analysis tools which will eventually help you to improve the quality of your code. So you'll know more about what kinds of tools actually exist and how they work under the hood. And this will help you to use the right tool so that you have to spend less time on manual testing and manual program analysis. Second, understanding program analysis actually also makes you a better developer because you will get a better understanding of program behavior in general. If you know how a program analysis is reasoning about code then you also will adapt some of this reasoning to reason about your own code and then prevent some errors that a program analysis may have helped to find. But because you know how the analysis would do it you just avoid this error altogether. And finally, at least some of you will hopefully also create their own tools. You'll do a little bit of this in the project that comes with this course but even after the course some of you may move on to some position where you're actually working on program analysis tools because large companies typically have people who do this to solve some custom problems that only they have or to just advance the state of the art of existing tools. All right, so this is the end of this very first part of the first lecture of this program analysis course. I hope you know what program analysis is. You know why it is important and you hopefully also have an idea of how it can help you in your life as a software developer or maybe as someone who actually develops these analyses. Thank you very much for listening and see you next time.